I am still stuck here. When I am trying to push a package deployment I am getting this error. I have tried to setup remote execution but so far no luck. I am following this link but it’s not working… please help
https://theforeman.org/plugins/foreman_remote_execution/1.7/index.html
Error: Host did not respond within 20 seconds. The task has been cancelled. Is katello-agent installed and goferd running on the Host?
Hi there. If you want to use REX instead of katello-agent (which is recommended), perhaps this blog post can give you a good intro Foreman :: Introduction to the Remote Execution Plugin
Once you have REX up and running, you can simply deploy using REX instead of katello-agent. And as a bonus, you can run other actions through the same stack.
You mentioned REX doesn’t work, do you see any error message?
The task progress at 25% and get stuck in a pending state. I checked /var/log/message but there is nothing there. Is there any specific log file that I should look into?
Click on the dynflow console so we can see what subtask it’s waiting for. If you click on it, we should even see more details. If something hangs, make sure there’s no DNS/firewall issue on your Foreman Proxy machine when it tries to connect to the vm-spacewalk2 machine. For debugging the ssh connection, it’s good to sudo -u foreman-proxy ssh root@vm-spacewalk2, the SSH key is to be found at ˜foreman-proxy/.ssh/
Don’t see much detail in Dynflow that why it is pending…
I did copy over ssh pub keys to the client from the foreman server
ssh-copy-id -i ~foreman-proxy/.ssh/id_rsa_foreman_proxy.pub root@vm-spacewalk02
when trying to ssh to root@vm-spacewalk02 from ssh console of the foreman server it does prompt me for the password
Actually, I am able to successfully ssh to the client from foreman server
[root@fmn01 .ssh]# ssh -i id_rsa_foreman_proxy root@vm-spacewalk02
Web console: https://vm-spacewalk02.***.com:9090/ or https://192.168.1.209:9090/
Last login: Wed Jun 23 17:39:25 2021 from ****
[root@vm-spacewalk02 ~]#Do I need foreman-tasks service? I don’t see it installed on my foreman server and I saw some reference to it on some old pages so I thought it may have been deprecated
Ok, it seems the task is not picked by the the background processing service. It’s good you verified that once it picks it up, it can connect. How did you install the REX plugin? If you run the install with foreman-installer --enable-foreman-plugin-remote-execution --enable-foreman-proxy-plugin-remote-execution-ssh it should configure everything that’s necessary.
There should be following services running
The foreman-tasks service was replaced by these some time ago. Sadly the manual hasn’t been updated for a while, cc @aruzicka (namely Foreman :: Plugin Manuals)
I used the following command to install REX plugin
foreman-installer --enable-foreman-plugin-remote-execution \
--enable-foreman-proxy-plugin-remote-execution-ssh \
--enable-foreman-cli-remote-execution
I do see the following services are running on foreman server:
UNIT LOAD ACTIVE SUB DESCRIPTION
dynflow-sidekiq@orchestrator.service loaded active running Foreman jobs daemon - orchestrator on sidekiq
dynflow-sidekiq@worker-hosts-queue.service loaded active running Foreman jobs daemon - worker-hosts-queue on sidekiq
dynflow-sidekiq@worker.service loaded active running Foreman jobs daemon - worker on sidekiq
foreman-proxy.service loaded active running Foreman Proxy
foreman.service loaded active running Foremanstill no luck 
Could you run a job and give us parts of /var/log/foreman/production.log, /var/log/foreman-proxy/proxy.log and /var/log/foreman-proxy/smart_proxy_dynflow_core.log?
Thanks for your response.
I am attaching the following two log files here.
Log_1.log : I get the following error on the frontend when submitting a package update
Host did not respond within 20 seconds. The task has been cancelled. Is katello-agent installed and goferd running on the Host?
Log_2.log : These logs show when I schedule a remote job to run “ping -c 2 google.com”, the job stayed in pending status on the frontend
The above logs outputs are capture at the run time by the following command
tail -f /var/log/foreman/production.log /var/log/foreman-proxy/proxy.log /var/log/foreman-proxy/smart_proxy_dynflow_core.log
The proxy-side logs are not verbose enough to tell us anything apart from the job actually being delegated there. Could you bump log level to debug in /etc/foreman-proxy/settings.yml and /etc/smart_proxy_dynflow_core/settings.yml, restart foreman-proxy and smart_proxy_dynflow_core services and do the same thing once more? Also output of journalctl -u smart_proxy_dynflow_core could be useful.
attaching two log files one when deploy a package and other one is schedule a remote job
There you have it. Either you’re using ed25519 keys which are unsupported before foreman 2.5 or your private key is in a format which makes foreman think it is an ed25519 key. Is the first line of the key -----BEGIN OPENSSH PRIVATE KEY-----?
Yes, that correct. Here is how the key looks like:
[root@vm-fmn01 .ssh]# pwd
/usr/share/foreman-proxy/.ssh
[root@vm-fmn01 .ssh]# cat id_rsa_foreman_proxy
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
NhAAAAAwEAAQAAAYEAtk0tJhK4/2qVkKt1opwML8vl1B2feYMQRofuuJNpsBtQ79sTCvwB
qNCVDid8SXly6PQUZV+b1vs4if4tYsJXM1MQEQf4uOPXWTi14tC0/cx0s56+BlFVHKUS2S
I think I followed this instruction to setup remote execution, It was not working so I may have tried other links to troubleshoot but this is one of them
https://www.linuxtechi.com/remote-execution-job-option-foreman-dashboard/actually, it was this link initially
https://theforeman.org/2021/02/introduction-to-the-remote-execution-plugin.html
by the way, I am able to ssh to the host from foreman server without a password prompt
[root@vm-fmn01 .ssh]# ssh -i ~foreman-proxy/.ssh/id_rsa_foreman_proxy root@vm-spacewalk02
[root@vm-spacewalk02 ~]#