Ssh_exchange_identification: Connection closed by remote host

[heyitsgary]

Problem: While running remote jobs I receive the following error:

  1:
Error initializing command: RuntimeError - Unable to create directory on remote system /var/tmp/foreman-ssh-cmd-ca5ff35e-3566-459a-adb9-33f3acfc26ed: exit code: 255
   2:
 ssh_exchange_identification: Connection closed by remote host
   3:
Exit status: EXCEPTION

This happens on some servers, from what I can see it may be because their realm is in a different realm than the content server. However, if you continue to run the remote execution job, it may some times work. In the settings I do have fallback to any proxy selected:

I am not seeing any information about the failed jobs on in the production.log file.

I am wondering if these clients are trying to process from a smart proxy but since we only have remote execution installed on the main Foreman server if that is the cause of the issue?

Expected outcome: Jobs work without the above error

Foreman and Proxy versions: 3.1.3

Foreman and Proxy plugin versions:

Distribution and version: Centos 7.9

Other relevant data:

[Wes]

[E] error while initializing command RuntimeError Unable to create directory on remote system /var/tmp/foreman-ssh-cmd-XXXX exit code: 255

There’s Administer > Settings > Remote Execution > Connect by IP. Alternatively you can set remote_execution_connect_by_ip to true as a parameter on host or hostgroup level.

You may be having DNS resolution issues and could try this to troubleshoot that.

Also, the solution to a problem that sounds very much like the one you’re experiencing is here.

All remote execution jobs fail immediately with exception

So basically, there are two solutions to this problem: set ProxyCommand=none in /usr/share/foreman-proxy/.ssh/config for foreman-proxy Put the ProxyCommand into a Match block in /etc/ssh/ssh_config Match exec true ProxyCommand /usr/bin/sss_ssh_knownhostsproxy -p %p %h That means: if it can run /bin/true with an exit code 0 it uses the ProxyCommand. Otherwise not. So that’s a system-wide solution for all users which don’t have a shell set.

[heyitsgary]

Thanks for that. I followed the steps but I still ran into issues. After investigating further I found that many subnets that did not have a proxy listed for remote execution. I added them and now all seems good. Another thing to check!