Problem:
I recently migrated from Satellite 6.7 to Foreman 3.5 and have two problems that appear related, so I am combining them into a single thread.
A few of my hosts are hidden behind a bastion host. To reach them, I created a file /var/lib/foreman-proxy/ssh/config
Match user svc-satellite
IdentityFile /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy
Host bastionhost.mydomain.com
User svc-satellite
Host *.behindbastion.mydomain.com
ProxyJump svc-satellite@bastionhost.mydomain.com
Permissions for this file are
-rw-------. 1 foreman-proxy foreman-proxy 629 Feb 19 16:31 config
(I also tried putting the same stanza into the global /etc/ssh/ssh_config.d/10_our_config.conf file; the result was the same)
This works, as confirmed with the following command succeeds as user foreman-proxy:
$ ssh -i ~/.ssh/id_rsa_foreman_proxy svc-satellite@cn01.behindbastion.example.com
Last login: Sun Feb 19 16:35:56 2023 from 192.168.101.10
[svc-satellite@cn01 ~]$
However, an SSH command job fails
1: Error initializing command: RuntimeError - Could not establish connection to remote host using any available authentication method, tried publickey
2: Exit status: EXCEPTION
3: StandardError: Job execution failed
This used to work in Satellite 6.7.
Second related problem: Trying to run a “Run ansible roles” job also fails. Unlike the previous issue, this also failed in Satellite 6.7 . Also note that the error here is “could not resolve hostname” rather than an authentication failure issue.
The error in this case is:
fatal: [cn01.behindbastion.example.com]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: Could not resolve hostname cn01.behindbastion.example.com: Name or service not known", "unreachable": true}
Expected outcome:
Foreman Proxy should honor the ProxyJump directive and other directives in its config directory.
Foreman and Proxy versions:
Version 3.5.1
Foreman and Proxy plugin versions:
foreman-tasks|7.1.1|
|foreman_ansible|10.4.0|
|foreman_bootdisk|21.0.3|
|foreman_openscap|5.2.2|
|foreman_remote_execution|8.2.0|
|foreman_templates|9.3.0|
|foreman_virt_who_configure|0.5.10|
|foreman_webhooks|3.0.5|
|katello|4.7.2|
Distribution and version:
AlmaLinux 8.7 (Stone Smilodon)
Other relevant data:
At some point, I got the suggestion to use a separate Foreman Proxy server to serve the subnet behind the bastion host. This is not an option in our case.