Remote Execution fails with authentication error on ssh, cli ssh works fine

Problem:
Error initializing command: Net::SSH::AuthenticationFailed - Authentication failed for user

Expected outcome:
Be able to patch hosts

Foreman and Proxy versions:
3.0.3

Foreman and Proxy plugin versions:
Katello 4.2.2

Distribution and version:

Other relevant data:

When you say “cli ssh works fine”, what exactly did you try?

I am able to ssh from the foreman host to another host from the command line from the foreman and foreman-proxy ids.

There might be subtle differences between what foreman does and what people do from the terminal. Could you post the exact command you used?

[foreman[DEV]@dev-foreman ~]$ ssh XXXXXX@dev-foreman.tla.uprr.com ‘uname -a’
Linux dev-foreman.tla.uprr.com 3.10.0-1160.53.1.el7.x86_64 #1 SMP Fri Jan 14 13:59:45 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
[foreman[DEV]@dev-foreman ~]$

The above is the ssh command line that works, performing the same ‘uname -a’ from foreman throws the error noted earlier.

Well, that’s not what foreman is doing when running a job. Try this insetad

sudo -u foreman-proxy ssh -i ~foreman-proxy/.ssh/id_rsa_foreman_proxy XXXXXX@dev-foreman.tla.uprr.com ‘uname -a’

Thank you, that was very helpful, did not realize that foreman-proxy used its own naming for id_rsa files.

Did want to provide one additional detail, while diagnosing the issue we enabled debug mode and discovered that for some reason it was trying to use id_rsa_foreman_proxy.pub despite specifying the -i to point to the private key file ~foreman-proxy/.ssh/id_rsa_foreman_proxy. We moved the public key file out of the way and it started working.