Perhaps a unique ask but I am trying to use Foreman’s remote execution plugin to connect over to our server but do this via an SSH proxy.
I have an ssh config in place under the root user and it proxies via an admin box we have running first. Manual SSH tests work fine.
I have also set Foreman to use root user for remote execution under the options. I’ve also added the key we use from out admin host into /usr/share/foreman-proxy/.ssh just in case its required there (even though we use root)
Tailing /var/log/secure on the end server does see attempts coming in from the remote job but its showing Foreman’s IP as source, not the proxy. And therefore fails due to some restrictions we have in sshd config
Any ideas where we can setup this proxy if /root/.ssh/config or /usr/share/Foreman-proxy/.ssh isn’t being looked at by Foreman?
if I understand your use-case correctly, you might want to take a look at Remote execution not using jump host/bastion host configuration
Iirc, this does not work out of the box currently but needs a little tweaking in the smart-proxy code. The last post in that thread here has a workaround/patch for an older Foreman version. Not sure, but it should probably still work somewhat alike.
thanks, that key path is good to know. I will copy the root key there as well. I would hazard a guess that the commad changes to sudo -u root or similar if you request Foreman to perform remote execution as root under the settings like I have
There a multiple ways to those files. There is a symbolic link from ~foreman-proxy/.ssh to /var/lib/foreman-proxy/ssh.
That depends on how you configure it. See the settings for remote execution. You can operate either with remote login as root directly or with remote login as a unprivileged user and then using sudo or su. So basically you can do it any way you want…
ah nevermind, i just checked /var/lib/foreman-proxy/ssh and it already contains the root ssh key in there for our infra. Must be symlinked with /usr/share/foreman-proxy/.ssh through foreman’s setup (not seeing the symlink in the listings)
thanks. I have ‘SSH user’ set as root. Effective user also root and effective user method is sudo. I want keys to be used though, and there is no section for key location
Well if you have ssh user root, the other options don’t really come to play. Foreman uses ssh to user root.
The key have mentioned is the default settings. It’s configure via ssh_identity_file in the installer. See option --foreman-proxy-plugin-remote-execution-script-ssh-identity-file. However, I recommend not to change it.
Agreed - that makes sense - Foreman will try as root so the other options don’t really matter
In that case, I don’t understand how it’s possible to use root with a key unless I run the installer again with that identity-file option pointing to root?
The foreman-proxy user runs the ssh command on the foreman server (or smart proxy) like this:
$ ssh -i /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy root@test.example.com date
So all you have to do is to put the key into /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy on the foreman server and of course make sure that it’s readable for user foreman-proxy.
If you use ssh user root (which is the “root” in the command above) then the command will directly log in as user root…
this is what I’ve already tried. copied both the root ssh key as well as the foreman-proxy key to our admin box authorized_keys file which we are using as a proxy.
The Foreman /root/.ssh config file contains a proxy command to go via the admin box. Manual tests from foreman master ssh to an end box work fine and I can see the proxy being used.
It’s just the scheduled job that fails. Logs show it doesnt hit the proxy admin box at all - just tries to hit the end box and fails
My concern is not so much around the key but why it doesnt read the config and try to proxy through the admin box
yes, I know the /root/.ssh/config is for root - I have already told Foreman to execute as root user, so it should use the root settings including key and config.
the manual test is just SSH with verbose output from foreman to the end user. Im not pasting it all here but key line is it connecting the proxy (I’ve removed hostnames):
debug1: Authentication succeeded (publickey).
Authenticated to end-client (via proxy).
As for yor test suggestion - I dont’ want to run the command as foreman-proxy so that test doesn’t make sense to me - I’ve already configured it to run as root in the Foreman settings.
Finally - the /root…ssh/proxy config has also been placed in both var/lib/foreman-proxy/ssh and /usr/share/foreman-proxy/.ssh JUST IN CASE Foreman decided to run ssh as root but using contents in these folders which seem to be only for the foreman-proxy user.
You can’t. As I wrote before: the smart proxy runs as foreman-proxy user. The remote execution plugin runs as foreman-proxy. The ssh command above runs as foreman-proxy. That’s why I keep repeating that
# sudo -u foreman-proxy ssh -i /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy root@test.example.com date
is effectively what you have to test. Run as foreman-proxy. The smart proxy does not run as root. It has no root access. It has no access to /root. It doesn’t have any way to become root.
You cannot change that. You really have to use that command above.
/usr/share/foreman-proxy/.ssh should be a sym link to /var/lib/foreman-proxy/ssh. If may also be an issue.
As I wrote before: the SSH user in the remote execution settings is the login user in the ssh command.
ssh -i /var/lib/foreman-proxy/ssh/id_rsa_foreman_proxy root@test.example.com date
SSH user is the login name at the client. Effective user is the user on the client system used to execute the command and effective user method is the method how to switch from the login user to the effective user. If login and effective user are the same, the method is not used.
If you set SSH user to foreman-exec-user, effective user to root and method to sudo, the command would change to something like:
But the foreman server/smart proxy runs the ssh command as foreman-proxy. The smart proxy runs as foreman-proxy and runs the plugin and ssh commands as foreman-proxy.
Are you sure you see the ssh command on the foreman server running as root? It’s definitively running as foreman-proxy on mine.
Understood, but the Foreman settings suggest effective user is used to run the script.
Efffective User: Default user to use for executing the script. If the user differs from the SSH user, su or sudo is used to switch the user.|
Both effective and ssh user are set to root which should overwrite the foreman-proxy user value and execute the script at root. Then attempt to login as root.
Yes. The script being “date” in my example above. It’s the user which is running the script on the client host.
Yes. As I have explained: the effective user is the user which is supposed to run the script on the client. SSH user is the user used to login into the client. If both different, there must be a switch of users somehow. Either by sudo or su.
No. It’s wrong. Everything on the foreman server/proxy runs at foreman-proxy. The “script” is what is running on the client, e.g. date. The effective user is the user which is running the script on the client host. The SSH user is the user used to log into the client. All those settings are about the client side.
The foreman proxy sets up the ssh client call and runs it. What would be the purpose of the script running as root on the foreman proxy itself? All it has to do is to set up the execution wrapper (for error handling, timeouts, etc.) and run it on the remote side. It absolutely has no need to have root access on the local foreman server.
The user on the server is the local foreman-proxy.
If you wanted to reboot a remote host (aka client) manually, you’ll need access to root on the remote host, not on your local computer…
OK, very confusing stuff. If you running a script and logging into a client as a specific SSH user, then 9 times out of 10, the task would be run as that user also. So these options simply allow you to run the end client task as a different user to the one you’ve logged in as. Seems overkill IMO
Back to my main issue then… yes I can login into both our admin proxy box and end client using that ‘id_rsa_foreman_proxy’ key as it was copied over.
However, I want to rely on just the admin box connecting into the client and Foreman proxies to the end client through this admin box, hence the hope that Foreman would be able to read the ssh proxy config I have created in /var/lib/foreman-proxy/ssh (as well as roots which we have established is likely not to be read at all by Foreman)
No. It depends on how you do it, but at least if you come from your local desktop and connect via SSH to any server you’ll have different users on the local and remote end.
And it’s not overkill but more secure: why would you want to run something as root on the foreman server which absolutely doesn’t require root rights? You want to log into the remote host and all you need is credentials to get in there.
Regarding the “issue”: as I wrote before: use the command I have posted multiple times by now, enable verbose output and look into the output. It tells you exactly what is done and why. If it is not doing what you expect then you’ll need to fix that.
Beyond that it’s really virtually impossible to help without knowing the verbose output nor the config you are using nor anything about your foreman server (all that information that you are asked about when you start a thread here but which you have removed completely…)
