On Foreman 3.8/Katello 4.10.0 and when performing some migrations of hosts from CentOS 7 to AlmaLinux 9 I noticed that after I perform this command from within the host to remove the old host from Foreman:
the entry for the host is still available in the /var/lib/foreman-proxy/ssh/known_hosts on the proxy. So when I now deploy the replacement host using the same IP and name and use remote execution, the proxy refuse to connect with the error:
Authentication method 'publickey' failed with:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
So to correct this I have to manually purge the old key from known_hosts on the proxy.
Is this the expected behaviour?
Would be nice if when removing a host from Foreman it also removes the entry for the host in known_hosts automatically.
It should work if you really remove the host from Foreman. As far as I know, the subscription-manager sequence you used only cleans the host itself, but the host record in foreman stays.
hmm, just tested again with a test machine and it for sure does not exist in Foreman after when looking for it with “hammer host list”.
I registered the host with the “curl https://:9090$proxyname/register…” command generated from the “Register Host” UI.
“subscription-manager unregister” gives me this back so I would assume all is good on the Foreman side:
Unregistering from: proxy.local:443/rhsm
System has been unregistered.
So is “subscription-manager unregister” not a supported way to unregister a host from Foreman? I mean all seems to work except the removal of the host from known_hosts on the proxy.
If there a way to remove a host from Foreman that cleans up known_hosts on the proxies?
Perhaps the difference is the host type. Registering with subscription-manager does not create a managed host if I remember correctly. Unregistering can so only remove the content facet of the host, so I would expect it to stay if managed and perhaps to be completely removed if unmanaged.
And SSH management is very likely part of a managed host, too.
Looking with “hammer host info --id ” I see this status on all my hosts:
Managed: no
Not really sure what this indicate. I mean I do manage the hosts with Foreman/Katello with subscriptions/repos/errata/remote execution, ansible, scap and so on.
Does managed here means they are provisioned from Foreman?
foreman keeps track if a rex job was execute through a particular smart proxy
if foreman determines the job goes through a specific proxy for the first time, the proxy should remove the entry from known hosts file just before trying to open the connection
when you remove the host, the record tying a host to the smart proxy gets dropped so on next execution foreman should think that it is being executed that way for the first time
So you should see the key there, until you try running the job again
OK, but since the remote execution stops working for any of my migrated hosts and Foreman complains that the key in known_hosts on the proxy no longer match the new host, something is broken in my end or something broken in Foreman?
I mean I can ad-hoc a fix for the issue, just need to add some automation that constantly erase the known_hosts files on the proxies. Pretty ugly though…
