RFC: Store OS/NIC inventory data as custom facts on clients

Hey,

while I believe that parsing and setting Foreman resources like OS, version, NIC (subnet, domain) on fact upload is a bad design from the moment we introduced provisioning and I think this should be explicit, I have an alternative idea.

I was made aware that subscription-manager also supports custom facts by creating /etc/rhsm/facts/*.facts file in JSON format. This is similar to Puppet and Ansible when both tools can do the same. We could leverage this by creating these files from provisioning templates and/or via Puppet or Ansible. We could simply store database IDs there.

Now, these facts would be sent over to Foreman by the tools and Foreman would prioritize during parsing. This could also speed up processing a ton - for managed hosts which were provisioned via Foreman, whole NIC processing could be skipped as these would be present as list of NIC IDs. When IDs do match, that’s a fast way to just confirm and nothing is done.

Of course things get complicated on a change (when user changes OS for example). And I think there could be an elegant solution - when IDs differ, we would simply defer the database update by setting a new Host Status field called “Custom Fact Out of Sync” or something similar. There would be a link to a new screen/dialog where Foreman could offer remote execution job (utilizing the same templates as in provisioning), Ansible or Puppet to remediate the fact drift. This would be only allowed within permission/filter/taxonomy bounds of course not allowing users to set arbitrary database IDs from managed hosts. Other than that, there should be no security concerns as current clients with valid SSL certificates can do the same (change OS, version, subnet, domain…).

We could even offload the drift calculation and do it on the background when we update host statuses which would further speed up fact parsing. I think we already have an infrastructure for host status updates (rake task or a dynflow - I am not sure).

This solution would stop “facter wars” when RHSM is updating OS in a different way than Puppet or Ansible for all the hosts that were provisioned by Foreman. Yet it would not change the behavior of hosts which were not provisioned by Foreman. More than that, it would open more possibilities to take similar approach for other hosts.

Once implemented, we could take it further by applying the same technique for regular fact uploads (those without Foreman custom facts) - the same host status could be set and on the confirmation screen/dialog could actually offer more options:

  • Set custom Foreman facts on the host from Foreman inventory (run SSH/Ansible/Puppet job).
  • Update Foreman DB back with values from custom facts (revert to the state how a host was provisioned).
  • Update Foreman DB from Puppet facts trying to detect OS/NICs/etc using the current way.
  • Update Foreman DB from Ansible facts trying to detect OS/NICs/etc using the current way.
  • Update Foreman DB from RHSM facts trying to detect OS/NICs/etc using the current way.

Also hosts which are registered via the new registration method could use the same snippets to deploy custom facts too.

This should not be really hard to implement, we are talking about few templates, Puppet class, Ansible role, new dialog, changing our parsing code, adding ability to re-parse existing facts already uploaded to database and a new host status and a new confirmation dialog.

Relevant threads:

One related item, when we get hw failures and nic cards get replaced, we have to update the facts thru a side channel, otherwise re-provisioning the system fails.