Facter 4 and unstable facts

Development
1

Hello,

I am trying to find what facts (in facter 4.x) are “unstable” or “volatile” should I say. These type of facts change very often, most likely on every single fact upload. This also means they put unnecessary stress on the Foreman database requiring update on every fact upload.

I am trying to find them, I’d appreciate help. On a host with Facter 4.x simply do:

facter --yaml > facter_$RANDOM.yaml

And after minutes, hours or days repeat. Then compare via GNU diff. I was able to identify the following facts:

load_averages:
  15m: 0.57
  1m: 0.34
  5m: 0.47

I am not sure how useful is to have load in Foreman fact database, it will update pretty much everytime. Do you even use this? Foreman is NOT the right tool for monitoring, we focus solely on Configuration Management monitoring (drift monitoring).

memory:
  swap:
    available: 33.00 GiB
    available_bytes: 35433476096
    capacity: 0.00%
    used: "0 bytes"
    used_bytes: 0
  system:
    available: 23.70 GiB
    available_bytes: 25444986880
    capacity: 13.77%
    used: 4.31 GiB
    used_bytes: 4627849216

The same case. Note there is a total which makes a lot of sense but that is a “stable” fact, it does not change (unless system is given more memory of course). Here are the facts which are OKAY:

memory:
  swap:
    total: 33.00 GiB
    total_bytes: 35433476096
  system:
    total: 31.29 GiB
    total_bytes: 33596174336

Unfortunately, the whole uptime is reported weirdly, all these facts are volatile:

system_uptime:
  days: 0
  hours: 1
  seconds: 5058
  uptime: "1:24 hours"

The correct approach would be to report something like “boot_time” in UTC which can be easily used to calculate uptime for humans. The last one I was able to identify on my system was:

processors:
  models:
  speed: 1.38 GHz

My CPU has variable speed depending on load. Facter reports current speed which is not much useful, I think this might be actually a bug. Usually you want to see number of CPUs, model, cores and clock information to plan deployments accordingly.

Now here is a trick - if you create a configuration file these facts can be actually blocklisted:

facts : {
  blocklist : [
    "load_averages",
    "identity",
    "memory.system.capacity",
    "memory.system.used*",
    "memory.system.available*",
    "memory.swap.capacity",
    "memory.swap.used*",
    "memory.swap.available*",
    "system_uptime",
    "processors.speed",
  ]
}

Then just call facter -c config_file.conf to ignore these for once and forever. I know that this canb be applied currently for all of them but we can probably create such a configuration and ship it with Foreman for the better.

I created a ticket for Puppetlabs to consider fixing the “cpu speed” and “uptime” by providing stable fact alternatives: https://tickets.puppetlabs.com/browse/FACT-2838

3 Likes
2

Thanks for this. These are indeed facts that aren’t very useful in most setups.

https://puppet.com/docs/puppet/6.18/configuring_facter.html documents it so you can apply the same trick in Facter 3 already. It may also work in Puppet 5.

Does it make sense to also filter these on the Foreman side? We also have a fact filter there and doing so once is probably easier than deploying it on all systems.

3

I think filtering them on Foreman side could be probably even better. I can file a PR for load average and memory, however system_uptime is used in Foreman to calculate boottime actually. And CPU speed is a key fact we don’t want to filter out - this needs to be fixed in Facter itself.

Once there is a “boot time” fact present, we can check for that fact and if it is, we can actually remove the unstable variants in Foreman code. But we need to wait for a fix in facter until upstream finds out a good fact name.

1 Like