Clustered Foreman Hosts with Audit Message "Spam"

Problem:
When utilizing clustered foreman (Multiple Foreman instances) and following the clustering guide(s) in the manual, the hosts end up with hundreds of thousands to millions of log entries a day around modifying the “default” items unattended_url, foreman_url, despite them being set properly in foreman.yaml and/or foreman’s settings UI.

This is a follow-up of this forum post: Please help: Foreman Audits shows settings updates stepping on each other
which references: Bug #14954: Stop auditing the default field in settings - Foreman
As of 1.24.3 this still doesn’t seem to be implemented.

In all of my environments, the audits page is “extremely” slow to load (minutes) as of upgrading to 1.24.3. In my production instance - where we keep 7 days of audits (and this equals something like 7 million entries, nearly all of which are this “spam”) - I cannot load the audits page at all, it simply times out after 300 or so seconds with a “500 internal server error” screen.

My yaml file (s) for output

Expected outcome:
I would expect - when configured above - these entries should “not” be spammed in audits.
[jlang1@fmnapdvl5 ~]$ sudo cat /etc/foreman/settings.yaml | grep ssl
:require_ssl: true
:websockets_ssl_key: /etc/puppetlabs/puppet/ssl/private_keys/host.pem
:websockets_ssl_cert: /etc/puppetlabs/puppet/ssl/certs/host.pem
:ssl_certificate: /etc/puppetlabs/puppet/ssl/certs/host.pem
:ssl_ca_file: /etc/puppetlabs/puppet/ssl/certs/ca.pem
:ssl_priv_key: /etc/puppetlabs/puppet/ssl/private_keys/host.pem

Host.pem is a symlink - this is so all servers in the “cluster” use the same value(s) above

[jlang1@fmnapdvl5 ~]$ sudo ls -al /etc/puppetlabs/puppet/ssl/private_keys
total 4
drwxr-x—. 2 puppet puppet 55 Dec 7 11:08 .
drwxrwx–x. 7 puppet puppet 116 Dec 7 11:08 …
-rw-r-----. 1 puppet puppet 3243 Dec 7 11:08 fmnapdvl5.pxlabus.com.pem
lrwxrwxrwx. 1 root root 65 Dec 7 11:08 host.pem -> /etc/puppetlabs/puppet/ssl/private_keys/fmnapdvl5.pxlabus.com.pem

Value of foreman_url and unattended_url in foreman are both:

myurl.mydomain.com

#Attached screenshot shows a subset of my audits on a lower-level environment. Tons and tons and tons of “flipping” of these values…

Foreman and Proxy versions:
Foreman 1.24.3

Foreman and Proxy plugin versions:

foreman-tasks The goal of this plugin is to unify the way of showing task statuses across the Foreman instance. It defines Task model for keeping the information about the tasks and Lock for assigning the tasks to resources. The locking allows dealing with preventing multiple colliding tasks to be run on the same resource. It also optionally provides Dynflow infrastructure for using it for managing the tasks. Ivan Nečas 0.17.5
foreman_ansible Ansible integration with Foreman Daniel Lobato Garcia 4.0.5
foreman_bootdisk Plugin for Foreman that creates iPXE-based boot disks to provision hosts without the need for PXE infrastructure. Dominic Cleal 16.0.0
foreman_cockpit This plugin adds a tab to see your host’s Cockpit components, such as console, journal, and networking if the host has Cockpit installed. Daniel Lobato Garcia 2.0.3
foreman_column_view Displays an additional column in the Foreman Hosts view and/or additional entries in the Host show page Greg Sutcliffe 0.4.0
foreman_dhcp_browser Plugin for Foreman to browse and add/edit/delete DHCP leases independent of Foreman’s host creation Ohad Levy 0.0.8
foreman_hooks Plugin engine for Foreman that enables running custom hook scripts on Foreman events Dominic Cleal 0.3.16
foreman_memcache Adds memcache support to foreman Ohad Levy 0.1.1
foreman_remote_execution A plugin bringing remote execution to the Foreman, completing the config management functionality with remote management functionality. Foreman Remote Execution team 2.0.8
foreman_setup Plugin for Foreman that helps set up provisioning. Dominic Cleal 7.0.0
foreman_snapshot_management Foreman-plugin to manage snapshots in a virtual-hardware environments. ATIX AG 1.7.1
foreman_templates Engine to synchronise provisioning templates from GitHub Greg Sutcliffe 7.0.7

Distribution and version:
RHEL 7.9

Other relevant data:

Not sure if production.log or similar will provide any additional help/insight here - but will be happy to provide it if needed.

1 Like