Foremain-installer takes a long time - Facter Timeout error shown

Problem: foremain-installer takes a long time - Facter Timeout error shown

Expected outcome: No delay

Foreman and Proxy versions: 3.10

Foreman and Proxy plugin versions: F 3.10 K 4.12

Distribution and version: Rocky 8

Other relevant data:
Every time I run foreman-installer on my proxy servers I have to either wait facter command to time out or I have to kill the process of it. I am wondering why this keeps happening and if there is something I can do to resolve it.
When I first start forman-installer, it will sit at this step for many minutes:

root@uscdc01plsat004:~# foreman-installer
2024-09-05 11:08:57 [NOTICE] [root] Loading installer configuration. This will take some time.

I usually have to run a loop command to kill the the PID that is hung, we have found the installer seems to hang on:
sh -c /bin/bash" | grep -i true | awk ‘{print $2}’

So I have to run a loop to kill it any processes with that command:

for i in {1..100}; do  pid=$(ps -ef | grep "sh -c /bin/bash" | grep -i true | awk '{print $2}') ;  child=$(echo $pid + 1 | bc)  ;  kill $child ;  sleep 1 ;  pid=$(ps -ef | grep "/bin/bash -c if /bin/grep" | grep -i true | awk '{print $2}') ;  child=$(echo $pid + 1 | bc);  kill $child ;  sleep 1 ; done

Once it finds and kills the process than the installer will start to run:

root@uscdc01plsat004:~# foreman-installer
2024-09-05 10:13:04 [NOTICE] [root] Loading installer configuration. This will take some time.
2024-09-05 10:14:04 [NOTICE] [root] Running installer with log based terminal output at level NOTICE.
2024-09-05 10:14:04 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2024-09-05 10:14:07 [NOTICE] [configure] Starting system configuration.
**2024-09-05 10:20:28 [ERROR ] [configure] Facter: Error while resolving custom fact fact='nimsoft_port_48003', resolution='<anonymous>': Failed while executing/bin/echo 'true' ; else /bin/echo 'false' ; fi': Timeout encounter after 300s, killing process with pid: 533181**
2024-09-05 10:25:27 [NOTICE] [configure] 250 configuration steps out of 1151 steps complete.
2024-09-05 10:25:31 [NOTICE] [configure] 500 configuration steps out of 1154 steps complete.
2024-09-05 10:25:32 [NOTICE] [configure] 750 configuration steps out of 1159 steps complete.
2024-09-05 10:25:32 [NOTICE] [configure] 1000 configuration steps out of 1160 steps complete.
2024-09-05 10:26:24 [NOTICE] [configure] System configuration has finished.

However, if I kill my loop the installer will; more often than not, hang/time out on Facter as shown above.

Also, when it hangs at:


root@uscdc01plsat005:~# foreman-installer
2024-09-05 11:08:57 [NOTICE] [root] Loading installer configuration. This will take some time.

I see that the last log message in /var/log/foreman-installer/foreman-proxy-content.log hangs at:

      ' | RUBYLIB=/usr/share/gems/gems/kafo-7.4.0/lib/kafo/../..//modules: /opt/puppetlabs/bin/puppet apply --config=/tmp/kafo_installation20240905-67612-1it89q8/puppet.conf

The Timeout error already pretty much tells you where the problem is most likely to be found. It is trying to resolve a custom fact named nimsoft_port_48003 that performs some sort of shell execution. I am 99% certain this is some custom fact of yours that comes with some Puppetmodule you probably use outside of foreman-installer and the embedded call of puppet within the installer simply tries to resolv that fact and fails/times out.
Since nimsoft seems to be some proprietary software for monitoring (from what I found with a quick google search) I am pretty certain that this is not a problem with foreman-installer but rather your own custom facts.

Hi,

Yes, that is correct. We have Nimsoft installed on the servers. I’ll further investigate the Nimsoft issue but wasn’t sure if there was something else related to the foreman-installer command. Thanks!