Foreman installation failing due to GET request error

Support
1

Problem:
I ran into an issue after the foreman server (RHEL8) was patched. Tried to correct it by running sudo foreman-installer. It failed with

" /Stage[main]/Foreman::Register/Foreman_host[foreman-fw1-d-hq-frman.faps.net]: Could not evaluate: Error making GET request to Foreman at https://forman.server.net/api/v2/hosts: Response: 502 Proxy Error: The webserver received an invalid response from the backend service. Was Foreman at foreman.server.net unable to handle the request?"

Expected outcome:
Completed installation

Foreman and Proxy versions:

Foreman vs. 3.5.3-1.el8

Foreman and Proxy plugin versions:

Distribution and version:

Other relevant data:
There have been firewall changes done and I am suspicious that they are causing my issues. Just wanting to verify.

2

It’s quite possible. This is just saying “I can’t contact the Foreman server.”

If foreman-maintain service status says everything is okay, firewalls would be a good thing to check next.

3

Thanks Jeremylenz. I ran foreman-maintain service status and got the following:

(dynflow-sidekiq@orchestrator, foreman, dynflow-sidekiq@worker-1, dynflow-sidekiq@worker-hosts-queue-1, foreman-proxy)

Looks like all of these are stuck in a activating state.

4

And what about after a foreman-maintain service restart ?

5

Job for dynflow-sidekiq@orchestrator.service failed because the control process exited with error code.
See “systemctl status dynflow-sidekiq@orchestrator.service” and “journalctl -xe” for details.

Job for foreman.service failed.
See “systemctl status foreman.service” and “journalctl -xe” for details.
/ starting dynflow-sidekiq@worker-hosts-queue-1
Job for dynflow-sidekiq@worker-1.service failed because the control process exited with error code.
See “systemctl status dynflow-sidekiq@worker-1.service” and “journalctl -xe” for details.

Job for dynflow-sidekiq@worker-hosts-queue-1.service failed because the control process exited with error code.
See “systemctl status dynflow-sidekiq@worker-hosts-queue-1.service” and “journalctl -xe” for details.
\ starting foreman-proxy
Job for foreman-proxy.service failed because the control process exited with error code.
See “systemctl status foreman-proxy.service” and “journalctl -xe” for details.
| All services started

**sudo systemctl status dynflow-sidekiq@orchestrator.service results:

● dynflow-sidekiq@orchestrator.service - Foreman jobs daemon - orchestrator on sidekiq
Loaded: loaded (/usr/lib/systemd/system/dynflow-sidekiq@.service; enabled; vendor preset: disabled)
Active: activating (start) since Wed 2024-03-27 16:05:20 CDT; 1s ago
Docs: https://theforeman.org
Main PID: 584561 (sidekiq)
Tasks: 1 (limit: 126344)
Memory: 36.2M
CGroup: /system.slice/system-dynflow\x2dsidekiq.slice/dynflow-sidekiq@orchestrator.service
└─584561 /usr/bin/ruby /usr/bin/sidekiq -e production -r /usr/share/foreman/extras/dynflow-sidekiq.rb -C /etc/foreman/dynflow/orchestrator.yml

Mar 27 16:05:20 fw1-d-hq-frman systemd[1]: dynflow-sidekiq@orchestrator.service: Service RestartSec=1s expired, scheduling restart.
Mar 27 16:05:20 fw1-d-hq-frman systemd[1]: dynflow-sidekiq@orchestrator.service: Scheduled restart job, restart counter is at 29847.
Mar 27 16:05:20 fw1-d-hq-frman systemd[1]: Stopped Foreman jobs daemon - orchestrator on sidekiq.
Mar 27 16:05:20 fw1-d-hq-frman systemd[1]: Starting Foreman jobs daemon - orchestrator on sidekiq…
Mar 27 16:05:21 fw1-d-hq-frman dynflow-sidekiq@orchestrator[584561]: 2024-03-27T21:05:21.103Z pid=584561 tid=cjwh INFO: Enabling systemd notification integration

Systemctl status foreman.service results:

foreman.service - Foreman
Loaded: loaded (/usr/lib/systemd/system/foreman.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/foreman.service.d
└─installer.conf
Active: activating (start) since Wed 2024-03-27 16:08:21 CDT; 2s ago
Docs: https://theforeman.org
Main PID: 585783 (rails)
Tasks: 1 (limit: 126344)
Memory: 40.7M
CGroup: /system.slice/foreman.service
└─585783 /usr/bin/ruby /usr/share/foreman/bin/rails server --environment production

Mar 27 16:08:21 fw1-d-hq-frman systemd[1]: Starting Foreman…

sudo systemctl status dynflow-sidekiq@worker-hosts-queue-1.service results:

dynflow-sidekiq@worker-hosts-queue-1.service - Foreman jobs daemon - worker-hosts-queue-1 on sidekiq
Loaded: loaded (/usr/lib/systemd/system/dynflow-sidekiq@.service; enabled; vendor preset: disabled)
Active: activating (start) since Wed 2024-03-27 16:09:22 CDT; 1s ago
Docs: https://theforeman.org
Main PID: 586215 (sidekiq)
Tasks: 1 (limit: 126344)
Memory: 30.7M
CGroup: /system.slice/system-dynflow\x2dsidekiq.slice/dynflow-sidekiq@worker-hosts-queue-1.service
└─586215 /usr/bin/ruby /usr/bin/sidekiq -e production -r /usr/share/foreman/extras/dynflow-sidekiq.rb -C /etc/foreman/dynflow/worker-hosts-queue-1.yml

Mar 27 16:09:22 fw1-d-hq-frman systemd[1]: Starting Foreman jobs daemon - worker-hosts-queue-1 on sidekiq…
Mar 27 16:09:23 fw1-d-hq-frman dynflow-sidekiq@worker-hosts-queue-1[586215]: 2024-03-27T21:09:23.816Z pid=586215 tid=cl07 INFO: Enabling systemd notification integration

6

Journalcte:

Hint: You are currently not seeing messages from other users and the system.
Users in groups ‘adm’, ‘systemd-journal’, ‘wheel’ can see all messages.
Pass -q to turn off this notice.
– Defined-By: systemd
– Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal

– Unit UNIT has finished starting up.

– The start-up result is done.
Mar 27 15:13:57 fw1-d-hq-frman systemd[561010]: Listening on D-Bus User Message Bus Socket.
– Subject: Unit UNIT has finished start-up
– Defined-By: systemd
– Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal

– Unit UNIT has finished starting up.

– The start-up result is done.
Mar 27 15:13:57 fw1-d-hq-frman systemd[561010]: Reached target Sockets.
– Subject: Unit UNIT has finished start-up
– Defined-By: systemd
– Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal

– Unit UNIT has finished starting up.

– The start-up result is done.
Mar 27 15:13:57 fw1-d-hq-frman systemd[561010]: Reached target Basic System.
– Subject: Unit UNIT has finished start-up
– Defined-By: systemd
– Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal

– Unit UNIT has finished starting up.

– The start-up result is done.
Mar 27 15:13:57 fw1-d-hq-frman systemd[561010]: Reached target Default.
– Subject: Unit UNIT has finished start-up
– Defined-By: systemd
– Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal

– Unit UNIT has finished starting up.

– The start-up result is done.
Mar 27 15:13:57 fw1-d-hq-frman systemd[561010]: Startup finished in 116ms.
– Subject: User manager start-up is now complete
– Defined-By: systemd
– Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal

– The user manager instance for user 1000 has been started. All services queued
– for starting have been started. Note that other services might still be starting
– up or be started at any later time.

– Startup of the manager took 116839 microseconds.
Mar 27 15:15:57 fw1-d-hq-frman systemd[561010]: Starting Mark boot as successful…
– Subject: Unit UNIT has begun start-up
– Defined-By: systemd
– Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal

– Unit UNIT has begun starting up.
Mar 27 15:15:57 fw1-d-hq-frman systemd[561010]: Started Mark boot as successful.
– Subject: Unit UNIT has finished start-up
– Defined-By: systemd
– Support: Red Hat Customer Experience & Engagement - Red Hat Customer Portal

– Unit UNIT has finished starting up.

– The start-up result is done.