Foreman Discovery image does not reboot

Support
1

Problem:
Foreman Discovery image does not automatically reboot after discovery.

I have Settings>Discovered>Reboot = true
When I click to the Hosts>Discovered Hosts>Reboot(on a host), the system reboots.
Expected outcome:
System reboots.
Foreman and Proxy versions:
foreman-tasks 5.2.2
foreman_ansible 7.0.2
foreman_bootdisk 19.0.3
foreman_default_hostgroup 6.0.0
foreman_dhcp_browser 0.0.8
foreman_discovery 19.0.1
foreman_monitoring 2.1.0
foreman_puppet 2.0.4
foreman_remote_execution 5.0.3
foreman_statistics 2.0.0
foreman_templates 9.1.0
foreman_webhooks 3.0.0

Distribution and version:
RHEL 8.5

Am I missing a setting somewhere?

2

Are you on 4.0.3? There were some issues with reboots in the older 4.0.x releases:

https://downloads.theforeman.org/discovery/releases/4.0/

If you are, then SSH to the node and try:

  • reboot command
  • reboot -f command

Share your results. Previously we used just “reboot” but that no longer works in EL8, so we added “reboot -f” to workaround that.

3

I was able to reboot by running the “reboot” command over SSH.
Where do I check the version of the discovery image?

4

I am running 4.0.3. Try this on the FDI host:

[root@fdi log]# uname -a
Linux fdi 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Wed Dec 22 13:25:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
[root@fdi log]# cat /etc/redhat-release 
CentOS Stream release 8
[root@fdi log]#
1 Like
5

Does reboot -f work too? That is the command that is used via the API.

It’s printed on the screen, bottom-left corner.

Markus, your problem is different, in your case Foreman is unable to reach out to the FDI for some reason. When you try again via the Reboot button/feature, it works for you as you state in another thread. That is using the very same API call. You need to dig deeper.

6

Confirmed, version is 4.0.3.
reboot -f works.

7

Now it seems that the foreman proxy does not notice that the system was discovered. (This was not an issue yesterday, and I don’t remember changing any settings). It is almost as if the Foreman API server cannot contact the foreman-proxy on the same system.

Foreman server log: ======= this file has been scrubbed of sensitive data ============== data remo - Pastebin.com
Proxy log: ======= this file has been scrubbed of sensitive data ============== data remo - Pastebin.com

The error might be related to the hostname issue, but I have checked everywhere I can think to and the hostname matches.

8

This is what smart-proxy running on the FDI performs when Foreman calls its reboot API. Can you tell if the request reaches the destination? If you log in FDI and perform Reboot, do you see anything in journald?

If you cannot discover hosts, that is typically a infra problem - DNS, firewall, routing… check all of that.

9

That is the strange thing, discovery works. It is just the automatic reboot function that does not.
(I checked again and my previous post was incorrect, my apologies.)
I can reboot via the web interface (Hosts>Discovered Hosts>Actions>Reboot)
It seems that the Reboot setting is not recognized.

Here is the description of that setting:

Automatically reboot or kexec discovered host during provisioning

Am I misunderstanding the way that the automatic reboot works?

10

Foreman calls either HTTP REST “reboot” or “kexec” command depending on if KExec template is associated with your OS. But it is the very same HTTP REST call that is done if you click on Reboot button. Why that would not work.

What is your environment? VM?

11

Interesting.
The Foreman server logging my button click to reboot the host:

2022-04-06T12:11:31 [I|app|9d533075] Started PUT "/discovered_hosts/macbc97e11c1e60/reboot" for <my_ip_address> at 2022-04-06 12:11:31 -0500

The foreman-proxy server logging its API call to the discovery image:

2022-04-06T12:11:31 08978e15 [I] Started PUT /discovery/10.113.2.227/power/reboot                                                                                                                                                                │·
2022-04-06T12:11:31 08978e15 [I] Finished PUT /discovery/10.113.2.227/power/reboot with 200 (52.15 ms)

There is no log entry corresponding to the automatic reboot. The host does reboot when I push the button in the web UI.

Here is the configuration setting for reboot of discovered hosts:

Screen Shot 2022-04-06 at 1.38.26 PM
Screen Shot 2022-04-06 at 1.38.26 PM2920Ă—578 77.2 KB

I am using a Fujitsu physical server as the Foreman and foreman-proxy host, and a Lenovo physical server for the discovery image testing.
The only slight oddity with my environment is that DNS and DHCP are manged externally from Foreman. I have some limited access to change settings for those services but I cannot use Foreman to manage them. Does Foreman need to watch DHCP for the automatic reboot?

12

No.

Can you perform auto discovery with debug mode and pastebin the transaction (session id) from production.log of the whole discovery? There should be something. Reboot is not queued if there was an error during discovery or flag is disabled (which is not).

app/models/host/managed_extensions.rb#queue_reboot

13

Full production log: ======= this file has been scrubbed of sensitive data ============== data remo - Pastebin.com
Discovery of system under test: ======= this file has been scrubbed of sensitive data ============== data remo - Pastebin.com

14

I do not see any relevant requests in the production.log, can you grep just the relevant transaction? Search for power_api keyword, that is the request that is sent. Also I see many PostgreSQL connection problems, something might be very wrong with your instance.

15

Hold on - I think I am misunderstanding the automatic reboot setting.

The host reboots after provisioning, but not after discovery.
Is this expected behavior?

If so, is there a way to ensure that the system reboots after discovery but before provisioning?
In my environment, there are often systems that should be discovered but should remain unprovisioned by Foreman.

16

Hosts do not reboot until they are scheduled to be provisioned, either manually or via Discovery Rule.

1 Like