Delay on Build and Cancel build reqeusts from Foreman Server

Support
1

Hello everybody

Problem:
I have noticed quite a bit of delay when clicking Build and Cancel Build on a host, this delay is roughly around 50s of starting the request, I can’t see too much inside of the logs and have turned on Debug Logging on the Foreman server. I’m after some better ways to debug this issue, or if anybody else has experienced it, what they did to fix it.

Expected outcome:
We’ve got another Foreman instance on version 2.4, and a similar request takes a few seconds at most, this would be what I expected

Foreman and Proxy versions:
3.10.0 foreman
3.10.0 foreman-proxy

Foreman and Proxy plugin versions:
foreman and foreman-proxy versions:

Ansible

3.5.5

DNS

3.10.0

Discovery

1.0.5

Dynflow

0.9.2

HTTPBoot

3.10.0

Openscap

0.9.2

Content

3.3.0

Script

0.10.6

TFTP

3.10.0

Templates

3.10.0

Distribution and version:
AlmaLinux 8.10 for both foreman server and foreman-proxy

Other relevant data:
Thanks in advance

Attached are some images of the request

Screenshot 2024-07-31 at 10.36.48 AM
Screenshot 2024-07-31 at 10.36.48 AM1352×884 121 KB

Screenshot 2024-07-31 at 10.37.10 AM
Screenshot 2024-07-31 at 10.37.10 AM1344×890 118 KB

2

Hey

Going to add to this as its still happening to me after two upgrades, have uplifted everything to 3.12.

It occurs on the following actions afiak

  1. Cancel Build
  2. Build
  3. Submit on the Create host page.

Some extra info

  • It doesn’t seem matter which compute resource type is used on the VM, have tried it on Libvirt, oVirt and Proxmox, same exact delay on all.
  • Its also triggered via hammer, and it seems to be getting stuck on the following section

This is how i’m triggering it with verbose/debug output

hammer -v -d host update --name t2-testhost-27-mgmt --build true

This is where it stops, for the ~50 seconds.

[ INFO 2024-10-28T14:28:26 API] Server: https://foreman-url
[ INFO 2024-10-28T14:28:26 API] PUT /api/hosts/145
[DEBUG 2024-10-28T14:28:26 API] Params: {
    "host" => {
                                 "name" => "t2-testhost-27-mgmt",
                                "build" => true,
             "content_facet_attributes" => {},
        "subscription_facet_attributes" => {},
                    "puppet_attributes" => {}
    }
}
[DEBUG 2024-10-28T14:28:26 API] Headers: {}
[DEBUG 2024-10-28T14:28:26 API] Using authenticator: HammerCLIForeman::Api::InteractiveBasicAuth
[DEBUG 2024-10-28T14:29:18 API] Response: {
                            "ip" => "172.23.66.191",
                           "ip6" => "",
                   "last_report" => nil,
                           "mac" => "bc:24:11:29:a8:18",
                      "realm_id" => nil,
...

I have spun up an instance without Content, on 3.12 and it doesnt seem to have the same issues with the build delay, is there some way i can rule out katello/content on this instance for testing? Or has anybody else possibly got any other ideas for some additional investigation?

Thanks all

Foreman and Proxy versions:
3.12.0

Foreman and Proxy plugin versions:

Distribution and version:

3

No longer need the help, user error (me)!

In our case Foreman was trying to resolve the DNS name of the host that was getting rebuilt, against the SOA/NS, which was problematic in our environment. Switched the setting “Query local nameservers” to a Yes and problem resolved.

1 Like