New host provisioning failing after upgrading Foreman/Katello

Support
,
1

Problem: New host provisioning failing after upgrading Foreman/Katello

After discovering a new host and attempting to provision with custom parameters through the UI I am met with a ‘Host not found’ error. We are not using DHCP

image
image1923×695 37.5 KB

Expected outcome: Host provisioning succeeds

Foreman and Proxy versions: Foreman 3.18/Katello 4.20

Distribution and version: RedHat Enterprise Linux 9.8

Other relevant data:

When entering custom fields for provisioning the IPv4 subnet field appears empty even though this information should already be populated for the host from discovery

image

Looking at the production log at the time of the failed build on the Foreman server I see some 404s being thrown but I am having trouble deducing where this host information is being lost

2026-06-24T10:42:44 [I|app|16f9d0e4] Completed 200 OK in 4ms (Views: 0.6ms | ActiveRecord: 1.5ms | Allocations: 1598)
2026-06-24T10:42:44 [I|app|56dc8d0e] Started GET "/rhsm/consumers/57256280-9fa9-400e-9f62-6aa63f3507be/certificates/serials" for REDACTED at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|56dc8d0e] Processing by Katello::Api::Rhsm::CandlepinProxiesController#serials as JSON
2026-06-24T10:42:44 [I|app|56dc8d0e]   Parameters: {"id"=>"57256280-9fa9-400e-9f62-6aa63f3507be"}
2026-06-24T10:42:44 [E|app|56dc8d0e] RestClient::NotFound: Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"ede5781f-6b4f-43be-bf2f-3321a9817cba"} (GET /candlepin/consumers/57256280-9fa9-400e-9f62-6aa63f3507be)
 56dc8d0e | Body: {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"ede5781f-6b4f-43be-bf2f-3321a9817cba"}
 56dc8d0e | 
2026-06-24T10:42:44 [E|app|56dc8d0e] /usr/share/gems/gems/katello-4.19.2/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:316:in `block in find_host'
2026-06-24T10:42:44 [I|app|56dc8d0e]   Rendered text template (Duration: 0.0ms | Allocations: 5)
2026-06-24T10:42:44 [I|app|56dc8d0e] Completed 404 Not Found in 54ms (Views: 1.8ms | ActiveRecord: 1.6ms | Allocations: 7055)
2026-06-24T10:42:44 [I|app|b7a50709] Started GET "/rhsm/consumers/57256280-9fa9-400e-9f62-6aa63f3507be" for REDACTED at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|b7a50709] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_show as JSON
2026-06-24T10:42:44 [I|app|b7a50709]   Parameters: {"id"=>"57256280-9fa9-400e-9f62-6aa63f3507be"}
2026-06-24T10:42:44 [E|app|b7a50709] RestClient::NotFound: Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"bc6aecd3-0ba5-4e8e-a0a9-9721d509e9f2"} (GET /candlepin/consumers/57256280-9fa9-400e-9f62-6aa63f3507be)
 b7a50709 | Body: {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"bc6aecd3-0ba5-4e8e-a0a9-9721d509e9f2"}
 b7a50709 | 
2026-06-24T10:42:44 [E|app|b7a50709] /usr/share/gems/gems/katello-4.19.2/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:316:in `block in find_host'
2026-06-24T10:42:44 [I|app|b7a50709]   Rendered text template (Duration: 0.0ms | Allocations: 1)
2026-06-24T10:42:44 [I|app|b7a50709] Completed 404 Not Found in 23ms (Views: 0.4ms | ActiveRecord: 1.3ms | Allocations: 6725)
2026-06-24T10:42:44 [I|app|abec04f8] Started GET "/rhsm/consumers/57256280-9fa9-400e-9f62-6aa63f3507be/content_overrides" for REDACTED at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|abec04f8] Processing by Katello::Api::Rhsm::CandlepinProxiesController#get as JSON
2026-06-24T10:42:44 [I|app|abec04f8]   Parameters: {"id"=>"57256280-9fa9-400e-9f62-6aa63f3507be"}
2026-06-24T10:42:44 [E|app|abec04f8] RestClient::NotFound: 404 Not Found
 abec04f8 | Body: {"displayMessage":"Unit with ID \"57256280-9fa9-400e-9f62-6aa63f3507be\" could not be found.","requestUuid":"9bb3fe83-20bd-4de8-8ee6-222427c68ba8"}
 abec04f8 | 
2026-06-24T10:42:44 [E|app|abec04f8] /usr/share/gems/gems/katello-4.19.2/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:82:in `get'
2026-06-24T10:42:44 [I|app|abec04f8]   Rendered text template (Duration: 0.0ms | Allocations: 1)
2026-06-24T10:42:44 [I|app|abec04f8] Completed 404 Not Found in 23ms (Views: 0.3ms | ActiveRecord: 1.2ms | Allocations: 5888)
2026-06-24T10:42:44 [I|app|eaff8072] Started GET "/rhsm/status" for REDACTED at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|eaff8072] Processing by Katello::Api::Rhsm::CandlepinProxiesController#server_status as JSON
2026-06-24T10:42:44 [I|app|eaff8072] Completed 200 OK in 19ms (Views: 0.2ms | ActiveRecord: 1.3ms | Allocations: 6050)
2026-06-24T10:42:46 [I|app|2b3c061a] Started GET "/notification_recipients" for REDACTED at 2026-06-24 10:42:46 -0400
2026-06-24T10:42:46 [I|app|2b3c061a] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:42:46 [I|app|2b3c061a] Completed 200 OK in 4ms (Views: 0.1ms | ActiveRecord: 0.6ms | Allocations: 2049)
2026-06-24T10:42:48 [I|app|3f4efdee] Started PUT "/rhsm/consumers/57256280-9fa9-400e-9f62-6aa63f3507be/profiles" for REDACTED at 2026-06-24 10:42:48 -0400
2026-06-24T10:42:48 [I|app|3f4efdee] Processing by Katello::Api::Rhsm::CandlepinDynflowProxyController#upload_profiles as JSON
2026-06-24T10:42:48 [I|app|3f4efdee]   Parameters: {"id"=>"57256280-9fa9-400e-9f62-6aa63f3507be"}
2026-06-24T10:42:48 [E|app|3f4efdee] RestClient::NotFound: Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"4d26b017-b031-449a-a3a0-9e2ba3ff9a89"} (GET /candlepin/consumers/57256280-9fa9-400e-9f62-6aa63f3507be)
 3f4efdee | Body: {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"4d26b017-b031-449a-a3a0-9e2ba3ff9a89"}
 3f4efdee | 
2026-06-24T10:42:48 [E|app|3f4efdee] /usr/share/gems/gems/katello-4.19.2/app/controllers/katello/api/rhsm/candlepin_dynflow_proxy_controller.rb:48:in `block in find_host'
2026-06-24T10:42:48 [I|app|3f4efdee]   Rendered text template (Duration: 0.0ms | Allocations: 5)
2026-06-24T10:42:48 [I|app|3f4efdee] Completed 404 Not Found in 26ms (Views: 1.7ms | ActiveRecord: 2.1ms | Allocations: 7126)
2026-06-24T10:42:48 [I|app|765d82bc] Started GET "/rhsm/consumers/57256280-9fa9-400e-9f62-6aa63f3507be" for REDACTED at 2026-06-24 10:42:48 -0400
2026-06-24T10:42:48 [I|app|765d82bc] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_show as JSON
2026-06-24T10:42:48 [I|app|765d82bc]   Parameters: {"id"=>"57256280-9fa9-400e-9f62-6aa63f3507be"}
2026-06-24T10:42:48 [E|app|765d82bc] RestClient::NotFound: Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"edbc3a46-11cc-4554-bb6e-31dc16cae8d9"} (GET /candlepin/consumers/57256280-9fa9-400e-9f62-6aa63f3507be)
 765d82bc | Body: {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"edbc3a46-11cc-4554-bb6e-31dc16cae8d9"}
 765d82bc | 
2026-06-24T10:42:48 [E|app|765d82bc] /usr/share/gems/gems/katello-4.19.2/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:316:in `block in find_host'
2026-06-24T10:42:48 [I|app|765d82bc]   Rendered text template (Duration: 0.0ms | Allocations: 2)
2026-06-24T10:42:48 [I|app|765d82bc] Completed 404 Not Found in 24ms (Views: 1.6ms | ActiveRecord: 1.9ms | Allocations: 7046)
2026-06-24T10:42:56 [I|app|1273e51b] Started GET "/notification_recipients" for REDACTED at 2026-06-24 10:42:56 -0400
2026-06-24T10:42:56 [I|app|1273e51b] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:42:56 [I|app|1273e51b] Completed 200 OK in 4ms (Views: 0.1ms | ActiveRecord: 0.5ms | Allocations: 2165)
2026-06-24T10:43:06 [I|app|aa447e51] Started GET "/notification_recipients" for REDACTED at 2026-06-24 10:43:06 -0400
2026-06-24T10:43:06 [I|app|aa447e51] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:43:06 [I|app|aa447e51] Completed 200 OK in 4ms (Views: 0.1ms | ActiveRecord: 0.6ms | Allocations: 1963)
2026-06-24T10:43:08 [I|app|914b162e] Started PATCH "/hosts/598" for REDACTED at 2026-06-24 10:43:08 -0400
2026-06-24T10:43:08 [I|app|914b162e] Processing by HostsController#update as */*
2026-06-24T10:43:08 [I|app|914b162e]   Parameters: {"authenticity_token"=>"REDACTED", "host"=>{"name"=>"lnxtestb5", "hostgroup_id"=>"15", "content_facet_attributes"=>{"content_source_id"=>"", "lifecycle_environment_id"=>"3", "content_view_id"=>"25"}, "managed"=>"true", "progress_report_id"=>"[FILTERED]", "type"=>"Host::Managed", "interfaces_attributes"=>{"0"=>{"_destroy"=>"0", "mac"=>"00:50:56:ab:c8:c1", "identifier"=>"ens192", "name"=>"lnxtestb5", "domain_id"=>"2", "subnet_id"=>"4", "ip"=>"REDACTED", "ip6"=>"", "managed"=>"1", "primary"=>"1", "provision"=>"1", "execution"=>"0", "tag"=>"", "attached_to"=>"", "id"=>"1194"}}, "architecture_id"=>"1", "operatingsystem_id"=>"22", "build"=>"1", "medium_id"=>"23", "ptable_id"=>"181", "pxe_loader"=>"None", "disk"=>"", "root_pass"=>"[FILTERED]", "is_owned_by"=>"47-Users", "enabled"=>"1", "model_id"=>"9", "comment"=>"", "overwrite"=>"false"}, "media_selector"=>"install_media", "id"=>"598"}
2026-06-24T10:43:08 [I|app|914b162e]   Rendered common/404.html.erb within layouts/application (Duration: 0.9ms | Allocations: 562)
2026-06-24T10:43:08 [I|app|914b162e]   Rendered layouts/base.html.erb (Duration: 1.8ms | Allocations: 2280)
2026-06-24T10:43:08 [I|app|914b162e]   Rendered layout layouts/application.html.erb (Duration: 3.0ms | Allocations: 3130)
2026-06-24T10:43:08 [I|app|914b162e] Completed 404 Not Found in 11ms (Views: 3.7ms | ActiveRecord: 1.0ms | Allocations: 7285)
2026-06-24T10:43:16 [I|app|8af72708] Started GET "/notification_recipients" for REDACTED at 2026-06-24 10:43:16 -0400
2026-06-24T10:43:16 [I|app|8af72708] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:43:16 [I|app|8af72708] Completed 200 OK in 5ms (Views: 0.1ms | ActiveRecord: 0.5ms | Allocations: 1972)
2026-06-24T10:43:26 [I|app|c2a9a644] Started GET "/notification_recipients" for REDACTED at 2026-06-24 10:43:26 -0400

This process worked for us on Foreman 3.16 however this broke for us when we upgraded to Foreman 3.17 and after upgrading to Foreman 3.18 this issue is still occurring

1 Like
2

Hi @jakebr, where exactly does the “Host not found” error appear? Instead of the hosts Overview page that should appear directly after provisioning the host?

3

Yes, the host page is not shown and that host returns to the discovered host page and it is not being provisioned

4

This sounds to me like the host isn’t even created. looking at the Foreman production log, are there any “Host::Base () create event”entries?

What OS do you try to provision?

And just to be sure: Form what you describe you are using Discovery in PXEless mode (i.e. with kexec). Is the kexec template assigned to the OS you want to provision?

5

Yes we are using discovery in PXEless mode with kexec

From my log it does appear the host is being created at this point and the subnet information is gathered as well

We are provisioning a RHEL 9 host

2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on name mac005056abc8c1
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on last_compile 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on root_pass 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on architecture_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on operatingsystem_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on ptable_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on medium_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on build false
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on comment 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on disk 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on installed_at 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on model_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on hostgroup_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on owner_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on owner_type 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on enabled true
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on puppet_ca_proxy_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on managed false
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on use_image 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on image_file 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on uuid 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on compute_resource_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on puppet_proxy_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on certname 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on image_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on organization_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on location_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on otp 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on realm_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on compute_profile_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on provision_method 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on grub_pass 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on discovery_rule_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on global_status 0
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on lookup_value_matcher 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on pxe_loader 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on initiated_at 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on build_errors 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on openscap_proxy_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on creator_id 1
2026-06-24T10:42:02 [I|app|5839feae] Import facts for 'mac005056abc8c1' completed. Added: 308, Updated: 0, Deleted 0 facts
2026-06-24T10:42:02 [I|aud|5839feae] Nic::Managed (1194) update event on mac , 00:50:56:ab:c8:c1
2026-06-24T10:42:02 [I|aud|5839feae] Nic::Managed (1194) update event on identifier , ens192
2026-06-24T10:42:02 [I|app|5839feae] Detected IPv4 subnet: REDACTED/REDACTED with taxonomy ["REDACTED"]/["Default Location", "REDACTED", "Unknown"]

For some additional context we run an ipv4 only network

6

While the log says the host is being created the host is still sitting in the discovery page so the host is never actually being created

7

Thanks for the information so far. I will take a deeper look next week.

1 Like
8

Hey @jakebr, sorry, it took me some time to get back to this.

Maybe we should break this down into smaller bread crumbs. I think the problems you face are separate issues.

Regarding the “404 Not Found” on the Host Overview Page

I could not reproduce the “404 Not found” error so i will need further information to dig deeper. There are two things in the first log that stand out to me:

  • When clicking on Submit to start provisioning once you customized your host, there should be a Started PATCH "/discovered_hosts/xy” followed by Host::Base and Katello::Host::ContentFacet entries in the log. The Started PATCH "/hosts/598" statement at the end of the first log must have been logged later on and the actual “transformation” of the discovered host to a managed host must appear earlier in the logs.
  • The logged requests to /rhsm/consumers/ are unrelated to the provisioning of the discovered host. These come from subscription-manager actions. To be more precisely, they seem to be from a host that was/is registered to Foreman but got deleted from Candlepin. Running subscription-manager refresh on such a host should cause these 404 errors from the /rhsm/consumers/ endpoints.

Could you once more provision a discovered host and provide a redacted log that starts before you click on Customize Host and ends after you clicked on Submit after customizing hour host? That should give us a log that includes the host creation and the “transformation” of the discovered host to a managed host.

The empty “IPv4 Subnet” field

When customizing a host, there seems to be a glitch with prefilling the fields on the Interface edit form. In my recent tests, this was not the “IPv4 Subnet” field but the “Domain” field. I had to manually fill it and that caused the “IPv4 Subnet” and “IPv4 Address” to be cleared.

The host stays in the Discovery menu

For this we can do further investigation in the log of the FDI (assuming that the kexec call is triggered and reaches the host):

  1. On the host to discover, at the bottom of the Discovery Menu, there is a ribbon <SSH>. Select that, enter a password for the root account on the host and select Set and Enable to enable SSH on the host.
  2. From your Foreman Server connect to the host using SSH with ssh root@<IP_OF_MY_HOST_TO_DISCOVER>.
  3. Start discovery-debug and scroll down to the bottom.

Somewhere you should see a statement like Initiated kexec provisioning. Please share this and the lines above and below.

9

Hey @goarsna thank you for taking your time to look deeper into this, I really appreciate it

Regarding the “404 Not Found” on the Host Overview Page

I ran this upgrade in one environment with my system administrator on a VM with snapshots but we reverted our snapshot after we encountered build issues however I have the full log so I can provide the previous lines of the log from when Customize Host is selected

2026-06-24T10:40:49 [I|app|11590a77] Started GET "/new/hosts" for REDACTEDIP at 2026-06-24 10:40:49 -0400
2026-06-24T10:40:49 [I|app|11590a77] Processing by ReactController#index as HTML
2026-06-24T10:40:49 [I|app|11590a77]   Rendered react/index.html.erb within layouts/react_application (Duration: 0.3ms | Allocations: 164)
2026-06-24T10:40:49 [I|app|11590a77]   Rendered layouts/base.html.erb (Duration: 13.7ms | Allocations: 21947)
2026-06-24T10:40:49 [I|app|11590a77]   Rendered layout layouts/react_application.html.erb (Duration: 14.5ms | Allocations: 22510)
2026-06-24T10:40:49 [I|app|11590a77] Completed 200 OK in 23ms (Views: 16.6ms | ActiveRecord: 1.5ms | Allocations: 26670)
2026-06-24T10:40:53 [I|app|afb778e3] Started GET "/new/hosts/auto_complete_search?search=" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|777c74e3] Started GET "/api/v2/organizations?per_page=all" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|e42f7e1d] Started GET "/notification_recipients" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|7702dd57] Started GET "/api/v2/locations?per_page=all" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|afb778e3] Processing by HostsController#auto_complete_search as JSON
2026-06-24T10:40:53 [I|app|afb778e3]   Parameters: {"search"=>""}
2026-06-24T10:40:53 [I|app|777c74e3] Processing by Api::V2::OrganizationsController#index as JSON
2026-06-24T10:40:53 [I|app|777c74e3]   Parameters: {"per_page"=>"all", "apiv"=>"v2"}
2026-06-24T10:40:53 [I|app|7702dd57] Processing by Api::V2::LocationsController#index as JSON
2026-06-24T10:40:53 [I|app|e42f7e1d] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:40:53 [I|app|7702dd57]   Parameters: {"per_page"=>"all", "apiv"=>"v2"}
2026-06-24T10:40:53 [I|app|928377b9] Started GET "/api/users?per_page=all" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|610b86ba] Started GET "/api/v2/hostgroups?per_page=all" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|222d97f8] Started GET "/api/usergroups?per_page=all" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|7741bfcb] Started GET "/api/hosts?include_permissions=true&search=" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|222d97f8] Processing by Api::V2::UsergroupsController#index as JSON
2026-06-24T10:40:53 [I|app|222d97f8]   Parameters: {"per_page"=>"all", "apiv"=>"v2"}
2026-06-24T10:40:53 [I|app|7741bfcb] Processing by Api::V2::HostsController#index as JSON
2026-06-24T10:40:53 [I|app|7741bfcb]   Parameters: {"include_permissions"=>"true", "search"=>"", "apiv"=>"v2"}
2026-06-24T10:40:53 [I|app|928377b9] Processing by Api::V2::UsersController#index as JSON
2026-06-24T10:40:53 [I|app|928377b9]   Parameters: {"per_page"=>"all", "apiv"=>"v2"}
2026-06-24T10:40:53 [I|app|9311626a] Started GET "/api/v2/current_user" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|e42f7e1d] Completed 200 OK in 16ms (Views: 0.1ms | ActiveRecord: 5.4ms | Allocations: 7786)
2026-06-24T10:40:53 [I|app|afb778e3] Completed 200 OK in 22ms (Views: 3.9ms | ActiveRecord: 6.5ms | Allocations: 16712)
2026-06-24T10:40:53 [I|app|222d97f8]   Rendered api/v2/usergroups/index.json.rabl within api/v2/layouts/index_layout (Duration: 3.7ms | Allocations: 2240)
2026-06-24T10:40:53 [I|app|222d97f8]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 4.4ms | Allocations: 2851)
2026-06-24T10:40:53 [I|app|222d97f8] Completed 200 OK in 18ms (Views: 6.0ms | ActiveRecord: 7.9ms | Allocations: 9204)
2026-06-24T10:40:53 [I|app|610b86ba] Processing by Api::V2::HostgroupsController#index as JSON
2026-06-24T10:40:53 [I|app|610b86ba]   Parameters: {"per_page"=>"all", "apiv"=>"v2"}
2026-06-24T10:40:53 [I|app|9311626a] Processing by Api::V2::UsersController#show_current as JSON
2026-06-24T10:40:53 [I|app|9311626a]   Parameters: {"apiv"=>"v2"}
2026-06-24T10:40:53 [I|app|7702dd57]   Rendered api/v2/taxonomies/index.json.rabl within api/v2/layouts/index_layout (Duration: 24.3ms | Allocations: 11126)
2026-06-24T10:40:53 [I|app|7702dd57]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 25.2ms | Allocations: 11330)
2026-06-24T10:40:53 [I|app|7702dd57] Completed 200 OK in 36ms (Views: 23.7ms | ActiveRecord: 8.4ms | Allocations: 17346)
2026-06-24T10:40:53 [I|app|777c74e3]   Rendered api/v2/taxonomies/index.json.rabl within api/v2/layouts/index_layout (Duration: 2.0ms | Allocations: 1050)
2026-06-24T10:40:53 [I|app|777c74e3]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 2.3ms | Allocations: 1287)
2026-06-24T10:40:53 [I|app|777c74e3] Completed 200 OK in 37ms (Views: 5.2ms | ActiveRecord: 20.6ms | Allocations: 7811)
2026-06-24T10:40:53 [I|app|9311626a]   Rendered api/v2/users/show.json.rabl (Duration: 28.5ms | Allocations: 18342)
2026-06-24T10:40:53 [I|app|9311626a] Completed 200 OK in 37ms (Views: 17.7ms | ActiveRecord: 15.3ms | Allocations: 22562)
2026-06-24T10:40:53 [I|app|928377b9]   Rendered api/v2/users/index.json.rabl within api/v2/layouts/index_layout (Duration: 137.4ms | Allocations: 105935)
2026-06-24T10:40:53 [I|app|928377b9]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 138.6ms | Allocations: 106650)
2026-06-24T10:40:53 [I|app|928377b9] Completed 200 OK in 157ms (Views: 120.6ms | ActiveRecord: 19.7ms | Allocations: 121534)
2026-06-24T10:40:53 [I|app|0bfb0559] Started GET "/api/v2/users/47/table_preferences/hosts" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|0bfb0559] Processing by Api::V2::TablePreferencesController#show as JSON
2026-06-24T10:40:53 [I|app|0bfb0559]   Parameters: {"apiv"=>"v2", "user_id"=>"47", "id"=>"hosts"}
2026-06-24T10:40:53 [I|app|0bfb0559] ActiveRecord::RecordNotFound (ActiveRecord::RecordNotFound)
2026-06-24T10:40:53 [I|app|0bfb0559]   Rendered api/v2/errors/not_found.json.rabl within api/v2/layouts/error_layout (Duration: 0.3ms | Allocations: 169)
2026-06-24T10:40:53 [I|app|0bfb0559]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 0.4ms | Allocations: 264)
2026-06-24T10:40:53 [I|app|0bfb0559] Completed 404 Not Found in 6ms (Views: 0.7ms | ActiveRecord: 1.0ms | Allocations: 2954)
2026-06-24T10:40:53 [I|app|610b86ba]   Rendered api/v2/hostgroups/index.json.rabl within api/v2/layouts/index_layout (Duration: 170.6ms | Allocations: 105180)
2026-06-24T10:40:53 [I|app|610b86ba]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 171.6ms | Allocations: 105772)
2026-06-24T10:40:53 [I|app|610b86ba] Completed 200 OK in 190ms (Views: 141.1ms | ActiveRecord: 37.2ms | Allocations: 116268)
2026-06-24T10:40:53 [I|app|43f96e4d] Started GET "/api/v2/remote_execution_features" for REDACTEDIP at 2026-06-24 10:40:53 -0400
2026-06-24T10:40:53 [I|app|43f96e4d] Processing by Api::V2::RemoteExecutionFeaturesController#index as JSON
2026-06-24T10:40:53 [I|app|43f96e4d]   Parameters: {"apiv"=>"v2"}
2026-06-24T10:40:54 [I|app|43f96e4d]   Rendered /usr/share/gems/gems/foreman_remote_execution-16.3.1/app/views/api/v2/remote_execution_features/index.json.rabl within api/v2/layouts/index_layout (Duration: 36.1ms | Allocations: 28317)
2026-06-24T10:40:54 [I|app|43f96e4d]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 36.8ms | Allocations: 28673)
2026-06-24T10:40:54 [I|app|43f96e4d] Completed 200 OK in 42ms (Views: 31.4ms | ActiveRecord: 6.4ms | Allocations: 31210)
2026-06-24T10:40:54 [I|app|7741bfcb]   Rendered api/v2/hosts/index.json.rabl within api/v2/layouts/index_layout (Duration: 975.3ms | Allocations: 818483)
2026-06-24T10:40:54 [I|app|7741bfcb]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 976.6ms | Allocations: 819192)
2026-06-24T10:40:54 [I|app|7741bfcb] Completed 200 OK in 1062ms (Views: 874.5ms | ActiveRecord: 126.2ms | Allocations: 872932)
2026-06-24T10:41:03 [I|app|f84cbf11] Started GET "/notification_recipients" for REDACTEDIP at 2026-06-24 10:41:03 -0400
2026-06-24T10:41:03 [I|app|f84cbf11] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:41:03 [I|app|f84cbf11] Completed 200 OK in 5ms (Views: 0.1ms | ActiveRecord: 0.6ms | Allocations: 2297)
2026-06-24T10:42:01 [I|app|5839feae] Started POST "/api/v2/discovered_hosts/facts" for 10.245.173.238 at 2026-06-24 10:42:01 -0400
2026-06-24T10:42:01 [I|app|5839feae] Processing by Api::V2::DiscoveredHostsController#facts as JSON
2026-06-24T10:42:01 [I|app|5839feae]   Parameters: {"facts"=>"[FILTERED]", "apiv"=>"v2", "discovered_host"=>{"facts"=>"[FILTERED]"}}
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on mac 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on ip 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on type Nic::Managed
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on name mac005056abc8c1
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on host_id 598
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on subnet_id 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on domain_id 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on attrs {}
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on provider 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on username 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on password [redacted]
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on virtual false
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on link true
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on identifier 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on tag 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on attached_to 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on managed true
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on mode balance-rr
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on attached_devices 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on bond_options 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on primary true
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on provision true
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on compute_attributes {}
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on execution false
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on ip6 
2026-06-24T10:42:01 [I|aud|5839feae] Nic::Managed (1194) create event on subnet6_id 
2026-06-24T10:42:01 [I|aud|5839feae] Host::Base (598) create event on name mac005056abc8c1

The empty “IPv4 Subnet” field

This is interesting to note, I will run this with my system admin and get back to you with logs from our discovery-debug

10

Thanks @jakebr for the logs. And you’re welcome. :slight_smile:

Ok, we now have the logs from where the host is being created, perfect. But I have to correct myself: This is not the point in time you clicked on Customize Host, it happens by the fact upload from the host to Foreman Server. The log statement is Started POST “/api/v2/discovered_hosts/facts” at timestamp 2026-06-24T10:42:01 in the last log.

What is still missing in the logs is the part from 10:42:02 to 10:42:44. Could you also provide this part? This should contain

  • a Started GET "/discovered_hosts/<MAC_ADDRESS>/edit..." (the point in time where the Customize Host button is clicked),
  • a Started PATCH “/discovered_hosts/598” (the point in time the host type is changed from Host::Discovered to Host::Managed) followed by some Host::Base (598) update log statements, and
  • a redirect to the newly created host like Redirected to https://foreman.example.com/new/hosts/my_new_hosts_fqdn

Somewhere in here I hope to see the error :slight_smile:

11

Thanks @goarsna

I’ve included the missing segment of the log below:

2026-06-24T10:42:02 [I|app|5839feae] Import facts for 'mac005056abc8c1' completed. Added: 308, Updated: 0, Deleted 0 facts
2026-06-24T10:42:02 [I|aud|5839feae] Nic::Managed (1194) update event on mac , 00:50:56:ab:c8:c1
2026-06-24T10:42:02 [I|aud|5839feae] Nic::Managed (1194) update event on identifier , ens192
2026-06-24T10:42:02 [I|app|5839feae] Detected IPv4 subnet: REDACTED-SUBNET with taxonomy ["REDACTED-ORG"]/["Default Location", "REDACTED-LOCATION", "Unknown"]
2026-06-24T10:42:02 [I|app|5839feae] Assigned location: Default Location/Unknown
2026-06-24T10:42:02 [I|app|5839feae] Assigned organization: REDACTED-ORG
2026-06-24T10:42:02 [I|aud|5839feae] Host::Base (598) update event on model_id , 9
2026-06-24T10:42:02 [I|aud|5839feae] Host::Base (598) update event on owner_id , 1
2026-06-24T10:42:02 [I|aud|5839feae] Host::Base (598) update event on owner_type , User
2026-06-24T10:42:02 [I|aud|5839feae] Host::Base (598) update event on organization_id , 3
2026-06-24T10:42:02 [I|aud|5839feae] Host::Base (598) update event on location_id , 4
2026-06-24T10:42:02 [I|aud|5839feae] Nic::Managed (1194) update event on subnet_id , 4
2026-06-24T10:42:02 [I|app|5839feae] Completed 201 Created in 762ms (Views: 2.6ms | ActiveRecord: 206.4ms | Allocations: 413102)
2026-06-24T10:42:10 [I|app|98f10483] Started GET "/discovered_hosts" for REDACTED-IP-1 at 2026-06-24 10:42:10 -0400
2026-06-24T10:42:10 [I|app|98f10483] Processing by DiscoveredHostsController#index as HTML
2026-06-24T10:42:10 [I|app|98f10483]   Rendered /usr/share/gems/gems/foreman_discovery-26.1.1/app/views/discovered_hosts/index.html.erb within layouts/application (Duration: 36.1ms | Allocations: 49142)
2026-06-24T10:42:10 [I|app|98f10483] e[1;32mDeface: [WARNING]e[0m No :original defined for 'styles', you should change its definition to include:
 98f10483 |  :original => 'fae5e7e8dc59947294caf0f77e3b9ddab650ba27' 
2026-06-24T10:42:10 [I|app|98f10483]   Rendered layouts/base.html.erb (Duration: 21.8ms | Allocations: 30258)
2026-06-24T10:42:10 [I|app|98f10483]   Rendered layout layouts/application.html.erb (Duration: 59.8ms | Allocations: 80841)
2026-06-24T10:42:10 [I|app|98f10483] Completed 200 OK in 87ms (Views: 61.9ms | ActiveRecord: 7.8ms | Allocations: 96189)
2026-06-24T10:42:13 [I|app|632d0298] Started GET "/notification_recipients" for REDACTED-IP-1 at 2026-06-24 10:42:13 -0400
2026-06-24T10:42:13 [I|app|632d0298] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:42:13 [I|app|632d0298] Completed 200 OK in 28ms (Views: 0.1ms | ActiveRecord: 10.6ms | Allocations: 16734)
2026-06-24T10:42:13 [I|app|83364a41] Started GET "/discovered_hosts/auto_complete_search?search=" for REDACTED-IP-1 at 2026-06-24 10:42:13 -0400
2026-06-24T10:42:13 [I|app|83364a41] Processing by DiscoveredHostsController#auto_complete_search as JSON
2026-06-24T10:42:13 [I|app|83364a41]   Parameters: {"search"=>""}
2026-06-24T10:42:13 [I|app|83364a41] Completed 200 OK in 7ms (Views: 0.8ms | ActiveRecord: 0.8ms | Allocations: 4245)
2026-06-24T10:42:23 [I|app|0d1ab28b] Started GET "/discovered_hosts/mac005056abc8c1/edit?host%5Bhostgroup_id%5D=15&host%5Borganization_id%5D=3&host%5Blocation_id%5D=5" for REDACTED-IP-1 at 2026-06-24 10:42:23 -0400
2026-06-24T10:42:23 [I|app|0d1ab28b] Processing by DiscoveredHostsController#edit as HTML
2026-06-24T10:42:23 [I|app|0d1ab28b]   Parameters: {"host"=>{"hostgroup_id"=>"15", "organization_id"=>"3", "location_id"=>"5"}, "id"=>"mac005056abc8c1"}
2026-06-24T10:42:23 [I|app|0d1ab28b] e[1;32mDeface: [WARNING]e[0m No :original defined for 'add_execution_interface_js', you should change its definition to include:
 0d1ab28b |  :original => '9a7f73e0d2fd3f998c3a832df9e7aadcab86d4ad' 
2026-06-24T10:42:23 [I|app|0d1ab28b] e[1;32mDeface: [WARNING]e[0m No :original defined for 'ansible_roles_tab', you should change its definition to include:
 0d1ab28b |  :original => '8c63c5282ee199068bfd65475e13066ee981089a' 
2026-06-24T10:42:23 [I|app|0d1ab28b] e[1;32mDeface: [WARNING]e[0m No :original defined for 'ansible_roles_tab_content', you should change its definition to include:
 0d1ab28b |  :original => '9a7f73e0d2fd3f998c3a832df9e7aadcab86d4ad' 
2026-06-24T10:42:23 [I|app|0d1ab28b] e[1;32mDeface: [WARNING]e[0m No :original defined for 'add_execution_interface', you should change its definition to include:
 0d1ab28b |  :original => 'dbe1da973bfb251a74b994e7ae11bc8d11502c97' 
2026-06-24T10:42:23 [I|app|0d1ab28b] e[1;32mDeface: [WARNING]e[0m No :original defined for 'hosts_select_media_type', you should change its definition to include:
 0d1ab28b |  :original => '2d9ab0e830d51c8cd4271ea3f3ade95cfbf3de75' 
2026-06-24T10:42:23 [I|app|0d1ab28b] e[1;32mDeface: [WARNING]e[0m No :original defined for 'hosts_select_synced_content', you should change its definition to include:
 0d1ab28b |  :original => '2d9ab0e830d51c8cd4271ea3f3ade95cfbf3de75' 
2026-06-24T10:42:23 [I|app|0d1ab28b]   Rendered /usr/share/gems/gems/foreman_discovery-26.1.1/app/views/discovered_hosts/edit.html.erb within layouts/application (Duration: 319.1ms | Allocations: 278114)
2026-06-24T10:42:23 [I|app|0d1ab28b] e[1;32mDeface: [WARNING]e[0m No :original defined for 'styles', you should change its definition to include:
 0d1ab28b |  :original => 'fae5e7e8dc59947294caf0f77e3b9ddab650ba27' 
2026-06-24T10:42:23 [I|app|0d1ab28b]   Rendered layouts/base.html.erb (Duration: 30.0ms | Allocations: 42057)
2026-06-24T10:42:23 [I|app|0d1ab28b]   Rendered layout layouts/application.html.erb (Duration: 351.5ms | Allocations: 321498)
2026-06-24T10:42:23 [I|app|0d1ab28b] Completed 200 OK in 413ms (Views: 323.8ms | ActiveRecord: 49.3ms | Allocations: 353868)
2026-06-24T10:42:26 [I|app|c658365c] Started GET "/notification_recipients" for REDACTED-IP-1 at 2026-06-24 10:42:26 -0400
2026-06-24T10:42:26 [I|app|c658365c] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:42:26 [I|app|c658365c] Completed 200 OK in 4ms (Views: 0.1ms | ActiveRecord: 0.6ms | Allocations: 2110)
2026-06-24T10:42:28 [I|app|7ac71bbf] Started POST "/hosts/process_hostgroup" for REDACTED-IP-1 at 2026-06-24 10:42:28 -0400
2026-06-24T10:42:28 [I|app|7ac71bbf] Processing by HostsController#process_hostgroup as */*
2026-06-24T10:42:28 [I|app|7ac71bbf]   Parameters: {"authenticity_token"=>"REDACTED", "host"=>{"name"=>"mac005056abc8c1", "organization_id"=>"3", "location_id"=>"5", "hostgroup_id"=>"15", "content_facet_attributes"=>{"content_source_id"=>"", "lifecycle_environment_id"=>"3", "content_view_id"=>"25"}, "managed"=>"true", "progress_report_id"=>"[FILTERED]", "type"=>"Host::Managed", "interfaces_attributes"=>{"0"=>{"_destroy"=>"0", "mac"=>"00:50:56:ab:c8:c1", "identifier"=>"ens192", "name"=>"mac005056abc8c1", "domain_id"=>"2", "subnet_id"=>"4", "ip"=>"REDACTED-IP-2", "ip6"=>"", "managed"=>"1", "primary"=>"1", "provision"=>"1", "execution"=>"0", "tag"=>"", "attached_to"=>"", "id"=>"1194"}}, "architecture_id"=>"1", "operatingsystem_id"=>"22", "build"=>"1", "medium_id"=>"", "ptable_id"=>"181", "pxe_loader"=>"None", "disk"=>"", "is_owned_by"=>"", "enabled"=>"1", "model_id"=>"9", "comment"=>"", "overwrite"=>"false", "id"=>"598"}, "fakepassword"=>"[FILTERED]"}
2026-06-24T10:42:28 [I|app|6df498ac] Started GET "/api/hostgroups/15" for REDACTED-IP-1 at 2026-06-24 10:42:28 -0400
2026-06-24T10:42:28 [I|app|6df498ac] Processing by Api::V2::HostgroupsController#show as JSON
2026-06-24T10:42:28 [I|app|6df498ac]   Parameters: {"apiv"=>"v2", "id"=>"15"}
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on host_id 598
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on uuid 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on kickstart_repository_id 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on content_source_id 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on installable_security_errata_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on installable_enhancement_errata_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on installable_bugfix_errata_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on applicable_rpm_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on upgradable_rpm_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on applicable_module_stream_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on upgradable_module_stream_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on applicable_deb_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on upgradable_deb_count 0
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on bootc_booted_image 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on bootc_booted_digest 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on bootc_available_image 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on bootc_available_digest 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on bootc_staged_image 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on bootc_staged_digest 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on bootc_rollback_image 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on bootc_rollback_digest 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on manifest_entity_type 
2026-06-24T10:42:28 [I|aud|7ac71bbf] Katello::Host::ContentFacet (375) create event on manifest_entity_id 
2026-06-24T10:42:28 [I|app|6df498ac]   Rendered /usr/share/gems/gems/katello-4.19.2/app/views/katello/api/v2/hostgroups_extensions/show.json.rabl (Duration: 43.9ms | Allocations: 32088)
2026-06-24T10:42:28 [I|app|6df498ac] Completed 200 OK in 51ms (Views: 38.4ms | ActiveRecord: 7.3ms | Allocations: 35913)
2026-06-24T10:42:28 [I|app|308deac7] Started GET "/ansible/ui_ansible_roles?search=&resource_id=598&resource_name=Host" for REDACTED-IP-1 at 2026-06-24 10:42:28 -0400
2026-06-24T10:42:28 [I|app|308deac7] Processing by UIAnsibleRolesController#index as JSON
2026-06-24T10:42:28 [I|app|308deac7]   Parameters: {"search"=>"", "resource_id"=>"598", "resource_name"=>"Host"}
2026-06-24T10:42:28 [I|app|308deac7]   Rendered /usr/share/gems/gems/foreman_ansible-17.0.2/app/views/ui_ansible_roles/index.json.rabl within api/v2/layouts/index_layout (Duration: 9.6ms | Allocations: 13269)
2026-06-24T10:42:28 [I|app|308deac7]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 10.3ms | Allocations: 13682)
2026-06-24T10:42:28 [I|app|308deac7] Completed 200 OK in 18ms (Views: 12.8ms | ActiveRecord: 0.8ms | Allocations: 17175)
2026-06-24T10:42:28 [I|app|7ac71bbf] Completed 200 OK in 302ms (Views: 214.5ms | ActiveRecord: 37.3ms | Allocations: 279144)
2026-06-24T10:42:28 [I|app|3f845c27] Started GET "/api/hostgroups/15" for REDACTED-IP-1 at 2026-06-24 10:42:28 -0400
2026-06-24T10:42:28 [I|app|3f845c27] Processing by Api::V2::HostgroupsController#show as JSON
2026-06-24T10:42:28 [I|app|3f845c27]   Parameters: {"apiv"=>"v2", "id"=>"15"}
2026-06-24T10:42:28 [I|app|3f845c27]   Rendered /usr/share/gems/gems/katello-4.19.2/app/views/katello/api/v2/hostgroups_extensions/show.json.rabl (Duration: 33.6ms | Allocations: 25371)
2026-06-24T10:42:28 [I|app|3f845c27] Completed 200 OK in 43ms (Views: 31.9ms | ActiveRecord: 5.7ms | Allocations: 29666)
2026-06-24T10:42:28 [I|app|7fb5edd2] Started GET "/ansible/ui_ansible_roles?search=&resource_id=598&resource_name=Host" for REDACTED-IP-1 at 2026-06-24 10:42:28 -0400
2026-06-24T10:42:28 [I|app|7fb5edd2] Processing by UIAnsibleRolesController#index as JSON
2026-06-24T10:42:28 [I|app|7fb5edd2]   Parameters: {"search"=>"", "resource_id"=>"598", "resource_name"=>"Host"}
2026-06-24T10:42:28 [I|app|7fb5edd2]   Rendered /usr/share/gems/gems/foreman_ansible-17.0.2/app/views/ui_ansible_roles/index.json.rabl within api/v2/layouts/index_layout (Duration: 9.2ms | Allocations: 13262)
2026-06-24T10:42:28 [I|app|7fb5edd2]   Rendered layout api/v2/layouts/index_layout.json.erb (Duration: 9.9ms | Allocations: 13671)
2026-06-24T10:42:28 [I|app|7fb5edd2] Completed 200 OK in 18ms (Views: 12.3ms | ActiveRecord: 0.8ms | Allocations: 16923)
2026-06-24T10:42:36 [I|app|8fce4b5c] Started GET "/notification_recipients" for REDACTED-IP-1 at 2026-06-24 10:42:36 -0400
2026-06-24T10:42:36 [I|app|8fce4b5c] Processing by NotificationRecipientsController#index as JSON
2026-06-24T10:42:36 [I|app|8fce4b5c] Completed 200 OK in 6ms (Views: 0.1ms | ActiveRecord: 0.9ms | Allocations: 2165)
2026-06-24T10:42:44 [I|app|16f9d0e4] Started GET "/rhsm/" for REDACTED-IP-3 at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|16f9d0e4] Processing by Katello::Api::V2::RootController#rhsm_resource_list as JSON
2026-06-24T10:42:44 [I|app|16f9d0e4]   Parameters: {"root"=>{}}
2026-06-24T10:42:44 [I|app|16f9d0e4] Completed 200 OK in 4ms (Views: 0.6ms | ActiveRecord: 1.5ms | Allocations: 1598)
2026-06-24T10:42:44 [I|app|56dc8d0e] Started GET "/rhsm/consumers/57256280-9fa9-400e-9f62-6aa63f3507be/certificates/serials" for REDACTED-IP-3 at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|56dc8d0e] Processing by Katello::Api::Rhsm::CandlepinProxiesController#serials as JSON
2026-06-24T10:42:44 [I|app|56dc8d0e]   Parameters: {"id"=>"57256280-9fa9-400e-9f62-6aa63f3507be"}
2026-06-24T10:42:44 [E|app|56dc8d0e] RestClient::NotFound: Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"ede5781f-6b4f-43be-bf2f-3321a9817cba"} (GET /candlepin/consumers/57256280-9fa9-400e-9f62-6aa63f3507be)
 56dc8d0e | Body: {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"ede5781f-6b4f-43be-bf2f-3321a9817cba"}
 56dc8d0e | 
2026-06-24T10:42:44 [E|app|56dc8d0e] /usr/share/gems/gems/katello-4.19.2/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:316:in `block in find_host'
2026-06-24T10:42:44 [I|app|56dc8d0e]   Rendered text template (Duration: 0.0ms | Allocations: 5)
2026-06-24T10:42:44 [I|app|56dc8d0e] Completed 404 Not Found in 54ms (Views: 1.8ms | ActiveRecord: 1.6ms | Allocations: 7055)
2026-06-24T10:42:44 [I|app|b7a50709] Started GET "/rhsm/consumers/57256280-9fa9-400e-9f62-6aa63f3507be" for REDACTED-IP-3 at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|b7a50709] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_show as JSON
2026-06-24T10:42:44 [I|app|b7a50709]   Parameters: {"id"=>"57256280-9fa9-400e-9f62-6aa63f3507be"}
2026-06-24T10:42:44 [E|app|b7a50709] RestClient::NotFound: Katello::Resources::Candlepin::Consumer: 404 Not Found {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"bc6aecd3-0ba5-4e8e-a0a9-9721d509e9f2"} (GET /candlepin/consumers/57256280-9fa9-400e-9f62-6aa63f3507be)
 b7a50709 | Body: {"displayMessage":"Consumer with ID(s) 57256280-9fa9-400e-9f62-6aa63f3507be could not be found.","requestUuid":"bc6aecd3-0ba5-4e8e-a0a9-9721d509e9f2"}
 b7a50709 | 
2026-06-24T10:42:44 [E|app|b7a50709] /usr/share/gems/gems/katello-4.19.2/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:316:in `block in find_host'
2026-06-24T10:42:44 [I|app|b7a50709]   Rendered text template (Duration: 0.0ms | Allocations: 1)
2026-06-24T10:42:44 [I|app|b7a50709] Completed 404 Not Found in 23ms (Views: 0.4ms | ActiveRecord: 1.3ms | Allocations: 6725)
2026-06-24T10:42:44 [I|app|abec04f8] Started GET "/rhsm/consumers/57256280-9fa9-400e-9f62-6aa63f3507be/content_overrides" for REDACTED-IP-3 at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|abec04f8] Processing by Katello::Api::Rhsm::CandlepinProxiesController#get as JSON
2026-06-24T10:42:44 [I|app|abec04f8]   Parameters: {"id"=>"57256280-9fa9-400e-9f62-6aa63f3507be"}
2026-06-24T10:42:44 [E|app|abec04f8] RestClient::NotFound: 404 Not Found
 abec04f8 | Body: {"displayMessage":"Unit with ID \"57256280-9fa9-400e-9f62-6aa63f3507be\" could not be found.","requestUuid":"9bb3fe83-20bd-4de8-8ee6-222427c68ba8"}
 abec04f8 | 
2026-06-24T10:42:44 [E|app|abec04f8] /usr/share/gems/gems/katello-4.19.2/app/controllers/katello/api/rhsm/candlepin_proxies_controller.rb:82:in `get'
2026-06-24T10:42:44 [I|app|abec04f8]   Rendered text template (Duration: 0.0ms | Allocations: 1)
2026-06-24T10:42:44 [I|app|abec04f8] Completed 404 Not Found in 23ms (Views: 0.3ms | ActiveRecord: 1.2ms | Allocations: 5888)
2026-06-24T10:42:44 [I|app|eaff8072] Started GET "/rhsm/status" for REDACTED-IP-3 at 2026-06-24 10:42:44 -0400
2026-06-24T10:42:44 [I|app|eaff8072] Processing by Katello::Api::Rhsm::CandlepinProxiesController#server_status as JSON
2026-06-24T10:42:44 [I|app|eaff8072] Completed 200 OK in 19ms (Views: 0.2ms | ActiveRecord: 1.3ms | Allocations: 6050)

This was the log from our test in 3.17 but we observed the same behavior in 3.18 and I will include my results after I perform another test and acquire the FDI log

12

Thanks @jakebr!

I had a look at the logs again and I have to admit I am somehow clueless…

In your logs, you can see the Katello::Host::ContentFacet (375) create event entries. That is the moment when you click on Customize Host. The moment you click on Submit to actually start provisioning the host, there should be a Started PATCH "/discovered_hosts/598" followed by Host::Base (598) update event entries. So it seems the Submit is not even processed by Foreman. What also surprises me is the Started PATCH “/hosts/598” in the logs later on. This looks like Foreman thinks the host is already a managed host.

Regarding the calls to Candlepin (the Started GET "/rhsm/… entries in the log): I still think these are not related to the current provisioning as it is just about to start. AFAIK these would make sense later on when the host registers itself to Foreman.

a few things you could check in the next test run when the Host not found page from your first screenshot is displayed again:

  • Could you check what address is displayed in the browsers’ address bar?
  • Could you also run hammer host info --id <host_id> on your Foreman Server. You can find the host Id in the parentheses in the Host::Base (xyz) entries. It would be interesting if the host actually exists and if it is a managed host (Managed: in the output of the hammer host info command) and what the subscription status is (Subscription Information: section in the output of the hammer host info command).
  • If there is subscription information displayed, it would be interesting if it is present in Candlepin:
    • Open the rails console using foreman-rake console
    • Run Katello::Resources::Candlepin::Consumer.get('xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx') with the UUID visible in the subscription information.

(cc. @lstejska Do you have a clue what could cause the problems?)

13

I have the same problem.
Provision was working properly for me on foreman 3.16.2/katello 4.18.1
This broke after upgrading to version 3.18.2. I tried upgrading to version 3.19.1 in hopes that it would be fixed, but it didn’t work. I assumed that it was due to my actions to remove the ACD plugin.
I performed a clean new installation of foreman 3.19/katello 4.21 yesterday, but the error occurred again. After performing рhost discovery, it is not possible to perform host provision, and it returns the error same error “Host not found”.