Unable to create host from Discovered Hosts since updating to Foreman 3.0/Katello 4.2

Unable to create host from Discovere Hosts since updating to Foreman 3.0/Katello 4.2. When using the GUI, after hitting submit, the page goes to blank white page. The physical server will remain in Discovered Hosts. This problem has persisted for us through upgrades up to Foreman 3.2/Katello 4.4
Expected outcome:
New host created on physical servers.
Foreman and Proxy versions:
Foreman 3.2.1-1
Katello 4.4.0-1
Foreman and Proxy plugin versions:
foreman-tasks 6.0.1
foreman_discovery 20.0.1
foreman_docker 5.0.0
foreman_puppet 3.0.6
foreman_remote_execution 6.0.0
katello 4.2.1
Distribution and version:
CentOS Linux release 7.9.2009 (Core)
Other relevant data:

Here is some additional information since I’ve not received any responses. This issue is similar to what appears in Provisioning discovered hosts: hostgroup settings not applied - #22 by Thulium-Drake. I have run 3 different scenarios. One using hammer commands, and 2 using the GUI.

When trying to provision via hammer with the following command, the hostgroup is not followed. It is required currently to define a “medium-id” which selects the “All Media” option for “Media Selection” which points to the OS Media name rather than the “Synced Content” option defined in our hostgroup which points the correct “Synced Content” needed in our environment. Prior to Foreman 3.0, we were not required to use the medium-id in our hammer command, and it would follow the hostgroup selection.

  1. running without media-id which should be defined by the hostgroup

hammer discovery provision --name mac9440c9003ff3 --hostgroup-title prod_d_ol_8 --build false --new-name test123.d.filtered

Could not create the host:
  resource have no errors
  1. Running with the media-id defined which does not select the synced content

hammer discovery provision --name mac9440c9003ff3 --hostgroup-title prod_d_ol_8 --build false --new-name test123.d.filtered --medium-id 17

Host tab is correct:

Operating System tab with wrong Media Selection and Media:

When using the GUI, if you “Customize Host” without selecting the hostgroup

Then choose the hostgroup on the host editing page, everything will look correctly for building specifically the Sync Content will be selected, but when you hit submit, it will show a blank page, and a 404 error in the logs:

2022-09-07T11:02:35 [I|app|3b511ed6]   Parameters: {"utf8"=>"✓", "authenticity_token"=>"Ng8eWSPyTr+KzVYPVAnVLr1EIaBp8gSLl5Zs5TzDMkKtEHBm0TsumsjwHdIdv6NKjlTUVtAmaQBoLG6dlSAT6w==", "host"=>{"name"=>"test", "hostgroup_id"=>"274", "content_facet_attributes"=>{"lifecycle_environment_id"=>"4", "content_view_id"=>"53", "content_source_id"=>"8", "kickstart_repository_id"=>"3575"}, "puppet_attributes"=>{"environment_id"=>""}, "managed"=>"true", "progress_report_id"=>"[FILTERED]", "type"=>"Host::Managed", "interfaces_attributes"=>{"0"=>{"_destroy"=>"0", "mac"=>"94:40:c9:00:3f:f3", "identifier"=>"eno1", "name"=>"test", "domain_id"=>"3", "subnet_id"=>"", "ip"=>"", "ip6"=>"", "managed"=>"1", "primary"=>"1", "provision"=>"1", "execution"=>"1", "tag"=>"", "attached_to"=>"", "id"=>"19343"}, "1"=>{"_destroy"=>"0", "mac"=>"94:40:c9:00:3f:f4", "identifier"=>"eno1d1", "name"=>"", "domain_id"=>"", "subnet_id"=>"", "ip"=>"", "ip6"=>"", "managed"=>"0", "primary"=>"0", "provision"=>"0", "execution"=>"0", "tag"=>"", "attached_to"=>"", "id"=>"19344"}, "2"=>{"_destroy"=>"0", "mac"=>"", "identifier"=>"ipmi", "name"=>"", "domain_id"=>"", "subnet_id"=>"", "ip"=>"", "ip6"=>"", "managed"=>"0", "primary"=>"0", "provision"=>"0", "execution"=>"0", "username"=>"", "password"=>"[FILTERED]", "provider"=>"IPMI", "id"=>"19345"}}, "architecture_id"=>"1", "operatingsystem_id"=>"9", "build"=>"1", "medium_id"=>"", "ptable_id"=>"138", "pxe_loader"=>"Grub2 UEFI", "disk"=>"", "root_pass"=>"[FILTERED]", "is_owned_by"=>"1-Usergroups", "enabled"=>"1", "model_id"=>"2", "comment"=>"", "overwrite"=>"false"}, "media_selector"=>"synced_content", "fakepassword"=>"[FILTERED]", "id"=>"7830"}
2022-09-07T11:02:35 [I|app|3b511ed6]   Rendering common/404.html.erb within layouts/application
2022-09-07T11:02:35 [I|app|3b511ed6]   Rendered common/404.html.erb within layouts/application (Duration: 2.8ms | Allocations: 447)
2022-09-07T11:02:35 [I|app|3b511ed6]   Rendered layouts/_application_content.html.erb (Duration: 5.8ms | Allocations: 5750)
2022-09-07T11:02:35 [I|app|3b511ed6]   Rendering layouts/base.html.erb
2022-09-07T11:02:36 [I|app|3b511ed6]   Rendered layouts/base.html.erb (Duration: 10.8ms | Allocations: 7813)
2022-09-07T11:02:36 [I|app|3b511ed6] Completed 404 Not Found in 86ms (Views: 64.8ms | ActiveRecord: 5.0ms | Allocations: 24458)
2022-09-07T11:02:36 [I|app|54f36754] Started POST "/api/v2/discovered_hosts/facts" for at 2022-09-07 11:02:36 -0500
2022-09-07T11:02:36 [I|app|54f36754] Processing by Api::V2::DiscoveredHostsController#facts as JSON
2022-09-07T11:02:36 [I|app|54f36754]   Parameters: {"facts"=>"[FILTERED]", "apiv"=>"v2", "discovered_host"=>{"facts"=>"[FILTERED]"}}
2022-09-07T11:02:36 [I|app|54f36754] Import facts for 'mac9440c9006c1c' completed. Added: 0, Updated: 1, Deleted 0 facts
2022-09-07T11:02:36 [I|app|54f36754] Detected IPv4 subnet: site_d_vlan_116 with taxonomy ["carfax"]/["site d"]
2022-09-07T11:02:36 [I|app|54f36754] Assigned location: site d
2022-09-07T11:02:36 [I|app|54f36754] Assigned organization: "[FILTERED]"
2022-09-07T11:02:36 [I|app|54f36754] Completed 201 Created in 190ms (Views: 1.7ms | ActiveRecord: 63.5ms | Allocations: 35138)
2022-09-07T11:02:39 [I|app|88d1ce86] Started GET "/notification_recipients" for at 2022-09-07 11:02:39 -0500
2022-09-07T11:02:39 [I|app|88d1ce86] Processing by NotificationRecipientsController#index as JSON
2022-09-07T11:02:39 [I|app|88d1ce86] Completed 200 OK in 13ms (Views: 0.1ms | ActiveRecord: 2.7ms | Allocations: 2510)

And finally if you choose the hostgroup on the Customize Host page:

The Host tab will not have not have the Lifecycle Environment and Content View that should be filled in from the hostgroup definitions.

However, the Operating System tab will be filled out correctly according to the hostgroup definitions.

The hammer command is the most problematic since this also breaks our VM build automation. We really need to be able to select “Synced Content” which does not appear to be a hammer command option. Again, all of this worked prior to Foreman 3.0, so not sure why this hasn’t been addressed in versions 3.0, 3.1, 3.2 or 3.3.