Foreman-installer command get response: 500 Internal Server Error

Problem:

foreman-installer --scenario katello --foreman-initial-organization "Velocix" --foreman-initial-location "Chennai" --foreman-initial-admin-username admin --foreman-initial-admin-password velocix1
2024-07-03 16:48:39 [NOTICE] [root] Loading installer configuration. This will tak                                                                                      e some time.
2024-07-03 16:48:44 [NOTICE] [root] Running installer with log based terminal outp                                                                                      ut at level NOTICE.
2024-07-03 16:48:44 [NOTICE] [root] Use -l to set the terminal output log level to                                                                                       ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2024-07-03 16:52:02 [NOTICE] [configure] Starting system configuration.
2024-07-03 16:56:33 [NOTICE] [configure] 250 configuration steps out of 1390 steps complete.
2024-07-03 16:57:26 [NOTICE] [configure] 500 configuration steps out of 1392 steps complete.
2024-07-03 16:59:15 [NOTICE] [configure] 750 configuration steps out of 1397 steps complete.
2024-07-03 16:59:29 [NOTICE] [configure] 1000 configuration steps out of 1420 steps complete.
2024-07-03 17:04:02 [NOTICE] [configure] 1250 configuration steps out of 1420 steps complete.
2024-07-03 17:06:39 [ERROR ] [configure] /Stage[main]/Foreman::Register/Foreman_host[foreman-rhss.example.com]: Could not evaluate: Error making GET request to Foreman at https://rhss.example.com/api/v2/hosts: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 17:07:54 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_host[foreman-proxy-rhss.example.com]: Could not evaluate: Error making GET request to Foreman at https://rhss.example.com/api/v2/hosts: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 17:07:54 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[rhss.example.com]: Could not evaluate: Error making GET request to Foreman at https://rhss.example.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 17:07:55 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[rhss.example.com]: Failed to call refresh: Error making GET request to Foreman at https://rhss.example.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 17:07:55 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[rhss.example.com]: Error making GET request to Foreman at https://rhss.example.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 17:07:59 [NOTICE] [configure] System configuration has finished.

  There were errors detected during install.
  Please address the errors and re-run the installer to ensure the system is properly configured.
  Failing to do so is likely to result in broken functionality.

  The full log is at /var/log/foreman-installer/katello.log

Expected outcome:
All steps in foreman-installer command run successfully
Foreman and Proxy versions:
Foreman - 3.5 , Katello - 4.7
Distribution and version:
Rocky Linux release 8.10 (Green Obsidian)
Other relevant data:

[root@rhss ~]# cat /var/log/foreman/production.log
2024-07-03T16:53:01 [I|app|] Rails cache backend: File
2024-07-03T16:53:09 [I|app|] Rails cache backend: File
2024-07-03T17:00:10 [I|app|] Rails cache backend: File
2024-07-03T17:00:13 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:00:14 [W|app|] Failed to register foreman_remote_execution plugin (FATAL:  role "foreman" does not exist
 | )
2024-07-03T17:00:20 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:00:35 [I|app|] Rails cache backend: File
2024-07-03T17:00:37 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:00:38 [W|app|] Failed to register foreman_remote_execution plugin (PG::UndefinedTable: ERROR:  relation "job_invocations" does not exist
 | LINE 8:  WHERE a.attrelid = '"job_invocations"'::regclass
 |                             ^
 | )
2024-07-03T17:00:43 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:00:45 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:00:57 [I|app|] Rails cache backend: File
2024-07-03T17:01:00 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:01:01 [W|app|] Failed to register foreman_remote_execution plugin (PG::UndefinedTable: ERROR:  relation "job_invocations" does not exist
 | LINE 8:  WHERE a.attrelid = '"job_invocations"'::regclass
 |                             ^
 | )
2024-07-03T17:01:06 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:01:08 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:01:15 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:01:22 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:01:22 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:01:23 [I|app|] Finished cleaning duplicate table rows
2024-07-03T17:01:23 [I|app|] Finished renaming duplicate table rows
2024-07-03T17:01:33 [I|app|] Rails cache backend: File
2024-07-03T17:01:35 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:01:37 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T17:01:37 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T17:01:42 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:01:44 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T17:01:44 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:01:57 [I|app|] Rails cache backend: File
2024-07-03T17:02:00 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:02:01 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T17:02:01 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T17:02:06 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:02:08 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T17:02:08 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:02:20 [I|app|] Rails cache backend: File
2024-07-03T17:02:23 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:02:24 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T17:02:25 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T17:02:30 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:02:31 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T17:02:31 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:02:35 [I|app|] Seeding /usr/share/foreman/db/seeds.d/010-calibrate_bcrypt_cost.rb
2024-07-03T17:02:35 [I|aud|] Setting (1) create event on value --- 8
 | ...
2024-07-03T17:02:36 [I|app|] Seeding /usr/share/foreman/db/seeds.d/020-permissions_list.rb
2024-07-03T17:02:36 [I|app|] Seeding /usr/share/foreman/db/seeds.d/020-roles_list.rb
2024-07-03T17:02:36 [I|app|] Seeding /usr/share/foreman/db/seeds.d/030-auth_sources.rb
2024-07-03T17:02:36 [I|app|] Seeding /usr/share/foreman/db/seeds.d/030-permissions.rb
2024-07-03T17:02:37 [I|app|] Seeding /usr/share/foreman/db/seeds.d/035-admin.rb
2024-07-03T17:02:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:38 [I|app|] Seeding /usr/share/foreman/db/seeds.d/040-roles.rb
2024-07-03T17:02:38 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:39 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:41 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:41 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:41 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:02:41 [I|app|] Seeding /usr/share/foreman/db/seeds.d/050-taxonomies.rb
2024-07-03T17:02:42 [I|aud|] Setting (2) create event on value --- Velocix
 | ...
2024-07-03T17:02:42 [I|aud|] Setting (3) create event on value --- Chennai
 | ...
2024-07-03T17:02:42 [I|aud|] Setting (4) create event on value --- Chennai
 | ...
2024-07-03T17:02:42 [I|app|] Seeding /usr/share/foreman/db/seeds.d/060-architectures.rb
2024-07-03T17:02:42 [I|app|] Seeding /usr/share/foreman/db/seeds.d/070-provisioning_templates.rb
2024-07-03T17:02:43 [I|app|] Template Boot disk Grub2 EFI - generic host requires plugin foreman_bootdisk, skipping import.
2024-07-03T17:02:43 [I|app|] Template Boot disk iPXE - generic host requires plugin foreman_bootdisk, skipping import.
2024-07-03T17:02:43 [I|app|] Template Boot disk iPXE - generic static host requires plugin foreman_bootdisk, skipping import.
2024-07-03T17:02:43 [I|app|] Template Boot disk iPXE - host requires plugin foreman_bootdisk, skipping import.
2024-07-03T17:02:44 [I|app|] Template Discovery Debian kexec requires plugin foreman_discovery, skipping import.
2024-07-03T17:02:44 [I|app|] Template Discovery Red Hat kexec requires plugin foreman_discovery, skipping import.
2024-07-03T17:02:44 [I|app|] Template Discovery Ubuntu kexec Autoinstall requires plugin foreman_discovery, skipping import.
2024-07-03T17:02:48 [I|app|] Seeding /usr/share/foreman/db/seeds.d/080-partition_tables.rb
2024-07-03T17:02:49 [I|app|] Seeding /usr/share/foreman/db/seeds.d/090-report_templates.rb
2024-07-03T17:02:50 [I|app|] Template Host - Vulnerabilities requires plugin foreman_openscap, skipping import.
2024-07-03T17:02:50 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/100-assign_features_with_templates.rb
2024-07-03T17:02:50 [I|app|] Seeding /usr/share/foreman/db/seeds.d/100-installation_media.rb
2024-07-03T17:02:51 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/101-locations.rb
2024-07-03T17:02:51 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/102-organizations.rb
2024-07-03T17:02:51 [I|aud|] Katello::KTEnvironment (1) create event on name Library
2024-07-03T17:02:51 [I|aud|] Katello::KTEnvironment (1) create event on description
2024-07-03T17:02:51 [I|aud|] Katello::KTEnvironment (1) create event on library true
2024-07-03T17:02:51 [I|aud|] Katello::KTEnvironment (1) create event on organization_id 1
2024-07-03T17:02:51 [I|aud|] Katello::KTEnvironment (1) create event on label Library
2024-07-03T17:02:51 [I|aud|] Katello::KTEnvironment (1) create event on registry_name_pattern
2024-07-03T17:02:51 [I|aud|] Katello::KTEnvironment (1) create event on registry_unauthenticated_pull false
2024-07-03T17:02:51 [I|aud|] Katello::KTEnvironment (1) create event on content_facet_ids []
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on name Default Organization View
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on label Default_Organization_View
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on description
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on organization_id 1
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on default true
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on composite false
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on next_version 1
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on force_puppet_environment false
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on auto_publish false
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on solve_dependencies false
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on import_only false
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on generated_for 0
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on repository_ids []
2024-07-03T17:02:51 [I|aud|] Katello::ContentView (1) create event on environment_ids []
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewVersion (1) create event on content_view_id 1
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewVersion (1) create event on major 1
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewVersion (1) create event on definition_archive_id
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewVersion (1) create event on minor 0
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewVersion (1) create event on content_counts
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewVersion (1) create event on repository_ids []
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewVersion (1) create event on environment_ids []
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewEnvironment (1) create event on name Library
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewEnvironment (1) create event on label Library
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewEnvironment (1) create event on cp_id d2656a9f5dbd3ac3d9837a12b3dc7e05
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewEnvironment (1) create event on content_view_id 1
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewEnvironment (1) create event on environment_id 1
2024-07-03T17:02:51 [I|aud|] Katello::ContentViewEnvironment (1) create event on content_view_version_id 1
2024-07-03T17:02:51 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/104-proxy.rb
2024-07-03T17:02:51 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/106-mail_notifications.rb
2024-07-03T17:02:51 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/108-subcription-bookmarks.rb
2024-07-03T17:02:51 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/109-katello-notification-blueprints.rb
2024-07-03T17:02:51 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/110-content-view-autopublish.rb
2024-07-03T17:02:51 [I|app|] Seeding /usr/share/foreman/db/seeds.d/110-smart_proxy_features.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/111-container-image-bookmarks.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/111-upgrade_tasks.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/foreman/db/seeds.d/130-compute_profiles.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/foreman/db/seeds.d/150-bookmarks.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/150-module_job_templates.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/foreman/db/seeds.d/160-mail_notifications.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/foreman/db/seeds.d/170-notification_blueprints.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/foreman/db/seeds.d/180-common_parameters.rb
2024-07-03T17:02:52 [I|app|] Seeding /usr/share/gems/gems/foreman-tasks-7.2.1/db/seeds.d/20-foreman_tasks_permissions.rb
2024-07-03T17:02:53 [I|aud|] Filter (142) create event on search owner.id = current_user
2024-07-03T17:02:53 [I|aud|] Filter (142) create event on role_id 1
2024-07-03T17:02:53 [I|aud|] Filter (142) create event on taxonomy_search
2024-07-03T17:02:53 [I|aud|] Filter (142) create event on override false
2024-07-03T17:02:53 [I|aud|] Filter (142) create event on organization_ids []
2024-07-03T17:02:53 [I|aud|] Filter (142) create event on location_ids []
2024-07-03T17:02:53 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/20-permissions.rb
2024-07-03T17:02:53 [I|aud|] Filter (143) create event on search user = current_user
2024-07-03T17:02:53 [I|aud|] Filter (143) create event on role_id 1
2024-07-03T17:02:53 [I|aud|] Filter (143) create event on taxonomy_search
2024-07-03T17:02:53 [I|aud|] Filter (143) create event on override false
2024-07-03T17:02:53 [I|aud|] Filter (143) create event on organization_ids []
2024-07-03T17:02:53 [I|aud|] Filter (143) create event on location_ids []
2024-07-03T17:02:53 [I|app|] Seeding /usr/share/gems/gems/foreman-tasks-7.2.1/db/seeds.d/30-notification_blueprints.rb
2024-07-03T17:02:53 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/50-notification_blueprints.rb
2024-07-03T17:02:53 [I|app|] Seeding /usr/share/gems/gems/foreman-tasks-7.2.1/db/seeds.d/60-dynflow_proxy_feature.rb
2024-07-03T17:02:53 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/60-ssh_proxy_feature.rb
2024-07-03T17:02:53 [I|app|] Seeding /usr/share/gems/gems/foreman-tasks-7.2.1/db/seeds.d/61-foreman_tasks_bookmarks.rb
2024-07-03T17:02:53 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/70-job_templates.rb
2024-07-03T17:02:53 [I|aud|] RemoteExecutionFeature (14) update event on job_template_id , 155
2024-07-03T17:02:53 [I|aud|] RemoteExecutionFeature (15) update event on job_template_id , 162
2024-07-03T17:02:53 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/75-job_templates.rb
2024-07-03T17:02:53 [I|aud|] RemoteExecutionFeature (10) update event on job_template_id , 166
2024-07-03T17:02:53 [I|aud|] RemoteExecutionFeature (11) update event on job_template_id , 167
2024-07-03T17:02:53 [I|aud|] RemoteExecutionFeature (7) update event on job_template_id , 168
2024-07-03T17:02:53 [I|aud|] RemoteExecutionFeature (1) update event on job_template_id , 169
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (2) update event on job_template_id , 170
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (9) update event on job_template_id , 171
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (5) update event on job_template_id , 172
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (6) update event on job_template_id , 173
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (13) update event on job_template_id , 174
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (12) update event on job_template_id , 175
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (8) update event on job_template_id , 176
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (3) update event on job_template_id , 177
2024-07-03T17:02:54 [I|aud|] RemoteExecutionFeature (4) update event on job_template_id , 178
2024-07-03T17:02:54 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/90-bookmarks.rb
2024-07-03T17:02:54 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/95-mail_notifications.rb
2024-07-03T17:02:54 [I|app|] All seed files executed
2024-07-03T17:02:54 [I|aud|] Setting (5) create event on value --- false
 | ...
2024-07-03T17:03:06 [I|app|] Rails cache backend: File
2024-07-03T17:03:08 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:03:10 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T17:03:10 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T17:03:15 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:03:17 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T17:03:17 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:03:29 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:29 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:29 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:29 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:29 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:30 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:31 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:31 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:32 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:32 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:32 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:32 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:33 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:34 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:35 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:36 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:38 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:38 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:38 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:38 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:38 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:41 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:42 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:03:43 [I|app|] No new seed file updates found. Skipping
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating throttle_limiter...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] start terminating client dispatcher...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] stop listening for new events...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:48 [I|dyn|] start terminating clock...
2024-07-03T17:03:59 [I|app|] Rails cache backend: File
2024-07-03T17:04:01 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:04:02 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T17:04:03 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T17:04:07 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:04:09 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T17:04:09 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:04:20 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:04:21 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:04:22 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:04:22 [I|app|] No new seed file updates found. Skipping
2024-07-03T17:05:32 [I|app|] Rails cache backend: File
2024-07-03T17:05:35 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:05:36 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T17:05:36 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T17:05:41 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:05:43 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T17:05:43 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:05:54 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:05:55 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:05:56 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:05:57 [I|app|] No new seed file updates found. Skipping
2024-07-03T17:05:59 [I|app|] orchestrator in passive mode
2024-07-03T17:05:59 [I|dyn|] Acquired orchestrator lock, entering active mode.
2024-07-03T17:05:59 [I|app|] Enqueued CreateRssNotifications (Job ID: 639d8b66-3868-47a7-9463-ad2caf7191bc) to Dynflow(default)
2024-07-03T17:06:00 [I|app|] Enqueued StoredValuesCleanupJob (Job ID: a947f735-3039-47db-8ff1-b3d9550ee64c) to Dynflow(default)
2024-07-03T17:06:00 [I|app|] Enqueued CreateExpiredManifestNotifications (Job ID: a568fea6-917c-41e3-aca7-76c34517ef90) to Dynflow(default)
2024-07-03T17:06:00 [I|app|] Enqueued CreatePulpDiskSpaceNotifications (Job ID: 33d1855d-83fc-4200-b678-f615cfab27ed) to Dynflow(default)
2024-07-03T17:06:00 [I|app|] Enqueued SendExpireSoonNotifications (Job ID: 5a31a421-2800-4d1a-9df2-6a5e72b715f2) to Dynflow(default)
2024-07-03T17:06:00 [I|app|] Everything ready for world: 49140b2b-f950-4e3b-9f0f-8fdb3dd75c0b
2024-07-03T17:06:09 [I|app|] Rails cache backend: File
2024-07-03T17:06:12 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:06:13 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T17:06:13 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T17:06:18 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:06:20 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T17:06:20 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:06:30 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:06:32 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:06:33 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:06:33 [I|app|] No new seed file updates found. Skipping
2024-07-03T17:06:36 [I|app|] Everything ready for world: 7731ce8c-a2a4-453a-85e1-93d834b3f37c
2024-07-03T17:06:36 [I|dyn|] Performing validity checks
2024-07-03T17:06:36 [I|app|] Performing CreateExpiredManifestNotifications (Job ID: a568fea6-917c-41e3-aca7-76c34517ef90) from Dynflow(default) enqueued at 2024-07-03T11:36:00Z
2024-07-03T17:06:36 [I|app|] Performing CreateRssNotifications (Job ID: 639d8b66-3868-47a7-9463-ad2caf7191bc) from Dynflow(default) enqueued at 2024-07-03T11:35:59Z
2024-07-03T17:06:36 [I|app|] Performing StoredValuesCleanupJob (Job ID: a947f735-3039-47db-8ff1-b3d9550ee64c) from Dynflow(default) enqueued at 2024-07-03T11:35:59Z
2024-07-03T17:06:36 [I|app|] Performing CreatePulpDiskSpaceNotifications (Job ID: 33d1855d-83fc-4200-b678-f615cfab27ed) from Dynflow(default) enqueued at 2024-07-03T11:36:00Z
2024-07-03T17:06:36 [I|app|] Performing SendExpireSoonNotifications (Job ID: 5a31a421-2800-4d1a-9df2-6a5e72b715f2) from Dynflow(default) enqueued at 2024-07-03T11:36:00Z
2024-07-03T17:06:36 [I|dyn|] Finished performing validity checks
2024-07-03T17:06:36 [I|app|] Enqueued CreatePulpDiskSpaceNotifications (Job ID: d6bb4a4b-3f75-43b9-a7db-eb0e11ca012e) to Dynflow(default) at 2024-07-03 23:36:36 UTC
2024-07-03T17:06:36 [I|app|] Performed CreatePulpDiskSpaceNotifications (Job ID: 33d1855d-83fc-4200-b678-f615cfab27ed) from Dynflow(default) in 164.12ms
2024-07-03T17:06:36 [I|app|] Enqueued StoredValuesCleanupJob (Job ID: 680217b0-3fb5-4a29-8817-432d4d32c368) to Dynflow(default) at 2024-07-03 23:36:36 UTC with arguments: {}
2024-07-03T17:06:36 [I|app|] Performed StoredValuesCleanupJob (Job ID: a947f735-3039-47db-8ff1-b3d9550ee64c) from Dynflow(default) in 176.92ms
2024-07-03T17:06:36 [I|app|] Enqueued SendExpireSoonNotifications (Job ID: 4c516e37-bc3a-4ea5-b00b-79a4d26e64a2) to Dynflow(default) at 2024-07-03 23:36:36 UTC
2024-07-03T17:06:36 [I|app|] Performed SendExpireSoonNotifications (Job ID: 5a31a421-2800-4d1a-9df2-6a5e72b715f2) from Dynflow(default) in 145.12ms
2024-07-03T17:06:37 [W|not|] Failed to handle notifications - this is most likely a bug: Katello::Resources::Candlepin::Owner: 404 Not Found {"displayMessage":"Organization with id Velocix could not be found.","requestUuid":"5492c7d7-2c51-4556-bb4f-b61a2310463c"} (GET /candlepin/owners/Velocix)
2024-07-03T17:06:37 [I|app|] Enqueued CreateExpiredManifestNotifications (Job ID: 4e8b42c2-df48-4333-929d-97ec676ba0a2) to Dynflow(default) at 2024-07-04 11:36:37 UTC
2024-07-03T17:06:37 [I|app|] Performed CreateExpiredManifestNotifications (Job ID: a568fea6-917c-41e3-aca7-76c34517ef90) from Dynflow(default) in 897.86ms
2024-07-03T17:06:37 [I|app|db58d5c2] Started GET "/api/v2/hosts?search=name%3D%22rhss.example.com%22" for 127.0.0.1 at 2024-07-03 17:06:37 +0530
2024-07-03T17:06:37 [I|app|db58d5c2] Katello event daemon started process=93039
2024-07-03T17:06:37 [I|app|db58d5c2] Subscribed to katello.candlepin.candlepin_events
2024-07-03T17:06:38 [I|kat|db58d5c2] Polling Katello Event Queue
2024-07-03T17:06:39 [F|app|db58d5c2]
 db58d5c2 | RestClient::BadRequest (Katello::Resources::Candlepin::Owner: 400 Bad Request {"displayMessage":"Problem generating ueber cert for owner Velocix","requestUuid":"243464c6-f191-4c42-8c1f-2e64534d2c35"} (POST /candlepin/owners/Velocix/uebercert)):
 db58d5c2 |
 db58d5c2 | katello (4.7.6) app/lib/katello/http_resource.rb:92:in `issue_request'
 db58d5c2 | katello (4.7.6) app/lib/katello/http_resource.rb:47:in `block (2 levels) in singleton class'
 db58d5c2 | katello (4.7.6) app/lib/katello/resources/candlepin/owner.rb:101:in `generate_ueber_cert'
 db58d5c2 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:59:in `generate_debug_cert'
 db58d5c2 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:65:in `rescue in load_debug_cert'
 db58d5c2 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:62:in `load_debug_cert'
 db58d5c2 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:14:in `block (2 levels) in included'
 db58d5c2 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `instance_eval'
 db58d5c2 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `run_initializer'
 db58d5c2 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:97:in `lazy_attribute_get'
 db58d5c2 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:44:in `block (2 levels) in lazy_accessor'
 db58d5c2 | katello (4.7.6) app/services/katello/organization_creator.rb:78:in `create_backend_objects!'
 db58d5c2 | katello (4.7.6) app/services/katello/organization_creator.rb:56:in `block in create!'
 db58d5c2 | katello (4.7.6) app/services/katello/organization_creator.rb:53:in `create!'
 db58d5c2 | katello (4.7.6) app/services/katello/organization_creator.rb:25:in `block (2 levels) in create_all_organizations!'
 db58d5c2 | katello (4.7.6) app/services/katello/organization_creator.rb:23:in `block in create_all_organizations!'
 db58d5c2 | app/models/concerns/foreman/thread_session.rb:108:in `as'
 db58d5c2 | app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 db58d5c2 | katello (4.7.6) app/services/katello/organization_creator.rb:22:in `create_all_organizations!'
 db58d5c2 | katello (4.7.6) lib/katello/middleware/organization_created_enforcer.rb:12:in `call'
 db58d5c2 | katello (4.7.6) lib/katello/middleware/event_daemon.rb:10:in `call'
 db58d5c2 | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 db58d5c2 | lib/foreman/middleware/telemetry.rb:10:in `call'
 db58d5c2 | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 db58d5c2 | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 db58d5c2 | katello (4.7.6) lib/katello/prevent_json_parsing.rb:12:in `call'
2024-07-03T17:06:39 [I|app|] Enqueued CreateRssNotifications (Job ID: a0f417c5-0b8e-4663-a4ac-b2c7e66c5710) to Dynflow(default) at 2024-07-03 23:36:39 UTC with arguments: {}
2024-07-03T17:06:39 [I|app|] Performed CreateRssNotifications (Job ID: 639d8b66-3868-47a7-9463-ad2caf7191bc) from Dynflow(default) in 3021.75ms
2024-07-03T17:06:49 [I|app|] Rails cache backend: File
2024-07-03T17:06:51 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T17:06:52 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T17:06:52 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T17:06:58 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T17:06:59 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T17:06:59 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T17:07:11 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:07:12 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:07:12 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T17:07:13 [I|app|] No new seed file updates found. Skipping
2024-07-03T17:07:15 [I|app|] Everything ready for world: d25010f7-5988-45e0-8d62-8ff6f2d18c55
2024-07-03T17:07:53 [I|app|b3da5724] Started GET "/api/v2/hosts?search=name%3D%22rhss.example.com%22" for 127.0.0.1 at 2024-07-03 17:07:53 +0530
2024-07-03T17:07:54 [F|app|b3da5724]
 b3da5724 | RestClient::BadRequest (Katello::Resources::Candlepin::Owner: 400 Bad Request {"displayMessage":"Problem generating ueber cert for owner Velocix","requestUuid":"ae6b5967-f5bd-4fc9-8733-069789382fdf"} (POST /candlepin/owners/Velocix/uebercert)):
 b3da5724 |
 b3da5724 | katello (4.7.6) app/lib/katello/http_resource.rb:92:in `issue_request'
 b3da5724 | katello (4.7.6) app/lib/katello/http_resource.rb:47:in `block (2 levels) in singleton class'
 b3da5724 | katello (4.7.6) app/lib/katello/resources/candlepin/owner.rb:101:in `generate_ueber_cert'
 b3da5724 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:59:in `generate_debug_cert'
 b3da5724 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:65:in `rescue in load_debug_cert'
 b3da5724 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:62:in `load_debug_cert'
 b3da5724 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:14:in `block (2 levels) in included'
 b3da5724 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `instance_eval'
 b3da5724 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `run_initializer'
 b3da5724 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:97:in `lazy_attribute_get'
 b3da5724 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:44:in `block (2 levels) in lazy_accessor'
 b3da5724 | katello (4.7.6) app/services/katello/organization_creator.rb:78:in `create_backend_objects!'
 b3da5724 | katello (4.7.6) app/services/katello/organization_creator.rb:56:in `block in create!'
 b3da5724 | katello (4.7.6) app/services/katello/organization_creator.rb:53:in `create!'
 b3da5724 | katello (4.7.6) app/services/katello/organization_creator.rb:25:in `block (2 levels) in create_all_organizations!'
 b3da5724 | katello (4.7.6) app/services/katello/organization_creator.rb:23:in `block in create_all_organizations!'
 b3da5724 | app/models/concerns/foreman/thread_session.rb:108:in `as'
 b3da5724 | app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 b3da5724 | katello (4.7.6) app/services/katello/organization_creator.rb:22:in `create_all_organizations!'
 b3da5724 | katello (4.7.6) lib/katello/middleware/organization_created_enforcer.rb:12:in `call'
 b3da5724 | katello (4.7.6) lib/katello/middleware/event_daemon.rb:10:in `call'
 b3da5724 | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 b3da5724 | lib/foreman/middleware/telemetry.rb:10:in `call'
 b3da5724 | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 b3da5724 | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 b3da5724 | katello (4.7.6) lib/katello/prevent_json_parsing.rb:12:in `call'
2024-07-03T17:07:54 [I|app|1222d472] Started GET "/api/v2/smart_proxies?search=name%3D%22rhss.example.com%22" for 127.0.0.1 at 2024-07-03 17:07:54 +0530
2024-07-03T17:07:54 [F|app|1222d472]
 1222d472 | RestClient::BadRequest (Katello::Resources::Candlepin::Owner: 400 Bad Request {"displayMessage":"Problem generating ueber cert for owner Velocix","requestUuid":"3be92650-7726-4667-993d-35102e3cefb2"} (POST /candlepin/owners/Velocix/uebercert)):
 1222d472 |
 1222d472 | katello (4.7.6) app/lib/katello/http_resource.rb:92:in `issue_request'
 1222d472 | katello (4.7.6) app/lib/katello/http_resource.rb:47:in `block (2 levels) in singleton class'
 1222d472 | katello (4.7.6) app/lib/katello/resources/candlepin/owner.rb:101:in `generate_ueber_cert'
 1222d472 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:59:in `generate_debug_cert'
 1222d472 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:65:in `rescue in load_debug_cert'
 1222d472 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:62:in `load_debug_cert'
 1222d472 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:14:in `block (2 levels) in included'
 1222d472 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `instance_eval'
 1222d472 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `run_initializer'
 1222d472 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:97:in `lazy_attribute_get'
 1222d472 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:44:in `block (2 levels) in lazy_accessor'
 1222d472 | katello (4.7.6) app/services/katello/organization_creator.rb:78:in `create_backend_objects!'
 1222d472 | katello (4.7.6) app/services/katello/organization_creator.rb:56:in `block in create!'
 1222d472 | katello (4.7.6) app/services/katello/organization_creator.rb:53:in `create!'
 1222d472 | katello (4.7.6) app/services/katello/organization_creator.rb:25:in `block (2 levels) in create_all_organizations!'
 1222d472 | katello (4.7.6) app/services/katello/organization_creator.rb:23:in `block in create_all_organizations!'
 1222d472 | app/models/concerns/foreman/thread_session.rb:108:in `as'
 1222d472 | app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 1222d472 | katello (4.7.6) app/services/katello/organization_creator.rb:22:in `create_all_organizations!'
 1222d472 | katello (4.7.6) lib/katello/middleware/organization_created_enforcer.rb:12:in `call'
 1222d472 | katello (4.7.6) lib/katello/middleware/event_daemon.rb:10:in `call'
 1222d472 | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 1222d472 | lib/foreman/middleware/telemetry.rb:10:in `call'
 1222d472 | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 1222d472 | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 1222d472 | katello (4.7.6) lib/katello/prevent_json_parsing.rb:12:in `call'
2024-07-03T17:07:54 [I|app|2b4fdfe0] Started GET "/api/v2/smart_proxies?search=name%3D%22rhss.example.com%22" for 127.0.0.1 at 2024-07-03 17:07:54 +0530
2024-07-03T17:07:55 [F|app|2b4fdfe0]
 2b4fdfe0 | RestClient::BadRequest (Katello::Resources::Candlepin::Owner: 400 Bad Request {"displayMessage":"Problem generating ueber cert for owner Velocix","requestUuid":"fd907590-426f-4203-90bd-bbd4b1843e3f"} (POST /candlepin/owners/Velocix/uebercert)):
 2b4fdfe0 |
 2b4fdfe0 | katello (4.7.6) app/lib/katello/http_resource.rb:92:in `issue_request'
 2b4fdfe0 | katello (4.7.6) app/lib/katello/http_resource.rb:47:in `block (2 levels) in singleton class'
 2b4fdfe0 | katello (4.7.6) app/lib/katello/resources/candlepin/owner.rb:101:in `generate_ueber_cert'
 2b4fdfe0 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:59:in `generate_debug_cert'
 2b4fdfe0 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:65:in `rescue in load_debug_cert'
 2b4fdfe0 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:62:in `load_debug_cert'
 2b4fdfe0 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:14:in `block (2 levels) in included'
 2b4fdfe0 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `instance_eval'
 2b4fdfe0 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `run_initializer'
 2b4fdfe0 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:97:in `lazy_attribute_get'
 2b4fdfe0 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:44:in `block (2 levels) in lazy_accessor'
 2b4fdfe0 | katello (4.7.6) app/services/katello/organization_creator.rb:78:in `create_backend_objects!'
 2b4fdfe0 | katello (4.7.6) app/services/katello/organization_creator.rb:56:in `block in create!'
 2b4fdfe0 | katello (4.7.6) app/services/katello/organization_creator.rb:53:in `create!'
 2b4fdfe0 | katello (4.7.6) app/services/katello/organization_creator.rb:25:in `block (2 levels) in create_all_organizations!'
 2b4fdfe0 | katello (4.7.6) app/services/katello/organization_creator.rb:23:in `block in create_all_organizations!'
 2b4fdfe0 | app/models/concerns/foreman/thread_session.rb:108:in `as'
 2b4fdfe0 | app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 2b4fdfe0 | katello (4.7.6) app/services/katello/organization_creator.rb:22:in `create_all_organizations!'
 2b4fdfe0 | katello (4.7.6) lib/katello/middleware/organization_created_enforcer.rb:12:in `call'
 2b4fdfe0 | katello (4.7.6) lib/katello/middleware/event_daemon.rb:10:in `call'
 2b4fdfe0 | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 2b4fdfe0 | lib/foreman/middleware/telemetry.rb:10:in `call'
 2b4fdfe0 | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 2b4fdfe0 | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 2b4fdfe0 | katello (4.7.6) lib/katello/prevent_json_parsing.rb:12:in `call'

Katello 4.7 needs a patched candlepin to work on 8.10, have a look at Candlepin does not support JSS versions less than 4.4?

But I’d also suggest deploying something newer if that’s a new setup.

After running foreman-installer need to restore backup which is using Foreman - 3.5 , Katello - 4.7.Is there any way to resolve the above issue?
Also facing the same issue in Rocky 8.7 OS version.

Can you show logs from a 8.7 install?

Steps followed during installation:

systemctl stop firewalld
systemctl disable firewalld
dnf clean all
dnf install https://yum.theforeman.org/releases/3.5/el8/x86_64/foreman-release.rpm -y
dnf install https://yum.theforeman.org/katello/4.7/katello/el8/x86_64/katello-repos-latest.rpm -y
dnf localinstall https://yum.puppet.com/puppet7-release-el-8.noarch.rpm
dnf config-manager --set-enabled powertools
dnf module enable katello:el8 pulpcore:el8
dnf install foreman-installer-katello
foreman-installer --scenario katello

[root@rhss ~]# cat /etc/rocky-release
Rocky Linux release 8.7 (Green Obsidian)

Foreman installer command:

[root@rhss ~]# foreman-installer --scenario katello
2024-07-03 18:12:34 [NOTICE] [root] Loading installer configuration. This will tak                                                                                      e some time.
2024-07-03 18:12:42 [NOTICE] [root] Running installer with log based terminal outp                                                                                      ut at level NOTICE.
2024-07-03 18:12:42 [NOTICE] [root] Use -l to set the terminal output log level to                                                                                       ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2024-07-03 18:16:16 [NOTICE] [configure] Starting system configuration.
2024-07-03 18:20:28 [NOTICE] [configure] 250 configuration steps out of 1390 steps complete.
2024-07-03 18:21:40 [NOTICE] [configure] 500 configuration steps out of 1392 steps complete.
2024-07-03 18:23:21 [NOTICE] [configure] 750 configuration steps out of 1397 steps complete.
2024-07-03 18:23:36 [NOTICE] [configure] 1000 configuration steps out of 1420 steps complete.
2024-07-03 18:28:12 [NOTICE] [configure] 1250 configuration steps out of 1420 steps complete.
2024-07-03 18:30:54 [ERROR ] [configure] /Stage[main]/Foreman::Register/Foreman_host[foreman-rhss.example.com]: Could not evaluate: Error making GET request to Foreman at https://rhss.example.com/api/v2/hosts: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 18:32:09 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_host[foreman-proxy-rhss.example.com]: Could not evaluate: Error making GET request to Foreman at https://rhss.example.com/api/v2/hosts: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 18:32:09 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[rhss.example.com]: Could not evaluate: Error making GET request to Foreman at https://rhss.example.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 18:32:10 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[rhss.example.com]: Failed to call refresh: Error making GET request to Foreman at https://rhss.example.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 18:32:10 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[rhss.example.com]: Error making GET request to Foreman at https://rhss.example.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on rhss.example.com for detailed information
2024-07-03 18:32:14 [NOTICE] [configure] System configuration has finished.

  There were errors detected during install.
  Please address the errors and re-run the installer to ensure the system is properly configured.
  Failing to do so is likely to result in broken functionality.

  The full log is at /var/log/foreman-installer/katello.log

Production logs:

[root@rhss ~]# cat /var/log/foreman/production.log
2024-07-03T18:17:34 [I|app|] Rails cache backend: File
2024-07-03T18:17:43 [I|app|] Rails cache backend: File
2024-07-03T18:24:43 [I|app|] Rails cache backend: File
2024-07-03T18:24:45 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:24:47 [W|app|] Failed to register foreman_remote_execution plugin (PG::UndefinedTable: ERROR:  relation "job_invocations" does not exist
 | LINE 8:  WHERE a.attrelid = '"job_invocations"'::regclass
 |                             ^
 | )
2024-07-03T18:24:52 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:24:53 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:25:06 [I|app|] Rails cache backend: File
2024-07-03T18:25:08 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:25:10 [W|app|] Failed to register foreman_remote_execution plugin (PG::UndefinedTable: ERROR:  relation "job_invocations" does not exist
 | LINE 8:  WHERE a.attrelid = '"job_invocations"'::regclass
 |                             ^
 | )
2024-07-03T18:25:15 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:25:16 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:25:24 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:25:30 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:25:30 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:25:31 [I|app|] Finished cleaning duplicate table rows
2024-07-03T18:25:31 [I|app|] Finished renaming duplicate table rows
2024-07-03T18:25:41 [I|app|] Rails cache backend: File
2024-07-03T18:25:44 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:25:45 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:25:46 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:25:51 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:25:52 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:25:52 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:26:05 [I|app|] Rails cache backend: File
2024-07-03T18:26:08 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:26:09 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:26:10 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:26:15 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:26:16 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:26:16 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:26:29 [I|app|] Rails cache backend: File
2024-07-03T18:26:31 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:26:33 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:26:33 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:26:38 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:26:40 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:26:40 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:26:44 [I|app|] Seeding /usr/share/foreman/db/seeds.d/010-calibrate_bcrypt_cost.rb
2024-07-03T18:26:44 [I|aud|] Setting (1) create event on value --- 8
 | ...
2024-07-03T18:26:44 [I|app|] Seeding /usr/share/foreman/db/seeds.d/020-permissions_list.rb
2024-07-03T18:26:44 [I|app|] Seeding /usr/share/foreman/db/seeds.d/020-roles_list.rb
2024-07-03T18:26:44 [I|app|] Seeding /usr/share/foreman/db/seeds.d/030-auth_sources.rb
2024-07-03T18:26:44 [I|app|] Seeding /usr/share/foreman/db/seeds.d/030-permissions.rb
2024-07-03T18:26:45 [I|app|] Seeding /usr/share/foreman/db/seeds.d/035-admin.rb
2024-07-03T18:26:45 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:46 [I|app|] Seeding /usr/share/foreman/db/seeds.d/040-roles.rb
2024-07-03T18:26:46 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:47 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:48 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:48 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:48 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:48 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:48 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:48 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:49 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:49 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:49 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:26:49 [I|app|] Seeding /usr/share/foreman/db/seeds.d/050-taxonomies.rb
2024-07-03T18:26:50 [I|aud|] Setting (2) create event on value --- Default Organization
 | ...
2024-07-03T18:26:50 [I|aud|] Setting (3) create event on value --- Default Location
 | ...
2024-07-03T18:26:50 [I|aud|] Setting (4) create event on value --- Default Location
 | ...
2024-07-03T18:26:50 [I|app|] Seeding /usr/share/foreman/db/seeds.d/060-architectures.rb
2024-07-03T18:26:50 [I|app|] Seeding /usr/share/foreman/db/seeds.d/070-provisioning_templates.rb
2024-07-03T18:26:51 [I|app|] Template Boot disk Grub2 EFI - generic host requires plugin foreman_bootdisk, skipping import.
2024-07-03T18:26:51 [I|app|] Template Boot disk iPXE - generic host requires plugin foreman_bootdisk, skipping import.
2024-07-03T18:26:51 [I|app|] Template Boot disk iPXE - generic static host requires plugin foreman_bootdisk, skipping import.
2024-07-03T18:26:51 [I|app|] Template Boot disk iPXE - host requires plugin foreman_bootdisk, skipping import.
2024-07-03T18:26:52 [I|app|] Template Discovery Debian kexec requires plugin foreman_discovery, skipping import.
2024-07-03T18:26:52 [I|app|] Template Discovery Red Hat kexec requires plugin foreman_discovery, skipping import.
2024-07-03T18:26:52 [I|app|] Template Discovery Ubuntu kexec Autoinstall requires plugin foreman_discovery, skipping import.
2024-07-03T18:26:56 [I|app|] Seeding /usr/share/foreman/db/seeds.d/080-partition_tables.rb
2024-07-03T18:26:57 [I|app|] Seeding /usr/share/foreman/db/seeds.d/090-report_templates.rb
2024-07-03T18:26:58 [I|app|] Template Host - Vulnerabilities requires plugin foreman_openscap, skipping import.
2024-07-03T18:26:58 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/100-assign_features_with_templates.rb
2024-07-03T18:26:58 [I|app|] Seeding /usr/share/foreman/db/seeds.d/100-installation_media.rb
2024-07-03T18:26:58 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/101-locations.rb
2024-07-03T18:26:58 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/102-organizations.rb
2024-07-03T18:26:59 [I|aud|] Katello::KTEnvironment (1) create event on name Library
2024-07-03T18:26:59 [I|aud|] Katello::KTEnvironment (1) create event on description
2024-07-03T18:26:59 [I|aud|] Katello::KTEnvironment (1) create event on library true
2024-07-03T18:26:59 [I|aud|] Katello::KTEnvironment (1) create event on organization_id 1
2024-07-03T18:26:59 [I|aud|] Katello::KTEnvironment (1) create event on label Library
2024-07-03T18:26:59 [I|aud|] Katello::KTEnvironment (1) create event on registry_name_pattern
2024-07-03T18:26:59 [I|aud|] Katello::KTEnvironment (1) create event on registry_unauthenticated_pull false
2024-07-03T18:26:59 [I|aud|] Katello::KTEnvironment (1) create event on content_facet_ids []
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on name Default Organization View
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on label Default_Organization_View
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on description
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on organization_id 1
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on default true
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on composite false
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on next_version 1
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on force_puppet_environment false
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on auto_publish false
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on solve_dependencies false
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on import_only false
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on generated_for 0
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on repository_ids []
2024-07-03T18:26:59 [I|aud|] Katello::ContentView (1) create event on environment_ids []
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewVersion (1) create event on content_view_id 1
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewVersion (1) create event on major 1
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewVersion (1) create event on definition_archive_id
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewVersion (1) create event on minor 0
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewVersion (1) create event on content_counts
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewVersion (1) create event on repository_ids []
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewVersion (1) create event on environment_ids []
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewEnvironment (1) create event on name Library
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewEnvironment (1) create event on label Library
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewEnvironment (1) create event on cp_id 64c2cce3716a5bf67b0413fac95e5daa
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewEnvironment (1) create event on content_view_id 1
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewEnvironment (1) create event on environment_id 1
2024-07-03T18:26:59 [I|aud|] Katello::ContentViewEnvironment (1) create event on content_view_version_id 1
2024-07-03T18:26:59 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/104-proxy.rb
2024-07-03T18:26:59 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/106-mail_notifications.rb
2024-07-03T18:26:59 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/108-subcription-bookmarks.rb
2024-07-03T18:26:59 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/109-katello-notification-blueprints.rb
2024-07-03T18:26:59 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/110-content-view-autopublish.rb
2024-07-03T18:26:59 [I|app|] Seeding /usr/share/foreman/db/seeds.d/110-smart_proxy_features.rb
2024-07-03T18:26:59 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/111-container-image-bookmarks.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/111-upgrade_tasks.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/foreman/db/seeds.d/130-compute_profiles.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/foreman/db/seeds.d/150-bookmarks.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/150-module_job_templates.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/foreman/db/seeds.d/160-mail_notifications.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/foreman/db/seeds.d/170-notification_blueprints.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/foreman/db/seeds.d/180-common_parameters.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/foreman-tasks-7.2.1/db/seeds.d/20-foreman_tasks_permissions.rb
2024-07-03T18:27:00 [I|aud|] Filter (142) create event on search owner.id = current_user
2024-07-03T18:27:00 [I|aud|] Filter (142) create event on role_id 1
2024-07-03T18:27:00 [I|aud|] Filter (142) create event on taxonomy_search
2024-07-03T18:27:00 [I|aud|] Filter (142) create event on override false
2024-07-03T18:27:00 [I|aud|] Filter (142) create event on organization_ids []
2024-07-03T18:27:00 [I|aud|] Filter (142) create event on location_ids []
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/20-permissions.rb
2024-07-03T18:27:00 [I|aud|] Filter (143) create event on search user = current_user
2024-07-03T18:27:00 [I|aud|] Filter (143) create event on role_id 1
2024-07-03T18:27:00 [I|aud|] Filter (143) create event on taxonomy_search
2024-07-03T18:27:00 [I|aud|] Filter (143) create event on override false
2024-07-03T18:27:00 [I|aud|] Filter (143) create event on organization_ids []
2024-07-03T18:27:00 [I|aud|] Filter (143) create event on location_ids []
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/foreman-tasks-7.2.1/db/seeds.d/30-notification_blueprints.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/50-notification_blueprints.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/foreman-tasks-7.2.1/db/seeds.d/60-dynflow_proxy_feature.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/60-ssh_proxy_feature.rb
2024-07-03T18:27:00 [I|app|] Seeding /usr/share/gems/gems/foreman-tasks-7.2.1/db/seeds.d/61-foreman_tasks_bookmarks.rb
2024-07-03T18:27:01 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/70-job_templates.rb
2024-07-03T18:27:01 [I|aud|] RemoteExecutionFeature (14) update event on job_template_id , 155
2024-07-03T18:27:01 [I|aud|] RemoteExecutionFeature (15) update event on job_template_id , 162
2024-07-03T18:27:01 [I|app|] Seeding /usr/share/gems/gems/katello-4.7.6/db/seeds.d/75-job_templates.rb
2024-07-03T18:27:01 [I|aud|] RemoteExecutionFeature (10) update event on job_template_id , 166
2024-07-03T18:27:01 [I|aud|] RemoteExecutionFeature (11) update event on job_template_id , 167
2024-07-03T18:27:01 [I|aud|] RemoteExecutionFeature (7) update event on job_template_id , 168
2024-07-03T18:27:01 [I|aud|] RemoteExecutionFeature (1) update event on job_template_id , 169
2024-07-03T18:27:01 [I|aud|] RemoteExecutionFeature (2) update event on job_template_id , 170
2024-07-03T18:27:01 [I|aud|] RemoteExecutionFeature (9) update event on job_template_id , 171
2024-07-03T18:27:02 [I|aud|] RemoteExecutionFeature (5) update event on job_template_id , 172
2024-07-03T18:27:02 [I|aud|] RemoteExecutionFeature (6) update event on job_template_id , 173
2024-07-03T18:27:02 [I|aud|] RemoteExecutionFeature (13) update event on job_template_id , 174
2024-07-03T18:27:02 [I|aud|] RemoteExecutionFeature (12) update event on job_template_id , 175
2024-07-03T18:27:02 [I|aud|] RemoteExecutionFeature (8) update event on job_template_id , 176
2024-07-03T18:27:02 [I|aud|] RemoteExecutionFeature (3) update event on job_template_id , 177
2024-07-03T18:27:02 [I|aud|] RemoteExecutionFeature (4) update event on job_template_id , 178
2024-07-03T18:27:02 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/90-bookmarks.rb
2024-07-03T18:27:02 [I|app|] Seeding /usr/share/gems/gems/foreman_remote_execution-8.2.0/db/seeds.d/95-mail_notifications.rb
2024-07-03T18:27:02 [I|app|] All seed files executed
2024-07-03T18:27:02 [I|aud|] Setting (5) create event on value --- false
 | ...
2024-07-03T18:27:14 [I|app|] Rails cache backend: File
2024-07-03T18:27:16 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:27:18 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:27:18 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:27:23 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:27:24 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:27:24 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:27:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:38 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:39 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:39 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:40 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:41 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:42 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:43 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:44 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:45 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:45 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:46 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:46 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:46 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:46 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:46 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:48 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:49 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:50 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:27:50 [I|app|] No new seed file updates found. Skipping
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] start terminating throttle_limiter...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] start terminating client dispatcher...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] stop listening for new events...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:27:56 [I|dyn|] start terminating clock...
2024-07-03T18:28:07 [I|app|] Rails cache backend: File
2024-07-03T18:28:09 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:28:10 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:28:11 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:28:16 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:28:17 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:28:17 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:28:28 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:28:30 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:28:30 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:28:31 [I|app|] No new seed file updates found. Skipping
2024-07-03T18:29:46 [I|app|] Rails cache backend: File
2024-07-03T18:29:49 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:29:50 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:29:51 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:29:55 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:29:57 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:29:57 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:30:08 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:10 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:10 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:11 [I|app|] Rails cache backend: File
2024-07-03T18:30:11 [I|app|] No new seed file updates found. Skipping
2024-07-03T18:30:13 [I|app|] orchestrator in passive mode
2024-07-03T18:30:13 [I|dyn|] Acquired orchestrator lock, entering active mode.
2024-07-03T18:30:13 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:30:14 [I|app|] Enqueued CreateRssNotifications (Job ID: d51f7a19-31f3-492f-be0a-7509ff034d40) to Dynflow(default)
2024-07-03T18:30:14 [I|app|] Enqueued StoredValuesCleanupJob (Job ID: 0c9769f7-a5e8-4efc-9b4c-dde12706caac) to Dynflow(default)
2024-07-03T18:30:14 [I|app|] Enqueued CreateExpiredManifestNotifications (Job ID: 8608ed3c-81a4-4743-9a4e-edac78f4911b) to Dynflow(default)
2024-07-03T18:30:14 [I|app|] Enqueued CreatePulpDiskSpaceNotifications (Job ID: 5e2df2b3-cbba-463f-a583-56195eef1775) to Dynflow(default)
2024-07-03T18:30:14 [I|app|] Enqueued SendExpireSoonNotifications (Job ID: 86b79a81-1d3a-4962-a549-e1f68d5df2c5) to Dynflow(default)
2024-07-03T18:30:14 [I|app|] Everything ready for world: a8c712d8-0a84-4216-9215-52c309195727
2024-07-03T18:30:14 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:30:15 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:30:20 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:30:22 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:30:22 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:30:24 [I|app|] Rails cache backend: File
2024-07-03T18:30:24 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:26 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:26 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:30:26 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:27 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:30:28 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:30:33 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:30:34 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:30:34 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:30:46 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:47 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:47 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:30:48 [I|app|] No new seed file updates found. Skipping
2024-07-03T18:30:51 [I|app|] Everything ready for world: 82ebdd4c-ae07-4efd-b890-737fe6814f37
2024-07-03T18:30:51 [I|dyn|] Performing validity checks
2024-07-03T18:30:51 [I|app|] Performing CreateRssNotifications (Job ID: d51f7a19-31f3-492f-be0a-7509ff034d40) from Dynflow(default) enqueued at 2024-07-03T13:00:14Z
2024-07-03T18:30:51 [I|app|] Performing CreateExpiredManifestNotifications (Job ID: 8608ed3c-81a4-4743-9a4e-edac78f4911b) from Dynflow(default) enqueued at 2024-07-03T13:00:14Z
2024-07-03T18:30:51 [I|app|] Performing CreatePulpDiskSpaceNotifications (Job ID: 5e2df2b3-cbba-463f-a583-56195eef1775) from Dynflow(default) enqueued at 2024-07-03T13:00:14Z
2024-07-03T18:30:51 [I|app|] Performing StoredValuesCleanupJob (Job ID: 0c9769f7-a5e8-4efc-9b4c-dde12706caac) from Dynflow(default) enqueued at 2024-07-03T13:00:14Z
2024-07-03T18:30:51 [I|app|] Performing SendExpireSoonNotifications (Job ID: 86b79a81-1d3a-4962-a549-e1f68d5df2c5) from Dynflow(default) enqueued at 2024-07-03T13:00:14Z
2024-07-03T18:30:51 [I|dyn|] Finished performing validity checks
2024-07-03T18:30:51 [I|app|] Enqueued CreatePulpDiskSpaceNotifications (Job ID: 45130f4f-3361-4c03-82c8-67577710a9a0) to Dynflow(default) at 2024-07-04 01:00:51 UTC
2024-07-03T18:30:51 [I|app|] Performed CreatePulpDiskSpaceNotifications (Job ID: 5e2df2b3-cbba-463f-a583-56195eef1775) from Dynflow(default) in 191.77ms
2024-07-03T18:30:51 [I|app|] Enqueued StoredValuesCleanupJob (Job ID: 16e96578-d3d8-416d-8ec8-99f2240bc058) to Dynflow(default) at 2024-07-04 01:00:51 UTC with arguments: {}
2024-07-03T18:30:51 [I|app|] Performed StoredValuesCleanupJob (Job ID: 0c9769f7-a5e8-4efc-9b4c-dde12706caac) from Dynflow(default) in 184.77ms
2024-07-03T18:30:51 [I|app|] Enqueued SendExpireSoonNotifications (Job ID: b8ac658a-cecb-4d1c-bfd2-8e64e77db005) to Dynflow(default) at 2024-07-04 01:00:51 UTC
2024-07-03T18:30:51 [I|app|] Performed SendExpireSoonNotifications (Job ID: 86b79a81-1d3a-4962-a549-e1f68d5df2c5) from Dynflow(default) in 188.69ms
2024-07-03T18:30:52 [W|not|] Failed to handle notifications - this is most likely a bug: Katello::Resources::Candlepin::Owner: 404 Not Found {"displayMessage":"Organization with id Default_Organization could not be found.","requestUuid":"4ec6c8c6-f55e-462e-aae4-5c79525d5def"} (GET /candlepin/owners/Default_Organization)
2024-07-03T18:30:52 [I|app|] Enqueued CreateExpiredManifestNotifications (Job ID: df282c54-42ea-4b10-8214-607fcd4eab0c) to Dynflow(default) at 2024-07-04 13:00:52 UTC
2024-07-03T18:30:52 [I|app|] Performed CreateExpiredManifestNotifications (Job ID: 8608ed3c-81a4-4743-9a4e-edac78f4911b) from Dynflow(default) in 1025.27ms
2024-07-03T18:30:52 [I|app|a84a58e6] Started GET "/api/v2/hosts?search=name%3D%22rhss.example.com%22" for 127.0.0.1 at 2024-07-03 18:30:52 +0530
2024-07-03T18:30:52 [I|app|a84a58e6] Katello event daemon started process=36449
2024-07-03T18:30:52 [I|app|a84a58e6] Subscribed to katello.candlepin.candlepin_events
2024-07-03T18:30:53 [I|kat|a84a58e6] Polling Katello Event Queue
2024-07-03T18:30:54 [I|app|] Enqueued CreateRssNotifications (Job ID: d3c04c70-a501-42d9-a1ca-c72498d6dc8e) to Dynflow(default) at 2024-07-04 01:00:54 UTC with arguments: {}
2024-07-03T18:30:54 [I|app|] Performed CreateRssNotifications (Job ID: d51f7a19-31f3-492f-be0a-7509ff034d40) from Dynflow(default) in 2920.81ms
2024-07-03T18:30:54 [F|app|a84a58e6]
 a84a58e6 | RestClient::BadRequest (Katello::Resources::Candlepin::Owner: 400 Bad Request {"displayMessage":"Problem generating ueber cert for owner Default_Organization","requestUuid":"2ce5f13f-4cc7-4e1d-8868-f3081d66df7f"} (POST /candlepin/owners/Default_Organization/uebercert)):
 a84a58e6 |
 a84a58e6 | katello (4.7.6) app/lib/katello/http_resource.rb:92:in `issue_request'
 a84a58e6 | katello (4.7.6) app/lib/katello/http_resource.rb:47:in `block (2 levels) in singleton class'
 a84a58e6 | katello (4.7.6) app/lib/katello/resources/candlepin/owner.rb:101:in `generate_ueber_cert'
 a84a58e6 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:59:in `generate_debug_cert'
 a84a58e6 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:65:in `rescue in load_debug_cert'
 a84a58e6 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:62:in `load_debug_cert'
 a84a58e6 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:14:in `block (2 levels) in included'
 a84a58e6 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `instance_eval'
 a84a58e6 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `run_initializer'
 a84a58e6 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:97:in `lazy_attribute_get'
 a84a58e6 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:44:in `block (2 levels) in lazy_accessor'
 a84a58e6 | katello (4.7.6) app/services/katello/organization_creator.rb:78:in `create_backend_objects!'
 a84a58e6 | katello (4.7.6) app/services/katello/organization_creator.rb:56:in `block in create!'
 a84a58e6 | katello (4.7.6) app/services/katello/organization_creator.rb:53:in `create!'
 a84a58e6 | katello (4.7.6) app/services/katello/organization_creator.rb:25:in `block (2 levels) in create_all_organizations!'
 a84a58e6 | katello (4.7.6) app/services/katello/organization_creator.rb:23:in `block in create_all_organizations!'
 a84a58e6 | app/models/concerns/foreman/thread_session.rb:108:in `as'
 a84a58e6 | app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 a84a58e6 | katello (4.7.6) app/services/katello/organization_creator.rb:22:in `create_all_organizations!'
 a84a58e6 | katello (4.7.6) lib/katello/middleware/organization_created_enforcer.rb:12:in `call'
 a84a58e6 | katello (4.7.6) lib/katello/middleware/event_daemon.rb:10:in `call'
 a84a58e6 | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 a84a58e6 | lib/foreman/middleware/telemetry.rb:10:in `call'
 a84a58e6 | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 a84a58e6 | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 a84a58e6 | katello (4.7.6) lib/katello/prevent_json_parsing.rb:12:in `call'
2024-07-03T18:31:04 [I|app|] Rails cache backend: File
2024-07-03T18:31:06 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2024-07-03T18:31:07 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2024-07-03T18:31:08 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2024-07-03T18:31:13 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2024-07-03T18:31:14 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2024-07-03T18:31:14 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2024-07-03T18:31:25 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:31:26 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:31:27 [W|app|] Scoped order is ignored, it's forced to be batch order.
2024-07-03T18:31:27 [I|app|] No new seed file updates found. Skipping
2024-07-03T18:31:30 [I|app|] Everything ready for world: b685fd95-6335-4dbd-a07a-1e6a7f9c408c
2024-07-03T18:32:08 [I|app|567b0078] Started GET "/api/v2/hosts?search=name%3D%22rhss.example.com%22" for 127.0.0.1 at 2024-07-03 18:32:08 +0530
2024-07-03T18:32:09 [F|app|567b0078]
 567b0078 | RestClient::BadRequest (Katello::Resources::Candlepin::Owner: 400 Bad Request {"displayMessage":"Problem generating ueber cert for owner Default_Organization","requestUuid":"bbf5b568-99b0-4933-b3c3-1cee2db262db"} (POST /candlepin/owners/Default_Organization/uebercert)):
 567b0078 |
 567b0078 | katello (4.7.6) app/lib/katello/http_resource.rb:92:in `issue_request'
 567b0078 | katello (4.7.6) app/lib/katello/http_resource.rb:47:in `block (2 levels) in singleton class'
 567b0078 | katello (4.7.6) app/lib/katello/resources/candlepin/owner.rb:101:in `generate_ueber_cert'
 567b0078 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:59:in `generate_debug_cert'
 567b0078 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:65:in `rescue in load_debug_cert'
 567b0078 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:62:in `load_debug_cert'
 567b0078 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:14:in `block (2 levels) in included'
 567b0078 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `instance_eval'
 567b0078 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `run_initializer'
 567b0078 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:97:in `lazy_attribute_get'
 567b0078 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:44:in `block (2 levels) in lazy_accessor'
 567b0078 | katello (4.7.6) app/services/katello/organization_creator.rb:78:in `create_backend_objects!'
 567b0078 | katello (4.7.6) app/services/katello/organization_creator.rb:56:in `block in create!'
 567b0078 | katello (4.7.6) app/services/katello/organization_creator.rb:53:in `create!'
 567b0078 | katello (4.7.6) app/services/katello/organization_creator.rb:25:in `block (2 levels) in create_all_organizations!'
 567b0078 | katello (4.7.6) app/services/katello/organization_creator.rb:23:in `block in create_all_organizations!'
 567b0078 | app/models/concerns/foreman/thread_session.rb:108:in `as'
 567b0078 | app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 567b0078 | katello (4.7.6) app/services/katello/organization_creator.rb:22:in `create_all_organizations!'
 567b0078 | katello (4.7.6) lib/katello/middleware/organization_created_enforcer.rb:12:in `call'
 567b0078 | katello (4.7.6) lib/katello/middleware/event_daemon.rb:10:in `call'
 567b0078 | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 567b0078 | lib/foreman/middleware/telemetry.rb:10:in `call'
 567b0078 | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 567b0078 | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 567b0078 | katello (4.7.6) lib/katello/prevent_json_parsing.rb:12:in `call'
2024-07-03T18:32:09 [I|app|5676ce5d] Started GET "/api/v2/smart_proxies?search=name%3D%22rhss.example.com%22" for 127.0.0.1 at 2024-07-03 18:32:09 +0530
2024-07-03T18:32:09 [F|app|5676ce5d]
 5676ce5d | RestClient::BadRequest (Katello::Resources::Candlepin::Owner: 400 Bad Request {"displayMessage":"Problem generating ueber cert for owner Default_Organization","requestUuid":"0471e872-7133-4bf3-b2e8-dec5f2dc0ed5"} (POST /candlepin/owners/Default_Organization/uebercert)):
 5676ce5d |
 5676ce5d | katello (4.7.6) app/lib/katello/http_resource.rb:92:in `issue_request'
 5676ce5d | katello (4.7.6) app/lib/katello/http_resource.rb:47:in `block (2 levels) in singleton class'
 5676ce5d | katello (4.7.6) app/lib/katello/resources/candlepin/owner.rb:101:in `generate_ueber_cert'
 5676ce5d | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:59:in `generate_debug_cert'
 5676ce5d | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:65:in `rescue in load_debug_cert'
 5676ce5d | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:62:in `load_debug_cert'
 5676ce5d | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:14:in `block (2 levels) in included'
 5676ce5d | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `instance_eval'
 5676ce5d | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `run_initializer'
 5676ce5d | katello (4.7.6) app/lib/katello/lazy_accessor.rb:97:in `lazy_attribute_get'
 5676ce5d | katello (4.7.6) app/lib/katello/lazy_accessor.rb:44:in `block (2 levels) in lazy_accessor'
 5676ce5d | katello (4.7.6) app/services/katello/organization_creator.rb:78:in `create_backend_objects!'
 5676ce5d | katello (4.7.6) app/services/katello/organization_creator.rb:56:in `block in create!'
 5676ce5d | katello (4.7.6) app/services/katello/organization_creator.rb:53:in `create!'
 5676ce5d | katello (4.7.6) app/services/katello/organization_creator.rb:25:in `block (2 levels) in create_all_organizations!'
 5676ce5d | katello (4.7.6) app/services/katello/organization_creator.rb:23:in `block in create_all_organizations!'
 5676ce5d | app/models/concerns/foreman/thread_session.rb:108:in `as'
 5676ce5d | app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 5676ce5d | katello (4.7.6) app/services/katello/organization_creator.rb:22:in `create_all_organizations!'
 5676ce5d | katello (4.7.6) lib/katello/middleware/organization_created_enforcer.rb:12:in `call'
 5676ce5d | katello (4.7.6) lib/katello/middleware/event_daemon.rb:10:in `call'
 5676ce5d | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 5676ce5d | lib/foreman/middleware/telemetry.rb:10:in `call'
 5676ce5d | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 5676ce5d | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 5676ce5d | katello (4.7.6) lib/katello/prevent_json_parsing.rb:12:in `call'
2024-07-03T18:32:09 [I|app|69ffe2c0] Started GET "/api/v2/smart_proxies?search=name%3D%22rhss.example.com%22" for 127.0.0.1 at 2024-07-03 18:32:09 +0530
2024-07-03T18:32:10 [F|app|69ffe2c0]
 69ffe2c0 | RestClient::BadRequest (Katello::Resources::Candlepin::Owner: 400 Bad Request {"displayMessage":"Problem generating ueber cert for owner Default_Organization","requestUuid":"ef38987a-51f8-46dd-abe9-6df120fd06bc"} (POST /candlepin/owners/Default_Organization/uebercert)):
 69ffe2c0 |
 69ffe2c0 | katello (4.7.6) app/lib/katello/http_resource.rb:92:in `issue_request'
 69ffe2c0 | katello (4.7.6) app/lib/katello/http_resource.rb:47:in `block (2 levels) in singleton class'
 69ffe2c0 | katello (4.7.6) app/lib/katello/resources/candlepin/owner.rb:101:in `generate_ueber_cert'
 69ffe2c0 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:59:in `generate_debug_cert'
 69ffe2c0 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:65:in `rescue in load_debug_cert'
 69ffe2c0 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:62:in `load_debug_cert'
 69ffe2c0 | katello (4.7.6) app/models/katello/glue/candlepin/owner.rb:14:in `block (2 levels) in included'
 69ffe2c0 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `instance_eval'
 69ffe2c0 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:118:in `run_initializer'
 69ffe2c0 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:97:in `lazy_attribute_get'
 69ffe2c0 | katello (4.7.6) app/lib/katello/lazy_accessor.rb:44:in `block (2 levels) in lazy_accessor'
 69ffe2c0 | katello (4.7.6) app/services/katello/organization_creator.rb:78:in `create_backend_objects!'
 69ffe2c0 | katello (4.7.6) app/services/katello/organization_creator.rb:56:in `block in create!'
 69ffe2c0 | katello (4.7.6) app/services/katello/organization_creator.rb:53:in `create!'
 69ffe2c0 | katello (4.7.6) app/services/katello/organization_creator.rb:25:in `block (2 levels) in create_all_organizations!'
 69ffe2c0 | katello (4.7.6) app/services/katello/organization_creator.rb:23:in `block in create_all_organizations!'
 69ffe2c0 | app/models/concerns/foreman/thread_session.rb:108:in `as'
 69ffe2c0 | app/models/concerns/foreman/thread_session.rb:114:in `as_anonymous_admin'
 69ffe2c0 | katello (4.7.6) app/services/katello/organization_creator.rb:22:in `create_all_organizations!'
 69ffe2c0 | katello (4.7.6) lib/katello/middleware/organization_created_enforcer.rb:12:in `call'
 69ffe2c0 | katello (4.7.6) lib/katello/middleware/event_daemon.rb:10:in `call'
 69ffe2c0 | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 69ffe2c0 | lib/foreman/middleware/telemetry.rb:10:in `call'
 69ffe2c0 | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 69ffe2c0 | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 69ffe2c0 | katello (4.7.6) lib/katello/prevent_json_parsing.rb:12:in `call'

Weird.

I can’t reproduce your error on a fresh CentOS Stream 8 install (which should mostly match Rocky 8.10) with Katello 4.7.

Is that a clean system you’re installing on, or could it be there are any existing traces of Candlepin already present and thus producing weird behaviour?