Problem:
could not access to foreman dashboard due to failed services
Expected outcome:
all services are up and could access to dashboard
Foreman and Proxy versions:
Foreman 3.1.1
Foreman and Proxy plugin versions:
Distribution and version:
Other relevant data:
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:14:in block in loading' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/share_lock.rb:151:in exclusive’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:13:in loading' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:40:in load_interlock’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:389:in require_or_load' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:544:in load_missing_constant’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:214:in const_missing' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-7.0.2/lib/foreman_ansible/register.rb:189:in block (2 levels) in <top (required)>’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/app/registries/foreman/plugin/graphql_types_registry.rb:53:in class_eval' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/app/registries/foreman/plugin/graphql_types_registry.rb:53:in block (2 levels) in realise_block_extensions’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/app/registries/foreman/plugin/graphql_types_registry.rb:52:in each' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/app/registries/foreman/plugin/graphql_types_registry.rb:52:in block in realise_block_extensions’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/app/registries/foreman/plugin/graphql_types_registry.rb:50:in each' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/app/registries/foreman/plugin/graphql_types_registry.rb:50:in realise_block_extensions’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/app/registries/foreman/plugin/graphql_types_registry.rb:46:in realise_extensions' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/config/application.rb:322:in block in class:Application’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:428:in instance_exec' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:428:in block in make_lambda’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:200:in block (2 levels) in halting' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:605:in block (2 levels) in default_terminator’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:604:in catch' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:604:in block in default_terminator’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:201:in block in halting' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:513:in block in invoke_before’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:513:in each' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:513:in invoke_before’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:134:in run_callbacks' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/reloader.rb:88:in prepare!’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application/finisher.rb:113:in block in <module:Finisher>' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:32:in instance_exec’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:32:in run' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/config/initializers/0_print_time_spent.rb:45:in block in run’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/config/initializers/0_print_time_spent.rb:17:in benchmark' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/config/initializers/0_print_time_spent.rb:45:in run’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:61:in block in run_initializers' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:228:in block in tsort_each’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:350:in block (2 levels) in each_strongly_connected_component' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:431:in each_strongly_connected_component_from’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:349:in block in each_strongly_connected_component' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in each’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in call' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in each_strongly_connected_component’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:226:in tsort_each' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:205:in tsort_each’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:60:in run_initializers' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application.rb:363:in initialize!’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in public_send' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in method_missing’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/config/environment.rb:5:in <top (required)>' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in require’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in require' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in block in require' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:291:in load_dependency’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in require' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /usr/share/foreman/extras/dynflow-sidekiq.rb:21:in <top (required)>’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in require' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in require’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/cli.rb:292:in boot_system' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/cli.rb:46:in run’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/bin/sidekiq:12:in <top (required)>' Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/bin/sidekiq:23:in load’
Feb 7 13:23:04 theforeman dynflow-sidekiq@orchestrator: /opt/theforeman/tfm/root/usr/bin/sidekiq:23:in `’
Feb 7 13:23:04 theforeman systemd: dynflow-sidekiq@orchestrator.service: main process exited, code=exited, status=1/FAILURE
Feb 7 13:23:04 theforeman systemd: Failed to start Foreman jobs daemon - orchestrator on sidekiq.
Feb 7 13:23:04 theforeman systemd: Unit dynflow-sidekiq@orchestrator.service entered failed state.
Feb 7 13:23:04 theforeman systemd: dynflow-sidekiq@orchestrator.service failed.
Feb 7 13:23:06 theforeman systemd: dynflow-sidekiq@orchestrator.service holdoff time over, scheduling restart.