kimpsik
September 6, 2021, 7:03am
1
Hi All!
Problem:
Error message: the server returns an error
HTTP status code: 500
Response headers: {"date"=>"Mon, 06 Sep 2021 06:46:27 GMT", "server"=>"gunicorn", "content-type"=>"text/html; charset=UTF-8", "x-frame-options"=>"SAMEORIGIN", "content-length"=>"27", "vary"=>"Cookie", "correlation-id"=>"b3eee2a3c22f468c8bafaa5f9c34130c", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 foreman.xx.xxx", "connection"=>"close"}
Response body: <h1>Server Error (500)</h1>
Any ideas, what may cause that ?
Distribution and version:
When you see the 500 error, we typically need the production.log from that time, otherwise we can’t tell what the issue is.
kimpsik
September 16, 2021, 11:25am
3
Sure! Here is the output from production.log after i tried to run task “Remove Orphans”
2021-09-16T14:17:46 [I|bac|] Task {label: Actions::Katello::OrphanCleanup::RemoveOrphans, id: 1a803dff-69de-4276-b2ec-b4c5782e48f5, execution_plan_id: e8803f91-9ef0-4674-8d29-ce0cc7cc23f7} state changed: runn
ing
2021-09-16T14:17:47 [E|bac|] Error message: the server returns an error
| HTTP status code: 500
| Response headers: {"date"=>"Thu, 16 Sep 2021 11:17:47 GMT", "server"=>"gunicorn", "content-type"=>"text/html; charset=UTF-8", "x-frame-options"=>"SAMEORIGIN", "content-length"=>"27", "vary"=>"Cookie", "cor
relation-id"=>"331b4fdacca74987ab22adb57580fc3b", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 foreman.xx.xx", "connection"=>"close"}
| Response body: <h1>Server Error (500)</h1> (PulpRpmClient::ApiError)
| /opt/theforeman/tfm/root/usr/share/gems/gems/pulp_rpm_client-3.13.3/lib/pulp_rpm_client/api_client.rb:83:in `call_api'
| /opt/theforeman/tfm/root/usr/share/gems/gems/pulp_rpm_client-3.13.3/lib/pulp_rpm_client/api/repositories_rpm_versions_api.rb:196:in `list_with_http_info'
| /opt/theforeman/tfm/root/usr/share/gems/gems/pulp_rpm_client-3.13.3/lib/pulp_rpm_client/api/repositories_rpm_versions_api.rb:109:in `list'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/api/core.rb:165:in `block in versions_list_for_repository'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/api/core.rb:210:in `block in fetch_from_list'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/api/core.rb:205:in `loop'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/api/core.rb:205:in `fetch_from_list'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/api/core.rb:165:in `versions_list_for_repository'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/api/core.rb:158:in `block in repository_versions'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/api/core.rb:157:in `collect'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/api/core.rb:157:in `repository_versions'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/smart_proxy_repository.rb:59:in `block in orphan_repository_versions'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/smart_proxy_repository.rb:57:in `each'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/smart_proxy_repository.rb:57:in `orphan_repository_versions'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/services/katello/pulp3/smart_proxy_repository.rb:39:in `delete_orphan_repository_versions'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/lib/actions/pulp3/orphan_cleanup/delete_orphan_repository_versions.rb:10:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:572:in `block (3 levels) in execute_run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:32:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.3/app/lib/actions/middleware/remote_action.rb:16:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/execution_wrapper.rb:88:in `wrap'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/execution_wrapper.rb:88:in `wrap'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action/progress.rb:17:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/load_setting_values.rb:20:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:32:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:27:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware.rb:19:in `pass'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.5/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/stack.rb:23:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/middleware/world.rb:31:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:571:in `block (2 levels) in execute_run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:570:in `catch'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:570:in `block in execute_run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:473:in `block in with_error_handling'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:473:in `catch'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:473:in `with_error_handling'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:565:in `execute_run'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/action.rb:286:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/director.rb:69:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/executors.rb:18:in `run_user_code'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.9/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
| [ sidekiq ]
| [ concurrent-ruby ]
2021-09-16T14:17:47 [I|bac|] Task {label: Actions::Katello::OrphanCleanup::RemoveOrphans, id: 1a803dff-69de-4276-b2ec-b4c5782e48f5, execution_plan_id: e8803f91-9ef0-4674-8d29-ce0cc7cc23f7} state changed: paused result: error
