Luna nightly pipeline failed:
https://ci.theforeman.org/job/luna-nightly-rpm-pipeline/272/
foreman-pipeline-luna-nightly-centos8-stream-install (failed) (remote job)
foreman-pipeline-luna-nightly-centos8-install (failed) (remote job)
foreman-pipeline-luna-nightly-centos7-install (failed) (remote job)
foreman-pipeline-luna-nightly-centos8-upgrade (failed) (remote job)
foreman-pipeline-luna-nightly-centos7-upgrade (failed) (remote job)
Some errors I found in the log:
2021-04-30T03:11:39 [E|kat|] GET: https://localhost/pulp/api/v2/consumers/: {"content_type"=>"application/json", "accept"=>"application/json"}
| SSL_connect returned=1 errno=0 state=error: certificate verify failed (error number 1):
2021-04-30T03:11:39 [W|app|] Failed upgrade task: katello:clean_backend_objects
2021-04-30T03:11:39 [I|app|] Backtrace for 'Failed upgrade task: katello:clean_backend_objects' error (RestClient::SSLCertificateNotVerified): SSL_connect returned=1 errno=0 state=error: certificate verify failed (error number 1)
| /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:758:in `rescue in transmit'
| /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:642:in `transmit'
| /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute'
| /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/resource.rb:51:in `get'
| /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-2.13.1/lib/runcible/base.rb:94:in `get_response'
| /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-2.13.1/lib/runcible/base.rb:74:in `call'
| /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-2.13.1/lib/runcible/resources/consumer.rb:28:in `retrieve'
| /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-2.13.1/lib/runcible/resources/consumer.rb:35:in `retrieve_all'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0.pre.master/lib/katello/tasks/clean_backend_objects.rake:16:in `populate!'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0.pre.master/lib/katello/tasks/clean_backend_objects.rake:106:in `block (2 levels) in <top (required)>'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `block in execute'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `each'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `execute'
| /usr/share/foreman/lib/tasks/upgrade.rake:32:in `run_task'
| /usr/share/foreman/lib/tasks/upgrade.rake:27:in `block (3 levels) in <top (required)>'
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/delegation.rb:87:in `each'
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/delegation.rb:87:in `each'
| /usr/share/foreman/lib/tasks/upgrade.rake:19:in `each_with_index'
| /usr/share/foreman/lib/tasks/upgrade.rake:19:in `block (2 levels) in <top (required)>'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `block in execute'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `each'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `execute'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:199:in `synchronize'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:188:in `invoke'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:160:in `invoke_task'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:116:in `each'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:116:in `block in top_level'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:125:in `run_with_threads'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:110:in `top_level'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:83:in `block in run'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:80:in `run'
| /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
| /opt/rh/rh-ruby27/root/usr/bin/rake:23:in `load'
| /opt/rh/rh-ruby27/root/usr/bin/rake:23:in `<main>'
Why it’s using localhost is beyond me. Probably because this is still the Pulp 2 API and there is no more config for it. Feels like a leftover cleanup job.
Then there’s this:
2021-04-30T03:13:48 [I|app|44071ab7] Started POST "/api/job_invocations" for 192.168.121.2 at 2021-04-30 03:13:48 +0000
2021-04-30T03:13:48 [I|app|44071ab7] Processing by Api::V2::JobInvocationsController#create as JSON
2021-04-30T03:13:48 [I|app|44071ab7] Parameters: {"job_invocation"=>{"job_template_id"=>156, "inputs"=>{"command"=>"uptime"}, "ssh"=>{}, "recurrence"=>{}, "scheduling"=>{}, "concurrency_control"=>{}, "search_query"=>"name = pipe-luna-server-nightly-centos7.n16.example.com", "targeting_type"=>"static_query"}, "apiv"=>"v2"}
2021-04-30T03:13:48 [I|app|44071ab7] Authorized user admin(Admin User)
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on job_category Commands
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on description Run uptime
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on concurrency_level
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on time_span
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on execution_timeout_interval
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on password [redacted]
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on key_passphrase
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on remote_execution_feature_id
2021-04-30T03:13:48 [I|aud|44071ab7] JobInvocation (1) create event on effective_user_password
2021-04-30T03:13:48 [I|bac|44071ab7] Task {label: Actions::RemoteExecution::RunHostsJob, id: 4896031c-9564-4749-ab2a-58db56ca164b, execution_plan_id: 4d53901b-1ee5-455b-b1da-c79335b67321} state changed: planning
2021-04-30T03:13:48 [I|bac|44071ab7] Task {label: Actions::RemoteExecution::RunHostsJob, id: 4896031c-9564-4749-ab2a-58db56ca164b, execution_plan_id: 4d53901b-1ee5-455b-b1da-c79335b67321} state changed: planned
2021-04-30T03:13:48 [I|app|44071ab7] Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-4.3.0/app/views/api/v2/job_invocations/create.json.rabl
2021-04-30T03:13:49 [I|bac|44071ab7] Task {label: Actions::RemoteExecution::RunHostsJob, id: 4896031c-9564-4749-ab2a-58db56ca164b, execution_plan_id: 4d53901b-1ee5-455b-b1da-c79335b67321} state changed: running
2021-04-30T03:13:49 [I|app|44071ab7] Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-4.3.0/app/views/api/v2/job_invocations/create.json.rabl (Duration: 58.0ms | Allocations: 9695)
2021-04-30T03:13:49 [I|app|44071ab7] Completed 201 Created in 769ms (Views: 68.5ms | ActiveRecord: 167.8ms | Allocations: 110763)
2021-04-30T03:13:49 [I|bac|44071ab7] Task {label: Actions::RemoteExecution::RunHostJob, id: 242aff37-a976-4a00-8b00-2415004378c8, execution_plan_id: 7f4a376b-4ec3-48c2-9e6b-99e4959fa055} state changed: planning
2021-04-30T03:13:49 [W|app|44071ab7] Could not fetch task counts from pipe-luna-server-nightly-centos7.n16.example.com, skipped.
2021-04-30T03:13:49 [I|app|44071ab7] Backtrace for 'Could not fetch task counts from pipe-luna-server-nightly-centos7.n16.example.com, skipped.' error (RestClient::Forbidden): 403 Forbidden
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in `exception_with_response'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in `return!'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in `process_result'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in `block in transmit'
44071ab7 | /opt/rh/rh-ruby27/root/usr/share/ruby/net/http.rb:933:in `start'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:715:in `transmit'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/resource.rb:51:in `get'
44071ab7 | /usr/share/foreman/lib/proxy_api/resource.rb:72:in `block (2 levels) in get'
44071ab7 | /usr/share/foreman/app/services/foreman/telemetry_helper.rb:27:in `telemetry_duration_histogram'
44071ab7 | /usr/share/foreman/lib/proxy_api/resource.rb:69:in `block in get'
44071ab7 | /usr/share/foreman/lib/proxy_api/resource.rb:112:in `with_logger'
44071ab7 | /usr/share/foreman/lib/proxy_api/resource.rb:68:in `get'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/proxy_api/foreman_dynflow/dynflow_proxy.rb:35:in `tasks_count'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/services/foreman_tasks/proxy_selector.rb:48:in `block in get_counts'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/services/foreman_tasks/proxy_selector.rb:45:in `each'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/services/foreman_tasks/proxy_selector.rb:45:in `each_with_object'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/services/foreman_tasks/proxy_selector.rb:45:in `get_counts'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/services/foreman_tasks/proxy_selector.rb:35:in `select_by_jobs_count'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/services/foreman_tasks/proxy_selector.rb:25:in `block in determine_proxy'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/services/foreman_tasks/proxy_selector.rb:23:in `each'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/services/foreman_tasks/proxy_selector.rb:23:in `determine_proxy'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-4.3.0/app/lib/actions/remote_execution/run_host_job.rb:181:in `determine_proxy!'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-4.3.0/app/lib/actions/remote_execution/run_host_job.rb:37:in `plan'
<CUT>
2021-04-30T03:13:49 [E|bac|44071ab7] The only applicable proxy pipe-luna-server-nightly-centos7.n16.example.com is down (RuntimeError)
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-4.3.0/app/lib/actions/remote_execution/run_host_job.rb:185:in `determine_proxy!'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-4.3.0/app/lib/actions/remote_execution/run_host_job.rb:37:in `plan'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:514:in `block (3 levels) in execute_plan'
<CUT>
44071ab7 | [ sidekiq ]
44071ab7 | [ concurrent-ruby ]
2021-04-30T03:13:49 [I|bac|44071ab7] Task {label: Actions::RemoteExecution::RunHostJob, id: 242aff37-a976-4a00-8b00-2415004378c8, execution_plan_id: 7f4a376b-4ec3-48c2-9e6b-99e4959fa055} state changed: stopped result: error
2021-04-30T03:13:49 [I|bac|44071ab7] Task {label: Actions::RemoteExecution::RunHostJob, id: 242aff37-a976-4a00-8b00-2415004378c8, execution_plan_id: 7f4a376b-4ec3-48c2-9e6b-99e4959fa055} state changed: stopped result: error
2021-04-30T03:13:49 [W|app|44071ab7] Scoped order is ignored, it's forced to be batch order.
2021-04-30T03:13:49 [W|app|44071ab7] Scoped order is ignored, it's forced to be batch order.
2021-04-30T03:13:59 [E|bac|44071ab7] A sub task failed (RuntimeError)
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/with_sub_plans.rb:231:in `check_for_errors!'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/with_sub_plans.rb:137:in `try_to_finish'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/with_polling_sub_plans.rb:19:in `poll'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/with_polling_sub_plans.rb:11:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-4.3.0/app/lib/actions/remote_execution/run_hosts_job.rb:97:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:571:in `block (3 levels) in execute_run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/proxy_batch_triggering.rb:9:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/watch_delegated_proxy_sub_tasks.rb:17:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.5/lib/active_support/execution_wrapper.rb:88:in `wrap'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:17:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.0/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/world.rb:31:in `execute'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:570:in `block (2 levels) in execute_run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in `catch'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in `block in execute_run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `block in with_error_handling'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `catch'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `with_error_handling'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:564:in `execute_run'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:285:in `execute'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/director.rb:93:in `execute'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors.rb:18:in `run_user_code'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
44071ab7 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
44071ab7 | [ sidekiq ]
44071ab7 | [ concurrent-ruby ]
2021-04-30T03:13:59 [I|bac|44071ab7] Task {label: Actions::RemoteExecution::RunHostsJob, id: 4896031c-9564-4749-ab2a-58db56ca164b, execution_plan_id: 4d53901b-1ee5-455b-b1da-c79335b67321} state changed: stopped result: warning
2021-04-30T03:13:59 [I|bac|44071ab7] Task {label: Actions::RemoteExecution::RunHostsJob, id: 4896031c-9564-4749-ab2a-58db56ca164b, execution_plan_id: 4d53901b-1ee5-455b-b1da-c79335b67321} state changed: stopped result: warning
This is likely what causes the REX failures but this requires some more digging.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.