Remote Execution Failing for new Smart Proxy Server

Support
1

Problem:
Executing remote jobs failing

Expected outcome:
Run remote jobs w/o error.

Foreman and Proxy versions:
foreman 3.4
Smart proxy 3.4

Foreman and Proxy plugin versions:
katello 4.6

Distribution and version:
Rocky Linux 8.6
Other relevant data:

Blockquote
Action:
Actions::RemoteExecution::RunHostJob
Input:
{“host”=>{“id”=>5, “name”=>“mltlws-web-uat-01.speech.ready.sg”},
“job_category”=>“Packages”,
“description”=>“Check for package updates”,
“job_invocation_id”=>44,
“job_features”=>,
“proxy_id”=>2,
“delegated_action_id”=>2,
“current_request_id”=>“28114a9c-536b-4d82-bb59-555e7b6338d8”,
“current_timezone”=>“Asia/Singapore”,
“current_user_id”=>4,
“current_organization_id”=>1,
“current_location_id”=>2}
Output:
{}
Exception:
StandardError: Job execution failed
Backtrace:
/usr/share/gems/gems/algebrick-0.7.5/lib/algebrick/matchers/abstract.rb:74:in assigns' /usr/share/gems/gems/algebrick-0.7.5/lib/algebrick/matching.rb:56:in match_value’
/usr/share/gems/gems/algebrick-0.7.5/lib/algebrick/matching.rb:36:in block in match?' /usr/share/gems/gems/algebrick-0.7.5/lib/algebrick/matching.rb:35:in each’
/usr/share/gems/gems/algebrick-0.7.5/lib/algebrick/matching.rb:35:in match?' /usr/share/gems/gems/algebrick-0.7.5/lib/algebrick/matching.rb:23:in match’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/error.rb:13:in new' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:512:in set_error’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:475:in error!' /usr/share/gems/gems/foreman_remote_execution-8.0.0/app/lib/actions/remote_execution/run_host_job.rb:101:in check_exit_status’
/usr/share/gems/gems/foreman_remote_execution-8.0.0/app/lib/actions/remote_execution/run_host_job.rb:72:in finalize' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:604:in block (2 levels) in execute_finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:40:in finalize' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/rails_executor_wrap.rb:20:in block in finalize' /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/execution_wrapper.rb:91:in wrap’
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/rails_executor_wrap.rb:19:in finalize' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/progress.rb:31:in with_progress_calculation' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/progress.rb:23:in finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/load_setting_values.rb:25:in finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_request_id.rb:19:in block in finalize’
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_request_id.rb:52:in restore_current_request_id' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_request_id.rb:19:in finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_timezone.rb:19:in block in finalize’
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_timezone.rb:44:in restore_curent_timezone' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_timezone.rb:19:in finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:25:in block in finalize’
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:54:in restore_curent_user' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:25:in finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_taxonomies.rb:19:in block in finalize’
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in restore_current_taxonomies' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_taxonomies.rb:19:in finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/world.rb:31:in execute’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:603:in block in execute_finalize' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in block in with_error_handling’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in catch' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in with_error_handling’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:602:in execute_finalize' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:296:in execute’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in block (2 levels) in execute' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in with_meta_calculation’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in block in execute' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in open_action’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in execute' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director/sequential_manager.rb:78:in run_step’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director/sequential_manager.rb:63:in dispatch' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director/sequential_manager.rb:70:in block in run_in_sequence’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director/sequential_manager.rb:70:in all?' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director/sequential_manager.rb:70:in run_in_sequence’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director/sequential_manager.rb:59:in dispatch' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director/sequential_manager.rb:28:in block in finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:48:in finalize_phase' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:48:in finalize_phase' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:48:in finalize_phase' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:48:in finalize_phase' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:48:in finalize_phase' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:48:in finalize_phase' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:29:in block in finalize_phase' /usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:54:in restore_curent_user’
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:29:in finalize_phase' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:48:in finalize_phase' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/world.rb:31:in execute' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director/sequential_manager.rb:27:in finalize’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director.rb:143:in execute' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in block (2 levels) in perform’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors.rb:18:in run_user_code' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in block in perform’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in with_telemetry' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in perform’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in perform' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:192:in execute_job’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:165:in block (2 levels) in process' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/middleware/chain.rb:128:in block in invoke’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/middleware/chain.rb:133:in invoke' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:164:in block in process’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:137:in block (6 levels) in dispatch' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/job_retry.rb:109:in local’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:136:in block (5 levels) in dispatch' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq.rb:37:in block in module:Sidekiq’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:132:in block (4 levels) in dispatch' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:250:in stats’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:127:in block (3 levels) in dispatch' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/job_logger.rb:8:in call’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:126:in block (2 levels) in dispatch' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/job_retry.rb:74:in global’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:125:in block in dispatch' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/logging.rb:48:in with_context’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/logging.rb:42:in with_job_hash_context' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:124:in dispatch’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:163:in process' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:83:in process_one’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:71:in run' /usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/util.rb:16:in watchdog’
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/util.rb:25:in block in safe_thread' /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in block in create_with_logging_context’

2

What’s in the live output? Please do post logs from the smart proxy /var/glo/foreman-proxy/proxy.log. What you gave us so far doesn’t give anything to go by

3

Please see logs below.

2022-11-09T17:11:30 [E] Error initializing command - RuntimeError Failed to execute script on remote machine, exit code: 126.:
/usr/share/gems/gems/smart_proxy_remote_execution_ssh-0.8.0/lib/smart_proxy_remote_execution_ssh/runners/script_runner.rb:370:in ensure_remote_command' /usr/share/gems/gems/smart_proxy_remote_execution_ssh-0.8.0/lib/smart_proxy_remote_execution_ssh/runners/script_runner.rb:165:in preflight_checks’
/usr/share/gems/gems/smart_proxy_remote_execution_ssh-0.8.0/lib/smart_proxy_remote_execution_ssh/runners/script_runner.rb:150:in start' /usr/share/gems/gems/smart_proxy_dynflow-0.8.1/lib/smart_proxy_dynflow/runner/dispatcher.rb:32:in start_runner’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/actor.rb:13:in on_message' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/context.rb:46:in on_envelope’
/usr/share/gems/gems/smart_proxy_dynflow-0.8.1/lib/smart_proxy_dynflow/runner/dispatcher.rb:24:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/executes_context.rb:7:in on_envelope’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/actor.rb:122:in on_envelope’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/awaits.rb:15:in on_envelope’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/sets_results.rb:14:in on_envelope’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/actor.rb:56:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/buffer.rb:38:in process_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/buffer.rb:31:in process_envelopes?’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/buffer.rb:20:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/termination.rb:55:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/removes_child.rb:10:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/sets_results.rb:14:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/core.rb:162:in process_envelope’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/core.rb:96:in block in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/core.rb:119:in block (2 levels) in schedule_execution’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in block in synchronize' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in synchronize’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in synchronize' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/core.rb:116:in block in schedule_execution’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/serialized_execution.rb:18:in call' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/serialized_execution.rb:96:in work’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/serialized_execution.rb:77:in block in call_job' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:352:in run_task’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:343:in block (3 levels) in create_worker' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in loop’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in block (2 levels) in create_worker' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in catch’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in block in create_worker' /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in block in create_with_logging_context’
2022-11-09T17:11:30 28114a9c [E] Script execution failed
2022-11-09T17:11:40 28114a9c [E] A sub task failed
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/with_sub_plans.rb:231:in check_for_errors!' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/with_sub_plans.rb:137:in try_to_finish’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/with_polling_sub_plans.rb:19:in poll' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/with_polling_sub_plans.rb:11:in run’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:582:in block (3 levels) in execute_run' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/progress.rb:31:in with_progress_calculation’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/progress.rb:17:in run' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/smart_proxy_dynflow-0.8.1/lib/smart_proxy_dynflow/middleware/keep_current_request_id.rb:15:in block in run' /usr/share/gems/gems/smart_proxy_dynflow-0.8.1/lib/smart_proxy_dynflow/middleware/keep_current_request_id.rb:49:in restore_current_request_id’
/usr/share/gems/gems/smart_proxy_dynflow-0.8.1/lib/smart_proxy_dynflow/middleware/keep_current_request_id.rb:15:in run' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in pass’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:32:in run' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in call’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/world.rb:31:in execute' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:581:in block (2 levels) in execute_run’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:580:in catch' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:580:in block in execute_run’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in block in with_error_handling' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in catch’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in with_error_handling' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:575:in execute_run’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:296:in execute' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in block (2 levels) in execute’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in with_meta_calculation' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in block in execute’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in open_action' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in execute’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director.rb:94:in execute' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/parallel/worker.rb:15:in block in on_message’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors.rb:18:in run_user_code' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/parallel/worker.rb:14:in on_message’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/context.rb:46:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/executes_context.rb:7:in on_envelope’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass' /usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/actor.rb:122:in on_envelope’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/awaits.rb:15:in on_envelope’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/sets_results.rb:14:in on_envelope’
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/actor.rb:56:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/buffer.rb:38:in process_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/buffer.rb:31:in process_envelopes?’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/buffer.rb:20:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/termination.rb:55:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/removes_child.rb:10:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/abstract.rb:25:in pass’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/behaviour/sets_results.rb:14:in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/core.rb:162:in process_envelope’
/usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/core.rb:96:in block in on_envelope' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/core.rb:119:in block (2 levels) in schedule_execution’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in block in synchronize' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in synchronize’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/synchronization/mutex_lockable_object.rb:47:in synchronize' /usr/share/gems/gems/concurrent-ruby-edge-0.6.0/lib/concurrent-ruby-edge/concurrent/actor/core.rb:116:in block in schedule_execution’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/serialized_execution.rb:18:in call' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/serialized_execution.rb:96:in work’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/serialized_execution.rb:77:in block in call_job' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:352:in run_task’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:343:in block (3 levels) in create_worker' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in loop’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:334:in block (2 levels) in create_worker' /usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in catch’
/usr/share/gems/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/ruby_thread_pool_executor.rb:333:in block in create_worker' /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in block in create_with_logging_context’

4

Do you happen to have /var/tmp mounted with noexec?

5

Yup. I have that for the target machine.

6

Well then that’s it. You either need to either mount the location with exec or changed the remote_working_dir in /etc/foreman-proxy/settings.d/remote_execution_ssh.yml to some location that is not mounted with noexec

1 Like