Ansible dynamic inventory running forever

wernerma · March 23, 2020, 7:44pm

Problem:
We are using Ansible and AWX to get an inventory from forman.
After 2.4 the script use the report feature to get the hosts for ansible.
It worked only sometimes and i cannot find out why. It seems to be that’s the task running forever:

2020-03-23T20:28:27 [I|app|c922290b] Processing by Api::V2::ReportTemplatesController#report_data as JSON
2020-03-23T20:28:27 [I|app|c922290b] Parameters: {“apiv”=>“v2”, “id”=>“178-Ansible Inventory”, “job_id”=>“3d862f77-a3f9-4603-b74d-4ad390169bab”}
2020-03-23T20:28:27 [I|app|c922290b] Completed 204 No Content in 74ms (ActiveRecord: 14.4ms)

An never ends.

Expected outcome:

Ansible Inventory with the script on an short time :

github.com

ansible/ansible/blob/stable-2.9/contrib/inventory/foreman.py

#!/usr/bin/env python
# vim: set fileencoding=utf-8 :
#
# Copyright (C) 2016 Guido Günther <agx@sigxcpu.org>,
#                    Daniel Lobato Garcia <dlobatog@redhat.com>
#
# This script is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with it.  If not, see <http://www.gnu.org/licenses/>.
#
# This is somewhat based on cobbler inventory

This file has been truncated. show original

Foreman and Proxy versions:
2.4.2

Foreman and Proxy plugin versions:
Katello 3.14

Distribution and version:
Centos 7.7

Other relevant data:

Marek_Hulan · March 24, 2020, 8:45am

Can you take a look at Monitor -> Tasks and see if the rendering eventually finished? Or if there’s some error. It may just take a long time, try to limit the data in the response, e.g. omit facts.

wernerma · March 24, 2020, 10:45am

Hi thanks for your responce.

Yes it is long running an than crashed, see below.

Can you tell me how I can reduce the facts? I only need the hostname, hostgroups, collections.

2020-03-23T13:34:13 [E|app|338d7ce4] Error performing TemplateRenderJob (Job ID: 129c6844-a829-44b2-bf84-144de4ab1c42) from Dynflow(default) in 208198.91ms: Timeout::Error (execution expired):
/opt/theforeman/tfm/root/usr/share/gems/gems/sexp_processor-4.10.0/lib/sexp.rb:32:in initialize' /opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser_extras.rb:1147:in new’
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser_extras.rb:1147:in s' /opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby24_parser.rb:5847:in _reduce_448’
/opt/rh/rh-ruby25/root/usr/share/ruby/racc/parser.rb:259:in _racc_do_parse_c' /opt/rh/rh-ruby25/root/usr/share/ruby/racc/parser.rb:259:in do_parse’
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser_extras.rb:1087:in block in process' /opt/rh/rh-ruby25/root/usr/share/ruby/timeout.rb:108:in timeout’
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser_extras.rb:1075:in process' /opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser.rb:31:in block in process’
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser.rb:28:in each' /opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser.rb:28:in process’
/opt/theforeman/tfm/root/usr/share/gems/gems/safemode-1.3.5/lib/safemode/parser.rb:18:in parse' /opt/theforeman/tfm/root/usr/share/gems/gems/safemode-1.3.5/lib/safemode/parser.rb:9:in jail’
/opt/theforeman/tfm/root/usr/share/gems/gems/safemode-1.3.5/lib/safemode.rb:49:in eval' /usr/share/foreman/lib/foreman/renderer/safe_mode_renderer.rb:7:in render’
/usr/share/foreman/lib/foreman/renderer/base_renderer.rb:16:in render' /usr/share/foreman/lib/foreman/renderer.rb:46:in render’
/usr/share/foreman/app/models/template.rb:174:in `render’

wernerma · March 24, 2020, 11:00am

So here is one of the Update with errors :

Dynflow::ActiveJob::QueueAdapters::JobWrapper
Input:
{"job_class"=>"TemplateRenderJob",
 "queue"=>"default",
 "job_data"=>
  {"job_class"=>"TemplateRenderJob",
   "job_id"=>"ad4866f7-e76e-415b-8025-5a130b9cdaba",
   "provider_job_id"=>"ad4866f7-e76e-415b-8025-5a130b9cdaba",
   "queue_name"=>"default",
   "priority"=>nil,
   "arguments"=>
    [{"template_id"=>178,
      "gzip"=>false,
      "send_mail"=>false,
      "mail_to"=>nil,
      "format"=>"json",
      "input_values"=>
       {"68"=>{"value"=>"yes", "_aj_hash_with_indifferent_access"=>true},
        "69"=>{"value"=>"yes", "_aj_hash_with_indifferent_access"=>true},
        "70"=>{"value"=>"yes", "_aj_hash_with_indifferent_access"=>true},
        "71"=>{"value"=>"yes", "_aj_hash_with_indifferent_access"=>true},
        "72"=>{"value"=>"no", "_aj_hash_with_indifferent_access"=>true},
        "73"=>{"value"=>"yes", "_aj_hash_with_indifferent_access"=>true},
        "74"=>{"value"=>"no", "_aj_hash_with_indifferent_access"=>true},
        "75"=>{"value"=>"yes", "_aj_hash_with_indifferent_access"=>true},
        "76"=>{"value"=>"no", "_aj_hash_with_indifferent_access"=>true},
        "77"=>{"value"=>"yes", "_aj_hash_with_indifferent_access"=>true},
        "78"=>{"value"=>"no", "_aj_hash_with_indifferent_access"=>true},
        "79"=>{"value"=>"yes", "_aj_hash_with_indifferent_access"=>true},
        "_aj_hash_with_indifferent_access"=>true},
      "_aj_hash_with_indifferent_access"=>true},
     {"user_id"=>9, "_aj_symbol_keys"=>["user_id"]}],
   "executions"=>0,
   "locale"=>"en"},
 "current_request_id"=>"8d71c0c1-2004-4410-8b70-2a7acc92c5d8",
 "current_timezone"=>"UTC",
 "current_user_id"=>9,
 "current_organization_id"=>nil,
 "current_location_id"=>nil}
Output:
{}
Exception:
Timeout::Error: execution expired
Backtrace:
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby24_parser.rb:6708:in `initialize_dup'
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby24_parser.rb:6708:in `dup'
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby24_parser.rb:6708:in `_reduce_594'
/opt/rh/rh-ruby25/root/usr/share/ruby/racc/parser.rb:259:in `_racc_do_parse_c'
/opt/rh/rh-ruby25/root/usr/share/ruby/racc/parser.rb:259:in `do_parse'
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser_extras.rb:1087:in `block in process'
/opt/rh/rh-ruby25/root/usr/share/ruby/timeout.rb:108:in `timeout'
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser_extras.rb:1075:in `process'
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser.rb:31:in `block in process'
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser.rb:28:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/ruby_parser-3.10.1/lib/ruby_parser.rb:28:in `process'
/opt/theforeman/tfm/root/usr/share/gems/gems/safemode-1.3.5/lib/safemode/parser.rb:18:in `parse'
/opt/theforeman/tfm/root/usr/share/gems/gems/safemode-1.3.5/lib/safemode/parser.rb:9:in `jail'
/opt/theforeman/tfm/root/usr/share/gems/gems/safemode-1.3.5/lib/safemode.rb:49:in `eval'
/usr/share/foreman/lib/foreman/renderer/safe_mode_renderer.rb:7:in `render'
/usr/share/foreman/lib/foreman/renderer/base_renderer.rb:16:in `render'
/usr/share/foreman/lib/foreman/renderer.rb:46:in `render'
/usr/share/foreman/app/models/template.rb:174:in `render'
/usr/share/foreman/app/models/report_composer.rb:241:in `render'
/usr/share/foreman/app/jobs/template_render_job.rb:10:in `block in perform'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:108:in `as'
/usr/share/foreman/app/jobs/template_render_job.rb:7:in `perform'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/execution.rb:39:in `block in perform_now'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/i18n-1.4.0/lib/i18n.rb:284:in `with_locale'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `instance_exec'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/logging.rb:26:in `block (4 levels) in <module:Logging>'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `block in instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications.rb:168:in `instrument'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/logging.rb:25:in `block (3 levels) in <module:Logging>'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/logging.rb:48:in `tag_logger'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/logging.rb:22:in `block (2 levels) in <module:Logging>'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `instance_exec'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:136:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/execution.rb:35:in `perform_now'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/execution.rb:24:in `block in execute'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/railtie.rb:28:in `block (4 levels) in <class:Railtie>'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/reloader.rb:73:in `block in wrap'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/reloader.rb:72:in `wrap'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/railtie.rb:27:in `block (3 levels) in <class:Railtie>'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `instance_exec'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:136:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activejob-5.2.1/lib/active_job/execution.rb:22:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/active_job/queue_adapter.rb:47:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:563:in `block (3 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action/progress.rb:17:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.1/app/lib/actions/middleware/keep_locale.rb:11:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.1/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.14.1/app/lib/actions/middleware/keep_locale.rb:11:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_request_id.rb:49:in `restore_current_request_id'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_user.rb:44:in `restore_curent_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.17.5/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware.rb:32:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/middleware/world.rb:31:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:562:in `block (2 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:561:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:561:in `block in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:475:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:475:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:475:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:556:in `execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/action.rb:285:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/director.rb:68:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/executors/parallel/worker.rb:15:in `block in on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/executors.rb:12:in `run_user_code'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/executors/parallel/worker.rb:14:in `on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/context.rb:46:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/actor.rb:106:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/actor.rb:47:in `block in on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/actor.rb:59:in `with_backtrace'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.2/lib/dynflow/actor.rb:47:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/core.rb:162:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/core.rb:96:in `block in on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/core.rb:119:in `block (2 levels) in schedule_execution'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/synchronization/mutex_lockable_object.rb:41:in `block in synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/synchronization/mutex_lockable_object.rb:41:in `synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/synchronization/mutex_lockable_object.rb:41:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.4.1/lib-edge/concurrent/actor/core.rb:116:in `block in schedule_execution'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/serialized_execution.rb:18:in `call'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/serialized_execution.rb:96:in `work'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.1.4/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'