trenta
September 28, 2021, 2:42am
#1
Problem:
Puppet run on any client after Foreman 3 upgrade results in
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Failed when searching for node slurm-login01.svi.edu.au: Failed to find slurm-login01.svi.edu.au via exec: Execution of '/etc/puppetlabs/puppet/node.rb slurm-login01.svi.edu.au' returned 1:
Error logs from Forman production log
ActionController::RoutingError (No route matches [GET] "/node/<node-name>"):
Expected outcome:
Puppet runs to succeed
Foreman and Proxy versions:
3.0
Foreman and Proxy plugin versions:
3.0
Distribution and version:
Ubuntu 18
Puppet agent 7.11
Puppet server 7.4
I took a snapshot and rolling back was fine. Happy to do any testing suggested.
ekohl
September 28, 2021, 11:03am
#2
How did you upgrade? Did you not install the foreman_puppet plugin on upgrade?
trenta
September 29, 2021, 12:44am
#3
Thanks for the message. I installed it after the fact when I noticed things not working. Wrong order obviously. I’ve installed it now and puppet runs are working.
I encouter the same error with all my puppet agents after upgrading to 3.0.1, using 2 servers with foreman role only and 2 servers with puppetca/puppet proxy roles.
I added --enable-foreman-plugin-puppet to my foreman-installer parameters after rpms upgrade to upgrade the stack.
rubygem-foreman_puppet-1.0.4-1.fm3_0.el8.noarch rpm is installed.
It looks like my problem comes from generating host yaml data, puppet yaml button on each host gives me this : Unable to generate output, Check log files.
In the logs i can see this :
2021-11-04T13:42:28 [W|app|e5187b55] Failed to generate external nodes for xxxx.xxxx.fr
2021-11-04T13:42:28 [I|app|e5187b55] Backtrace for ‘Failed to generate external nodes for xxxx.xxxx.fr ’ error (TypeError): no implicit conversion of nil into Array
e5187b55 | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in +' e5187b55 | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in
+’
e5187b55 | /usr/share/gems/gems/foreman_puppet-1.0.4/app/services/foreman_puppet/host_info_providers/config_groups_info.rb:7:in host_info' e5187b55 | /usr/share/foreman/app/models/concerns/host_info_extensions.rb:27:in
block in info’
e5187b55 | /usr/share/foreman/app/models/concerns/host_info_extensions.rb:25:in each' e5187b55 | /usr/share/foreman/app/models/concerns/host_info_extensions.rb:25:in
info’
e5187b55 | /usr/share/gems/gems/foreman_puppet-1.0.4/app/controllers/concerns/foreman_puppet/extensions/hosts_controller_extensions.rb:100:in block in externalNodes' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/mime_responds.rb:205:in
respond_to’
e5187b55 | /usr/share/gems/gems/foreman_puppet-1.0.4/app/controllers/concerns/foreman_puppet/extensions/hosts_controller_extensions.rb:96:in externalNodes' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/basic_implicit_render.rb:6:in
send_action’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/base.rb:195:in process_action' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rendering.rb:30:in
process_action’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/callbacks.rb:42:in block in process_action' e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:112:in
block in run_callbacks’
e5187b55 | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in set_timezone' e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in
block in run_callbacks’
e5187b55 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in clear_thread' e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in
block in run_callbacks’
e5187b55 | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in set_topbar_sweeper_controller' e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in
block in run_callbacks’
e5187b55 | /usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in around' e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in
block in run_callbacks’
e5187b55 | /usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in around' e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in
block in run_callbacks’
e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:139:in run_callbacks' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/callbacks.rb:41:in
process_action’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rescue.rb:22:in process_action' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:33:in
block in process_action’
e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in block in instrument' e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications/instrumenter.rb:24:in
instrument’
e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in instrument' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:32:in
process_action’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/params_wrapper.rb:245:in process_action' e5187b55 | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/railties/controller_runtime.rb:27:in
process_action’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/base.rb:136:in process' e5187b55 | /usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/rendering.rb:39:in
process’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal.rb:190:in dispatch' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal.rb:254:in
dispatch’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:50:in dispatch' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:33:in
serve’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:49:in block in serve' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in
each’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in serve' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:834:in
call’
e5187b55 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:527:in call' e5187b55 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in
public_send’
e5187b55 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in method_missing' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:19:in
block in class:Constraints ’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/mapper.rb:48:in serve' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:49:in
block in serve’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in each' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in
serve’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:834:in call' e5187b55 | /usr/share/gems/gems/apipie-dsl-2.4.0/lib/apipie_dsl/static_dispatcher.rb:67:in
call’
e5187b55 | /usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in call' e5187b55 | /usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in call' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in call' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in call' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in
call’
e5187b55 | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in call' e5187b55 | /usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in
call’
e5187b55 | /usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in call' e5187b55 | /usr/share/gems/gems/prometheus-client-1.0.0/lib/prometheus/middleware/exporter.rb:32:in
call’
e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in call' e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/etag.rb:27:in
call’
e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/conditional_get.rb:27:in call' e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/head.rb:12:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/http/content_security_policy.rb:18:in call' e5187b55 | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in
call’
e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in context' e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/cookies.rb:648:in call' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/callbacks.rb:27:in
block in call’
e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:101:in run_callbacks' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/callbacks.rb:26:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in call' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/debug_exceptions.rb:32:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/show_exceptions.rb:33:in call' e5187b55 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/rack/logger.rb:37:in
call_app’
e5187b55 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/rack/logger.rb:28:in call' e5187b55 | /usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in
call’
e5187b55 | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in call' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/remote_ip.rb:81:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/request_id.rb:27:in call' e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/method_override.rb:24:in
call’
e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/runtime.rb:22:in call' e5187b55 | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in
call’
e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/executor.rb:14:in call' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in
call’
e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in call' e5187b55 | /usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/host_authorization.rb:76:in
call’
e5187b55 | /usr/share/gems/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in call' e5187b55 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:527:in
call’
e5187b55 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in public_send' e5187b55 | /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in
method_missing’
e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in block in call' e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in
each’
e5187b55 | /usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in call' e5187b55 | /usr/share/gems/gems/puma-5.3.2/lib/puma/configuration.rb:249:in
call’
e5187b55 | /usr/share/gems/gems/puma-5.3.2/lib/puma/request.rb:77:in block in handle_request' e5187b55 | /usr/share/gems/gems/puma-5.3.2/lib/puma/thread_pool.rb:338:in
with_force_shutdown’
e5187b55 | /usr/share/gems/gems/puma-5.3.2/lib/puma/request.rb:76:in handle_request' e5187b55 | /usr/share/gems/gems/puma-5.3.2/lib/puma/server.rb:438:in
process_client’
e5187b55 | /usr/share/gems/gems/puma-5.3.2/lib/puma/thread_pool.rb:145:in block in spawn_thread' e5187b55 | /usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in
block in create_with_logging_context’