foreman-nightly-rpm-pipeline 1124 failed

jenkins · July 19, 2021, 10:49pm

Foreman RPM nightly pipeline failed:

https://ci.theforeman.org/job/foreman-nightly-rpm-pipeline/1124/

foreman-pipeline-foreman-nightly-centos7-install (failed) (remote job)
foreman-pipeline-foreman-nightly-centos8-stream-install (failed) (remote job)
foreman-pipeline-foreman-nightly-centos8-install (failed) (remote job)
foreman-pipeline-foreman-nightly-centos7-upgrade (failed) (remote job)
foreman-pipeline-foreman-nightly-centos8-upgrade (failed) (remote job)

ekohl · July 20, 2021, 10:35am

It looks like Puppet doesn’t really work. It appears to assign the class (at least no error is returned) and Puppet runs, but then it doesn’t apply the class so /etc/motd isn’t updated.

Marek_Hulan · July 20, 2021, 11:43am

It seems the puppet server couldn’t get the ENC from Foreman

puppetserver.log:

2021-07-19T22:16:43.649Z WARN  [qtp2135865030-35] [c.p.p.ShellUtils] Executed an external process which logged to STDERR: Error retrieving node pipe-foreman-server-nightly-centos7.n62.example.com: Net::HTTPNotFound
Check Foreman's /var/log/foreman/production.log for more information.

2021-07-19T22:16:43.711Z ERROR [qtp2135865030-35] [puppetserver] Puppet Server Error: Failed to find pipe-foreman-server-nightly-centos7.n62.example.com via exec: Execution of '/etc/puppetlabs/puppet/node.rb pipe-foreman-server-nightly-centos7.n62.example.com' returned 1: 
uri:classloader:/puppetserver-lib/puppet/server/execution.rb:50:in `execute'
uri:classloader:/puppetserver-lib/puppet/server/execution.rb:14:in `block in initialize_execution_stub'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:222:in `execute'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/node/exec.rb:35:in `execute'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/exec.rb:19:in `find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/node/exec.rb:17:in `find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:223:in `find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:138:in `do_find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:54:in `block in call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:314:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:53:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `block in process'
org/jruby/RubyArray.java:1809:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:87:in `block in process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:70:in `block in with_request_profiling'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:66:in `with_request_profiling'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:86:in `block in process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:93:in `respond_to_errors'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:85:in `process'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:65:in `block in handleRequest'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:314:in `override'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:64:in `handleRequest'
2021-07-19T22:16:54.328Z WARN  [qtp2135865030-35] [puppetserver] Puppet The node parameter 'fqdn' for node 'pipe-foreman-server-nightly-centos7.n62.example.com' was already set to 'pipe-foreman-server-nightly-centos7.n62.example.com'. It could not be set to 'pipe-foreman-server-nightly-centos7.n62.example.com'
2021-07-19T22:16:54.329Z WARN  [qtp2135865030-35] [puppetserver] Puppet The node parameter 'hostname' for node 'pipe-foreman-server-nightly-centos7.n62.example.com' was already set to 'pipe-foreman-server-nightly-centos7'. It could not be set to 'pipe-foreman-server-nightly-centos7'

I don’t see the foreman.log in the sosreport, are we bundling it too somewhere? Is there a way to access that if it’s not in sosreport?

Marek_Hulan · July 20, 2021, 11:49am

ignore me, I see the Foreman production.log now, investigating

ekohl · July 20, 2021, 12:10pm

From production.log:

2021-07-19T22:09:11 [W|app|] ForemanPuppet: skipping engine hook (PG::UndefinedTable: ERROR:  relation "hosts" does not exist
 | LINE 8:  WHERE a.attrelid = '"hosts"'::regclass
 |                             ^
 | )
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:65:in `async_exec'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:65:in `block (2 levels) in query'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:64:in `block in query'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:722:in `block (2 levels) in log'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:721:in `block in log'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:712:in `log'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:63:in `query'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:820:in `column_definitions'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:114:in `columns'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/schema_cache.rb:76:in `columns'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/schema_cache.rb:82:in `columns_hash'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/model_schema.rb:488:in `load_schema!'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/attributes.rb:247:in `load_schema!'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/attribute_decorators.rb:50:in `load_schema!'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/model_schema.rb:478:in `block in load_schema'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/model_schema.rb:475:in `synchronize'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/model_schema.rb:475:in `load_schema'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/model_schema.rb:352:in `columns_hash'
 | /usr/share/foreman/app/services/graphql_attribute.rb:15:in `required?'
 | /usr/share/foreman/app/graphql/types/base_object.rb:54:in `nullable?'
 | /usr/share/foreman/app/graphql/types/base_object.rb:9:in `field'
 | /usr/share/foreman/app/graphql/types/base_object.rb:14:in `timestamps'
 | /usr/share/foreman/app/graphql/types/host.rb:8:in `<class:Host>'
 | /usr/share/foreman/app/graphql/types/host.rb:2:in `<module:Types>'
 | /usr/share/foreman/app/graphql/types/host.rb:1:in `<top (required)>'
 | /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
 | /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in `block in require'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:291:in `load_dependency'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in `require'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:411:in `block in require_or_load'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:40:in `block in load_interlock'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:14:in `block in loading'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/share_lock.rb:151:in `exclusive'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:13:in `loading'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:40:in `load_interlock'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:389:in `require_or_load'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:544:in `load_missing_constant'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:214:in `const_missing'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_puppet-1.0.0.rc.1/lib/foreman_puppet/engine.rb:96:in `block in <class:Engine>'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:428:in `instance_exec'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:428:in `block in make_lambda'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:200:in `block (2 levels) in halting'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:605:in `block (2 levels) in default_terminator'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:604:in `catch'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:604:in `block in default_terminator'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:201:in `block in halting'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:513:in `block in invoke_before'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:513:in `each'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:513:in `invoke_before'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:134:in `run_callbacks'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/reloader.rb:88:in `prepare!'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application/finisher.rb:113:in `block in <module:Finisher>'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:32:in `instance_exec'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:32:in `run'
 | /usr/share/foreman/config/initializers/0_print_time_spent.rb:45:in `block in run'
 | /usr/share/foreman/config/initializers/0_print_time_spent.rb:17:in `benchmark'
 | /usr/share/foreman/config/initializers/0_print_time_spent.rb:45:in `run'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:61:in `block in run_initializers'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:228:in `block in tsort_each'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:431:in `each_strongly_connected_component_from'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:349:in `block in each_strongly_connected_component'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in `each'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in `call'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in `each_strongly_connected_component'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:226:in `tsort_each'
 | /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:205:in `tsort_each'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:60:in `run_initializers'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application.rb:363:in `initialize!'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `public_send'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `method_missing'
 | /usr/share/foreman/config/environment.rb:5:in `<top (required)>'
 | /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
 | /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in `block in require'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:291:in `load_dependency'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in `require'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application.rb:339:in `require_environment!'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application.rb:523:in `block in run_tasks_blocks'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `block in execute'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `each'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:281:in `execute'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:199:in `synchronize'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:243:in `block in invoke_prerequisites'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:241:in `each'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:241:in `invoke_prerequisites'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:218:in `block in invoke_with_call_chain'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:199:in `synchronize'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:243:in `block in invoke_prerequisites'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:241:in `each'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:241:in `invoke_prerequisites'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:218:in `block in invoke_with_call_chain'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:199:in `synchronize'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/task.rb:188:in `invoke'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:160:in `invoke_task'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:116:in `each'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:116:in `block in top_level'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:125:in `run_with_threads'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:110:in `top_level'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:83:in `block in run'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:186:in `standard_exception_handling'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/lib/rake/application.rb:80:in `run'
 | /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
 | /opt/rh/rh-ruby27/root/usr/bin/rake:23:in `load'
 | /opt/rh/rh-ruby27/root/usr/bin/rake:23:in `<main>'

I wouldn’t expect to see that. It looks like it needs DB access somewhere. I’m not sure which rake task that is.

Is it triggering on this line? 96 is empty

github.com

theforeman/foreman_puppet/blob/47ab184f1e5aec687903c709f25443bf18909dc3/lib/foreman_puppet/engine.rb#L95

    
      
          ::Api::V2::HostgroupsController.include ForemanPuppet::Extensions::ApiHostgroupsController
          ::Api::V2::TemplateCombinationsController.include ForemanPuppet::Extensions::ApiTemplateCombinationsController
          ::Api::V2::HostsController.include ForemanPuppet::Extensions::ParametersHost
          ::Api::V2::HostgroupsController.include ForemanPuppet::Extensions::ParametersHostgroup
          ::OperatingsystemsController.prepend ForemanPuppet::Extensions::OperatingsystemsController
          ::HostsController.include ForemanPuppet::Extensions::HostsControllerExtensions
          ::HostsController.include ForemanPuppet::Extensions::ParametersHost
          ::HostgroupsController.include ForemanPuppet::Extensions::HostgroupsControllerExtensions
          ::HostgroupsController.include ForemanPuppet::Extensions::ParametersHostgroup
          
          
::SmartProxiesHelper::TABBED_FEATURES << 'Puppet'
          
          
unless ForemanPuppet.extracted_from_core?
            ::HostInfo.local_entries.delete('HostInfoProviders::PuppetInfo'.safe_constantize)
            ::HostInfo.local_entries.delete('HostInfoProviders::ConfigGroupsInfo'.safe_constantize)
            ::ProxyStatus.status_registry.delete('ProxyStatus::Puppet'.safe_constantize)
            Foreman.input_types_registry.input_types.delete('puppet_parameter')
          end
          Foreman.input_types_registry.register(ForemanPuppet::InputType::PuppetParameterInput)
          ::ProxyStatus.status_registry.add(ForemanPuppet::ProxyStatus::Puppet)

Another thing:

2021-07-19T22:19:14 [W|api|b3e3e842] /api/v2/environments API endpoints are deprecated, please use /foreman_puppet/api/v2/environments instead

That’s probably from Hammer.

evgeni · July 22, 2021, 9:16am

So I spun up a box here, and could reproduce it.

The problem seems to be that while our tests execute hammer host update --puppet-class-ids $id --name $(hostname -f), the values are correct and hammer answers Host updated., the puppet class is not actually assigned.

Assigning the class in the WebUI or via the API (curl -XPUT -d'{"host":{"puppetclass_ids":[1]}}' -H 'Content-Type: application/json' -u admin:changeme -k https://localhost/api/hosts/1) seems to work fine, and a Puppet run afterwards does apply the motd module.

And indeed, if you run hammer --debug with the above parameters, the resulting update call is actually empty:

[ INFO 2021-07-22T08:41:35 HammerCLIForeman::Host::UpdateCommand] Called with options: {"option_puppetclass_ids"=>[1], "option_volume_list"=>[], "option_interface_list"=>[], "option_name"=>"pipe-foreman-server-nightly-centos7.yatsu.example.com", "option_id"=>1}
[DEBUG 2021-07-22T08:41:35 HammerCLIForeman::CommandExtensions::UpdateCommon] Called block for HammerCLIForeman::UpdateCommand request params:
        #<Proc:0x00000000027d84b0 /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-2.6.0.pre.develop/lib/hammer_cli_foreman/command_extensions/update_common.rb:8>
[ INFO 2021-07-22T08:41:35 API] Server: https://pipe-foreman-server-nightly-centos7.yatsu.example.com
[ INFO 2021-07-22T08:41:35 API] PUT /api/hosts/1
[DEBUG 2021-07-22T08:41:35 API] Params: {
    "host" => {
        "name" => "pipe-foreman-server-nightly-centos7.yatsu.example.com"
    }
}

I don’t know hammer well enough to understand why it stopped translating this, but maybe someone wiser than me can chime in here.

FWIW, I did find GitHub - theforeman/hammer-cli-foreman-puppet but didn’t manage to install it on a nightly box, as the commands clash with the ones from hammer_cli_foreman and I really have no clue how to make them not clash.

evgeni · July 22, 2021, 12:11pm

FWIW2:

on a working 2.5 install, the same hammer call generates:

[ INFO 2021-07-22T12:09:04 HammerCLIForeman::Host::UpdateCommand] Called with options: {"option_puppetclass_ids"=>[1], "option_volume_list"=>[], "option_interface_list"=>[], "option_name"=>"centos7-foreman-2-5.yatsu.example.com", "option_id"=>1}
[DEBUG 2021-07-22T12:09:04 HammerCLIForeman::CommandExtensions::UpdateCommon] Called block for HammerCLIForeman::UpdateCommand request params:
	#<Proc:0x000000000281b6c0 /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-2.5.1/lib/hammer_cli_foreman/command_extensions/update_common.rb:8>
[ INFO 2021-07-22T12:09:04 API] Server: https://centos7-foreman-2-5.yatsu.example.com
[ INFO 2021-07-22T12:09:04 API] PUT /api/hosts/1
[DEBUG 2021-07-22T12:09:04 API] Params: {
    "host" => {
                   "name" => "centos7-foreman-2-5.yatsu.example.com",
        "puppetclass_ids" => [
            [0] 1
        ]
    }
}

evgeni · July 22, 2021, 1:09pm

The winner gets 3.0 points, and ows both @Marek_Hulan and me 3.0 pints of beer each.

foreman_puppet has a nice check to only be overriding things when it’s actually needed: ForemanPuppet.extracted_from_core?:

github.com

theforeman/foreman_puppet/blob/eab57b72349ed148f2c785884d4ad8c205dd758a/lib/foreman_puppet.rb#L2-L7

    
      
          FOREMAN_EXTRACTION_VERSION = '3.0'.freeze
          
          
def self.extracted_from_core?
            ENV['PUPPET_EXTRACTED'].to_s == '1' ||
              Gem::Dependency.new('', ">= #{FOREMAN_EXTRACTION_VERSION}").match?('', SETTINGS[:version].notag)
          end

As you can see there, the checked version is “3.0”, but our develop branch is still versioned 2.6. That means that this method returns false. And then the API override does not get applied in

github.com

theforeman/foreman_puppet/blob/eab57b72349ed148f2c785884d4ad8c205dd758a/app/controllers/concerns/foreman_puppet/extensions/api_v2_hosts_controller.rb#L9-L31

    
      
          if ForemanPuppet.extracted_from_core?
            method_desc = Apipie.get_method_description(self, :index)
            method_desc.apis << Apipie::MethodDescription::Api.new(:GET, '/environments/:environment_id/hosts', N_('List hosts per environment'), {})
          
          
  apipie_update_methods([:index]) do
              param :environment_id, String, desc: N_('ID of puppet environment')
            end
          
          
  apipie_update_methods(%i[create update]) do
              param :host, Hash do
                param :environment_id, String, desc: N_('Deprecated in favor of host/puppet_attributes/environment_id')
                param :puppetclass_ids, Array, desc: N_('Deprecated in favor of host/puppet_attributes/puppetclass_ids')
                param :config_group_ids, Array, desc: N_('Deprecated in favor of host/puppet_attributes/config_group_ids')
          
          
      param :puppet_attributes, Hash do
                  param :environment_id, String, desc: N_('ID of associated puppet Environment')
                  param :puppetclass_ids, Array, desc: N_('IDs of associated Puppetclasses')
                  param :config_group_ids, Array, desc: N_('IDs of associated ConfigGroups')
                end
              end

This file has been truncated. show original

Which means the apidoc is missing the puppetclasses_ids parameter when creating/updating hosts (and hostgroups). And well, if something is not in the apidoc, hammer ignores it. And so would FAM.

So we either need to revert the extraction, or bump to 3.0 with red nightlies and hope it works. (Given it’s the only failing test, I do expect it to become green then)

ekohl · July 22, 2021, 1:18pm

Let’s bump to 3.0:

ehelms · July 22, 2021, 1:21pm

Bump!

evgeni · July 23, 2021, 5:36am

It’s been bumped and foreman-nightly-rpm-pipeline #1127 [Jenkins] was green – besides the fact that web01 ran out of diskspace (I’m fixing this now)

evgeni · July 23, 2021, 6:33am

And foreman-nightly-rpm-pipeline #1128 [Jenkins] was fully green now.

Debian still needs building, and Katello needs an adjustment of its deps (Update Katello requirement on Foreman by evgeni · Pull Request #6934 · theforeman/foreman-packaging · GitHub), but then we’re good

system · July 30, 2021, 6:33am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.