Foreman upgrade 3.4 from 3.3

SarathHewage · September 9, 2022, 8:07am

Problem:
The 12th step of foreman upgrade 3.4, ‘foreman-installer’ gives unusual errors. This error or slimier didn’t happen when doing 3.3 upgrade from 3.2
errors are:

2022-09-09 14:21:22 [ERROR ] [configure] /Stage[main]/Foreman::Register/Foreman_host[foreman-sh-flag21.bbs.bunnings.com.au]: Could not evaluate: Error making GET request to Foreman at https://sh-flag21.bbs.bunnings.com.au/api/v2/hosts: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on sh-flag21.bbs.bunnings.com.au for detailed information
3. 2022-09-09 14:21:46 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_host[foreman-proxy-sh-flag21.bbs.bunnings.com.au]: Could not evaluate: Error making GET request to Foreman at https://sh-flag21.bbs.bunnings.com.au/api/v2/hosts: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on sh-flag21.bbs.bunnings.com.au for detailed information
4. 2022-09-09 14:21:47 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[sh-flag21.bbs.bunnings.com.au]: Could not evaluate: Error making GET request to Foreman at https://sh-flag21.bbs.bunnings.com.au/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on sh-flag21.bbs.bunnings.com.au for detailed information
5. 2022-09-09 14:21:47 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[sh-flag21.bbs.bunnings.com.au]: Failed to call refresh: Error making GET request to Foreman at https://sh-flag21.bbs.bunnings.com.au/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on sh-flag21.bbs.bunnings.com.au for detailed information
6. 2022-09-09 14:21:47 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[sh-flag21.bbs.bunnings.com.au]: Error making GET request to Foreman at https://sh-flag21.bbs.bunnings.com.au/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on sh-flag21.bbs.bunnings.com.au for detailed information

Expected outcome:
As 3.3 upgrade in same environment no errors is expected

Foreman and Proxy versions:
3.3 Foreman
3.3 Proxy

Foreman and Proxy plugin versions:

Distribution and version:

Other relevant data:

evgeni · September 9, 2022, 8:33am

Can you show us the contents of /var/log/foreman/production.log around the same timestamps?

SarathHewage · September 12, 2022, 1:45am

The log file is archived now
This is the records with time stamp 2022-09-09T14:21-

[root@sh-flag21 ~]# grep 2022-09-09T14:21 production.log-20220910
2022-09-09T14:21:03 [E|dyn|] Pruned 1 envelopes for invalidated world 363106c7-1d43-4291-ad1b-fd277422f59b
2022-09-09T14:21:03 [E|dyn|] invalid worlds found {"87b48dd3-d3cd-493f-a9d4-b8a28bbfe53c"=>:valid, "363106c7-1d43-4291-ad1b-fd277422f59b"=>:invalidated, "eae99e00-b482-464a-9a87-16600fb30685"=>:valid, "50009162-3fa6-4854-abfb-432eaf931b94"=>:valid, "f4a21981-4f99-48cb-93d2-391c808ab3ca"=>:valid, "6a2258ff-7691-413e-ac84-1d1ecb26546f"=>:valid, "b26f038f-d9e5-4673-a210-4c6b7c006022"=>:valid, "20086ea4-efb5-4c78-b387-e981b1ac4857"=>:valid}
2022-09-09T14:21:03 [I|app|] Everything ready for world: 87b48dd3-d3cd-493f-a9d4-b8a28bbfe53c
2022-09-09T14:21:10 [I|app|] Rails cache backend: File
2022-09-09T14:21:10 [I|app|] memcached cache backend enabled: [:dalli_store, "sh-mane21.bbs.bunnings.com.au", {:namespace=>"foreman", :expires_in=>86400, :compress=>true}]
2022-09-09T14:21:10 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:11 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2022-09-09T14:21:11 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2022-09-09T14:21:11 [W|app|] Creating scope :exportable. Overwriting existing method Katello::Repository.exportable.
2022-09-09T14:21:13 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2022-09-09T14:21:13 [W|app|] You are trying to replace view_hosts from foreman_ansible. Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:13 [W|app|] You are trying to replace view_hostgroups from foreman_ansible. Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:13 [W|app|] You are trying to replace edit_hosts from foreman_ansible. Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:13 [W|app|] You are trying to replace edit_hostgroups from foreman_ansible. Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:13 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2022-09-09T14:21:13 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2022-09-09T14:21:18 [W|app|] Scoped order is ignored, it's forced to be batch order.
2022-09-09T14:21:19 [W|app|] Scoped order is ignored, it's forced to be batch order.
2022-09-09T14:21:19 [W|app|] Scoped order is ignored, it's forced to be batch order.
2022-09-09T14:21:19 [I|app|] No new seed file updates found. Skipping
2022-09-09T14:21:21 [I|app|] Everything ready for world: b39760f2-d480-482b-a619-dfcae2b77def
2022-09-09T14:21:21 [I|dyn|] Performing validity checks
2022-09-09T14:21:21 [I|app|7bf5374f] Started GET "/api/v2/hosts?search=name%3D%22sh-flag21.bbs.bunnings.com.au%22" for 172.23.11.158 at 2022-09-09 14:21:21 +0800
2022-09-09T14:21:21 [I|app|7bf5374f] Katello event daemon started process=20360
2022-09-09T14:21:21 [I|app|7bf5374f] Fork detected, re-connecting child process...
2022-09-09T14:21:21 [I|dyn|] Finished performing validity checks
2022-09-09T14:21:21 [I|app|7bf5374f] Subscribed to katello.candlepin.candlepin_events
2022-09-09T14:21:21 [I|kat|7bf5374f] Polling Katello Event Queue
2022-09-09T14:21:22 [I|app|7bf5374f] Processing by Api::V2::HostsController#index as JSON
2022-09-09T14:21:22 [I|app|7bf5374f]   Parameters: {"search"=>"name=\"sh-flag21.bbs.bunnings.com.au\"", "apiv"=>"v2", "host"=>{}}
2022-09-09T14:21:22 [W|app|7bf5374f] Action failed
2022-09-09T14:21:22 [I|app|7bf5374f] Backtrace for 'Action failed' error (NoMethodError): undefined method `dup_value!' for 0:Integer
2022-09-09T14:21:22 [I|app|7bf5374f]   Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 1.1ms | Allocations: 263)
2022-09-09T14:21:22 [I|app|7bf5374f]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 2.1ms | Allocations: 499)
2022-09-09T14:21:22 [I|app|7bf5374f] Completed 500 Internal Server Error in 34ms (Views: 7.3ms | ActiveRecord: 1.4ms | Allocations: 22940)
2022-09-09T14:21:22 [F|app|7bf5374f]
2022-09-09T14:21:28 [I|app|] Rails cache backend: File
2022-09-09T14:21:28 [I|app|] memcached cache backend enabled: [:dalli_store, "sh-mane21.bbs.bunnings.com.au", {:namespace=>"foreman", :expires_in=>86400, :compress=>true}]
2022-09-09T14:21:28 [W|app|] You are trying to replace import_subnets from . Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:29 [W|app|] Setting remote_execution_workers_pool_size has no definition, please define it before using
2022-09-09T14:21:29 [W|app|] Setting host_tasks_workers_pool_size has no definition, please define it before using
2022-09-09T14:21:29 [W|app|] Creating scope :exportable. Overwriting existing method Katello::Repository.exportable.
2022-09-09T14:21:31 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2022-09-09T14:21:31 [W|app|] You are trying to replace view_hosts from foreman_ansible. Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:31 [W|app|] You are trying to replace view_hostgroups from foreman_ansible. Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:31 [W|app|] You are trying to replace edit_hosts from foreman_ansible. Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:31 [W|app|] You are trying to replace edit_hostgroups from foreman_ansible. Adding allowed actions from plugin permissions to the existing one.
2022-09-09T14:21:32 [W|app|] Creating scope :execution_scope. Overwriting existing method Host::Managed.execution_scope.
2022-09-09T14:21:32 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2022-09-09T14:21:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2022-09-09T14:21:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2022-09-09T14:21:37 [W|app|] Scoped order is ignored, it's forced to be batch order.
2022-09-09T14:21:38 [I|app|] No new seed file updates found. Skipping
2022-09-09T14:21:39 [I|app|] Everything ready for world: c99faf97-1112-464e-9773-2f56a0ce173f
2022-09-09T14:21:46 [I|app|83cbfc9f] Started GET "/api/v2/hosts?search=name%3D%22sh-flag21.bbs.bunnings.com.au%22" for 172.23.11.158 at 2022-09-09 14:21:46 +0800
2022-09-09T14:21:46 [I|app|83cbfc9f] Processing by Api::V2::HostsController#index as JSON
2022-09-09T14:21:46 [I|app|83cbfc9f]   Parameters: {"search"=>"name=\"sh-flag21.bbs.bunnings.com.au\"", "apiv"=>"v2", "host"=>{}}
2022-09-09T14:21:46 [W|app|83cbfc9f] Action failed
2022-09-09T14:21:46 [I|app|83cbfc9f] Backtrace for 'Action failed' error (NoMethodError): undefined method `dup_value!' for 0:Integer
2022-09-09T14:21:46 [I|app|83cbfc9f]   Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 12.0ms | Allocations: 5366)
2022-09-09T14:21:46 [I|app|83cbfc9f]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 17.6ms | Allocations: 10714)
2022-09-09T14:21:46 [I|app|83cbfc9f] Completed 500 Internal Server Error in 52ms (Views: 18.2ms | ActiveRecord: 11.2ms | Allocations: 12667)
2022-09-09T14:21:46 [F|app|83cbfc9f]
2022-09-09T14:21:47 [I|app|633bf22a] Started GET "/api/v2/smart_proxies?search=name%3D%22sh-flag21.bbs.bunnings.com.au%22" for 172.23.11.158 at 2022-09-09 14:21:47 +0800
2022-09-09T14:21:47 [I|app|633bf22a] Processing by Api::V2::SmartProxiesController#index as JSON
2022-09-09T14:21:47 [I|app|633bf22a]   Parameters: {"search"=>"name=\"sh-flag21.bbs.bunnings.com.au\"", "apiv"=>"v2", "smart_proxy"=>{}}
2022-09-09T14:21:47 [I|app|633bf22a] Fork detected, re-connecting child process...
2022-09-09T14:21:47 [W|app|633bf22a] Action failed
2022-09-09T14:21:47 [I|app|633bf22a] Backtrace for 'Action failed' error (NoMethodError): undefined method `dup_value!' for 0:Integer
2022-09-09T14:21:47 [I|app|633bf22a]   Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 1.2ms | Allocations: 264)
2022-09-09T14:21:47 [I|app|633bf22a]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 2.1ms | Allocations: 500)
2022-09-09T14:21:47 [I|app|633bf22a] Completed 500 Internal Server Error in 97ms (Views: 7.1ms | ActiveRecord: 21.5ms | Allocations: 22950)
2022-09-09T14:21:47 [F|app|633bf22a]
2022-09-09T14:21:47 [I|app|facd3841] Started GET "/api/v2/smart_proxies?search=name%3D%22sh-flag21.bbs.bunnings.com.au%22" for 172.23.11.158 at 2022-09-09 14:21:47 +0800
2022-09-09T14:21:47 [I|app|facd3841] Processing by Api::V2::SmartProxiesController#index as JSON
2022-09-09T14:21:47 [I|app|facd3841]   Parameters: {"search"=>"name=\"sh-flag21.bbs.bunnings.com.au\"", "apiv"=>"v2", "smart_proxy"=>{}}
2022-09-09T14:21:47 [W|app|facd3841] Action failed
2022-09-09T14:21:47 [I|app|facd3841] Backtrace for 'Action failed' error (NoMethodError): undefined method `dup_value!' for 0:Integer
2022-09-09T14:21:47 [I|app|facd3841]   Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 10.9ms | Allocations: 5366)
2022-09-09T14:21:47 [I|app|facd3841]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 16.5ms | Allocations: 10713)
2022-09-09T14:21:47 [I|app|facd3841] Completed 500 Internal Server Error in 54ms (Views: 17.0ms | ActiveRecord: 10.6ms | Allocations: 12607)
2022-09-09T14:21:47 [F|app|facd3841]

evgeni · September 12, 2022, 6:41am

I think this is foreman_memcache/dalli not properly working with Rails 6.1.

Can you disable memcache in your settings, restart Foreman and see if that changes anything?

SarathHewage · September 13, 2022, 3:32am

Thanks for the reply
I did disable the memcache in my settings like this
sudo yum remove rubygem-foreman_memcache
sudo foreman-maintain service stop --exclude postgresql
sudo foreman-maintain service start --exclude postgresql
Then run the sudo foreman-installer
Still get the same issue and also can see these two lines in /var/log/foreman/production.log
2022-09-13T11:23:24 [I|app|5e5000cd] Backtrace for ‘Action failed’ error (NoMethodError): undefined
method dup_value!' for 0:Integer 5e5000cd | NoMethodError (undefined method dup_value!’ for #String:0x0000559d9b6c2080):

SarathHewage · September 13, 2022, 7:01am

Tried again and as fresh upgrade from 3.3 to 3.4 and sill not success.
These are log entries in `/var/log/foreman/production.log.

 4eb36d9f | /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2022-09-13T14:58:25 [I|app|4eb36d9f]   Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 11.2ms | Allocations: 5370)
2022-09-13T14:58:25 [I|app|4eb36d9f]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 16.7ms | Allocations: 10721)
2022-09-13T14:58:25 [I|app|4eb36d9f] Completed 500 Internal Server Error in 50ms (Views: 17.9ms | ActiveRecord: 10.6ms | Allocations: 13032)
2022-09-13T14:58:25 [F|app|4eb36d9f]
 4eb36d9f | NoMethodError (undefined method `dup_value!' for #<String:0x000055fa6d9df798>):
 4eb36d9f |
 4eb36d9f | config/initializers/apipie.rb:64:in `block (3 levels) in <top (required)>'
 4eb36d9f | config/initializers/apipie.rb:73:in `block (4 levels) in <top (required)>'
 4eb36d9f | config/initializers/apipie.rb:73:in `each'
 4eb36d9f | config/initializers/apipie.rb:73:in `map'
 4eb36d9f | config/initializers/apipie.rb:73:in `block (3 levels) in <top (required)>'
 4eb36d9f | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 4eb36d9f | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 4eb36d9f | katello (4.6.0.rc2) lib/katello/prevent_json_parsing.rb:12:in `call'
2022-09-13T14:58:25 [I|app|6d13b90b] Started GET "/api/v2/smart_proxies?search=name%3D%22sh-flag21.bbs.bunnings.com.au%22" for 172.23.11.158 at 2022-09-13 14:58:25 +0800
2022-09-13T14:58:25 [I|app|6d13b90b] Processing by Api::V2::SmartProxiesController#index as JSON
2022-09-13T14:58:25 [I|app|6d13b90b]   Parameters: {"search"=>"name=\"sh-flag21.bbs.bunnings.com.au\"", "apiv"=>"v2", "smart_proxy"=>{}}
2022-09-13T14:58:25 [W|app|6d13b90b] Action failed
2022-09-13T14:58:25 [I|app|6d13b90b] Backtrace for 'Action failed' error (NoMethodError): undefined method `dup_value!' for 0:Integer
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/cache/strategy/local_cache.rb:68:in `write_entry'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/cache/strategy/local_cache.rb:162:in `write_entry'
 6d13b90b | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:144:in `block (2 levels) in write'
 6d13b90b | /usr/share/gems/gems/dalli-2.7.6/lib/dalli/client.rb:262:in `with'
 6d13b90b | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:82:in `with'
 6d13b90b | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:142:in `block in write'
 6d13b90b | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:371:in `block in instrument'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/notifications.rb:205:in `instrument'
 6d13b90b | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:371:in `instrument'
 6d13b90b | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:141:in `write'
 6d13b90b | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:115:in `fetch'
 6d13b90b | /usr/share/foreman/app/services/foreman/bruteforce_protection.rb:10:in `get_login_failures'
 6d13b90b | /usr/share/foreman/app/services/foreman/bruteforce_protection.rb:18:in `bruteforce_attempt?'
 6d13b90b | /usr/share/foreman/app/controllers/concerns/foreman/controller/bruteforce_protection.rb:9:in `bruteforce_attempt?'
 6d13b90b | /usr/share/foreman/app/controllers/api/base_controller.rb:194:in `authorize'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:427:in `block in make_lambda'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:198:in `block (2 levels) in halting'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:199:in `block in halting'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:512:in `block in invoke_before'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:512:in `each'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:512:in `invoke_before'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:115:in `block in run_callbacks'
 6d13b90b | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 6d13b90b | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 6d13b90b | /usr/share/gems/gems/audited-4.10.0/lib/audited/sweeper.rb:14:in `around'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 6d13b90b | /usr/share/gems/gems/audited-4.10.0/lib/audited/sweeper.rb:14:in `around'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:137:in `run_callbacks'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/notifications.rb:203:in `block in instrument'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/notifications.rb:203:in `instrument'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal/instrumentation.rb:33:in `process_action'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal/params_wrapper.rb:249:in `process_action'
 6d13b90b | /usr/share/gems/gems/activerecord-6.1.6.1/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/abstract_controller/base.rb:165:in `process'
 6d13b90b | /usr/share/gems/gems/actionview-6.1.6.1/lib/action_view/rendering.rb:39:in `process'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal.rb:190:in `dispatch'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal.rb:254:in `dispatch'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/route_set.rb:33:in `serve'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/mapper.rb:19:in `block in <class:Constraints>'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/mapper.rb:49:in `serve'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/journey/router.rb:50:in `block in serve'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/journey/router.rb:32:in `each'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/journey/router.rb:32:in `serve'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/route_set.rb:842:in `call'
 6d13b90b | /usr/share/gems/gems/katello-4.6.0.rc2/lib/katello/middleware/organization_created_enforcer.rb:18:in `call'
 6d13b90b | /usr/share/gems/gems/katello-4.6.0.rc2/lib/katello/middleware/event_daemon.rb:10:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in `call'
 6d13b90b | /usr/share/gems/gems/apipie-dsl-2.5.0/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
 6d13b90b | /usr/share/gems/gems/apipie-rails-0.5.20/lib/apipie/static_dispatcher.rb:66:in `call'
 6d13b90b | /usr/share/gems/gems/apipie-rails-0.5.20/lib/apipie/extractor/recorder.rb:137:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in `call'
 6d13b90b | /usr/share/foreman/lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in `call'
 6d13b90b | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
 6d13b90b | /usr/share/gems/gems/apipie-rails-0.5.20/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/tempfile_reaper.rb:15:in `call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/etag.rb:27:in `call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/conditional_get.rb:27:in `call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/head.rb:12:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/http/permissions_policy.rb:22:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/http/content_security_policy.rb:19:in `call'
 6d13b90b | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/session/abstract/id.rb:266:in `context'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/session/abstract/id.rb:260:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/cookies.rb:689:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
 6d13b90b | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/rack/logger.rb:37:in `call_app'
 6d13b90b | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/rack/logger.rb:28:in `call'
 6d13b90b | /usr/share/gems/gems/sprockets-rails-3.4.2/lib/sprockets/rails/quiet_assets.rb:13:in `call'
 6d13b90b | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/request_id.rb:26:in `call'
 6d13b90b | /usr/share/gems/gems/katello-4.6.0.rc2/lib/katello/prevent_json_parsing.rb:12:in `call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/method_override.rb:24:in `call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/runtime.rb:22:in `call'
 6d13b90b | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in `call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/sendfile.rb:110:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/ssl.rb:77:in `call'
 6d13b90b | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/host_authorization.rb:142:in `call'
 6d13b90b | /usr/share/gems/gems/secure_headers-6.3.4/lib/secure_headers/middleware.rb:11:in `call'
 6d13b90b | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/engine.rb:539:in `call'
 6d13b90b | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/railtie.rb:207:in `public_send'
 6d13b90b | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/railtie.rb:207:in `method_missing'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/urlmap.rb:74:in `block in call'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/urlmap.rb:58:in `each'
 6d13b90b | /usr/share/gems/gems/rack-2.2.4/lib/rack/urlmap.rb:58:in `call'
 6d13b90b | /usr/share/gems/gems/puma-5.6.4/lib/puma/configuration.rb:252:in `call'
 6d13b90b | /usr/share/gems/gems/puma-5.6.4/lib/puma/request.rb:77:in `block in handle_request'
 6d13b90b | /usr/share/gems/gems/puma-5.6.4/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
 6d13b90b | /usr/share/gems/gems/puma-5.6.4/lib/puma/request.rb:76:in `handle_request'
 6d13b90b | /usr/share/gems/gems/puma-5.6.4/lib/puma/server.rb:441:in `process_client'
 6d13b90b | /usr/share/gems/gems/puma-5.6.4/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
 6d13b90b | /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2022-09-13T14:58:25 [I|app|6d13b90b]   Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 5.2ms | Allocations: 5370)
2022-09-13T14:58:25 [I|app|6d13b90b]   Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 10.1ms | Allocations: 10721)
2022-09-13T14:58:25 [I|app|6d13b90b] Completed 500 Internal Server Error in 26ms (Views: 10.6ms | ActiveRecord: 5.6ms | Allocations: 12627)
2022-09-13T14:58:25 [F|app|6d13b90b]
 6d13b90b | NoMethodError (undefined method `dup_value!' for #<String:0x000055fa6ceec228>):
 6d13b90b |
 6d13b90b | config/initializers/apipie.rb:64:in `block (3 levels) in <top (required)>'
 6d13b90b | config/initializers/apipie.rb:73:in `block (4 levels) in <top (required)>'
 6d13b90b | config/initializers/apipie.rb:73:in `each'
 6d13b90b | config/initializers/apipie.rb:73:in `map'
 6d13b90b | config/initializers/apipie.rb:73:in `block (3 levels) in <top (required)>'
 6d13b90b | lib/foreman/middleware/logging_context_session.rb:22:in `call'
 6d13b90b | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 6d13b90b | katello (4.6.0.rc2) lib/katello/prevent_json_parsing.rb:12:in `call'
2022-09-13T14:59:01 [I|app|143bc3dd] Started GET "/about" for 172.17.219.6 at 2022-09-13 14:59:01 +0800
2022-09-13T14:59:01 [I|app|143bc3dd] Fork detected, re-connecting child process...
2022-09-13T14:59:01 [I|app|143bc3dd] Processing by AboutController#index as HTML
2022-09-13T14:59:01 [I|app|143bc3dd] Redirected to https://sh-flag21.bbs.bunnings.com.au/users/login
2022-09-13T14:59:01 [I|app|143bc3dd] Filter chain halted as :require_login rendered or redirected
2022-09-13T14:59:01 [I|app|143bc3dd] Completed 302 Found in 22ms (ActiveRecord: 1.6ms | Allocations: 20565)
2022-09-13T14:59:01 [F|app|143bc3dd]
 143bc3dd | NoMethodError (undefined method `dup_value!' for #<Hash:0x000055fa70cfda08>):
 143bc3dd |
 143bc3dd | lib/foreman/middleware/logging_context_request.rb:11:in `call'
 143bc3dd | katello (4.6.0.rc2) lib/katello/prevent_json_parsing.rb:12:in `call'

evgeni · September 13, 2022, 7:36am

But dalli seems to still be in the call-path, and thus still triggering the incompatiblity issue.

SarathHewage · September 14, 2022, 2:31am

2022-09-13 15:49:14 [NOTICE] [root] Loading installer configuration. This will take some time.
2022-09-13 15:49:16 [NOTICE] [root] Running installer with log based terminal output at level NOTICE.
2022-09-13 15:49:16 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions.
2022-09-13 15:49:21 [NOTICE] [configure] Starting system configuration.
2022-09-13 15:49:29 [NOTICE] [configure] 250 configuration steps out of 1570 steps complete.
2022-09-13 15:49:31 [NOTICE] [configure] 500 configuration steps out of 1571 steps complete.
2022-09-13 15:49:34 [NOTICE] [configure] 750 configuration steps out of 1575 steps complete.
2022-09-13 15:49:34 [NOTICE] [configure] 1000 configuration steps out of 1580 steps complete.
2022-09-13 15:50:19 [NOTICE] [configure] 1250 configuration steps out of 1580 steps complete.
2022-09-13 15:51:01 [ERROR ] [configure] /Stage[main]/Foreman::Register/Foreman_host[foreman-node.sample.com]: Could not evaluate: Error making GET request to Foreman at https://node.sample.com/api/v2/hosts: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on node.sample.com for detailed information
2022-09-13 15:51:21 [NOTICE] [configure] 1500 configuration steps out of 1580 steps complete.
2022-09-13 15:51:25 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_host[foreman-proxy-node.sample.com]: Could not evaluate: Error making GET request to Foreman at https://node.sample.com/api/v2/hosts: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on node.sample.com for detailed information
2022-09-13 15:51:25 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[node.sample.com]: Could not evaluate: Error making GET request to Foreman at https://node.sample.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on node.sample.com for detailed information
2022-09-13 15:51:26 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[node.sample.com]: Failed to call refresh: Error making GET request to Foreman at https://node.sample.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on node.sample.com for detailed information
2022-09-13 15:51:26 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[node.sample.com]: Error making GET request to Foreman at https://node.sample.com/api/v2/smart_proxies: Response: 500 Internal Server Error: Check /var/log/foreman/production.log on node.sample.com for detailed information
2022-09-13 15:51:28 [NOTICE] [configure] System configuration has finished.

There were errors detected during install.
Please address the errors and re-run the installer to ensure the system is properly configured.
Failing to do so is likely to result in broken functionality.

The full log is at /var/log/foreman-installer/katello.log

This is the log
cb66747b | /usr/share/gems/gems/puma-5.6.4/lib/puma/server.rb:441:in process_client’ cb66747b | /usr/share/gems/gems/puma-5.6.4/lib/puma/thread_pool.rb:147:in block in spawn_thread’
cb66747b | /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in block in create_with_logging_context’ 2022-09-13T15:51:25 [I|app|cb66747b] Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 11.6ms | Allocations: 5370) 2022-09-13T15:51:25 [I|app|cb66747b] Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 17.3ms | Allocations: 10721) 2022-09-13T15:51:25 [I|app|cb66747b] Completed 500 Internal Server Error in 48ms (Views: 18.4ms | ActiveRecord: 10.6ms | Allocations: 13032) 2022-09-13T15:51:25 [F|app|cb66747b] cb66747b | NoMethodError (undefined method dup_value!’ for #String:0x0000557c757cda70):
cb66747b |
cb66747b | config/initializers/apipie.rb:64:in block (3 levels) in <top (required)>’ cb66747b | config/initializers/apipie.rb:73:in block (4 levels) in <top (required)>’
cb66747b | config/initializers/apipie.rb:73:in each’ cb66747b | config/initializers/apipie.rb:73:in map’
cb66747b | config/initializers/apipie.rb:73:in block (3 levels) in <top (required)>’ cb66747b | lib/foreman/middleware/logging_context_session.rb:22:in call’
cb66747b | lib/foreman/middleware/logging_context_request.rb:11:in call’ cb66747b | katello (4.6.0.rc2) lib/katello/prevent_json_parsing.rb:12:in call’
2022-09-13T15:51:25 [I|app|7e26b9b1] Started GET “/api/v2/smart_proxies?search=name%3D%22node.sample.com%22” for 10.10.10.10 at 2022-09-13 15:51:25 +0800
2022-09-13T15:51:25 [I|app|7e26b9b1] Processing by Api::V2::SmartProxiesController#index as JSON
2022-09-13T15:51:25 [I|app|7e26b9b1] Parameters: {“search”=>“name=“node.sample.com””, “apiv”=>“v2”, “smart_proxy”=>{}}
2022-09-13T15:51:25 [W|app|7e26b9b1] Action failed
2022-09-13T15:51:25 [I|app|7e26b9b1] Backtrace for ‘Action failed’ error (NoMethodError): undefined method dup_value!’ for 0:Integer 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/cache/strategy/local_cache.rb:68:in write_entry’
7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/cache/strategy/local_cache.rb:162:in write_entry’ 7e26b9b1 | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:144:in block (2 levels) in write’
7e26b9b1 | /usr/share/gems/gems/dalli-2.7.6/lib/dalli/client.rb:262:in with’ 7e26b9b1 | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:82:in with’
7e26b9b1 | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:142:in block in write’ 7e26b9b1 | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:371:in block in instrument’
7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/notifications.rb:205:in instrument’ 7e26b9b1 | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:371:in instrument’
7e26b9b1 | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:141:in write’ 7e26b9b1 | /usr/share/gems/gems/dalli-2.7.6/lib/active_support/cache/dalli_store.rb:115:in fetch’
7e26b9b1 | /usr/share/foreman/app/services/foreman/bruteforce_protection.rb:10:in get_login_failures’ 7e26b9b1 | /usr/share/foreman/app/services/foreman/bruteforce_protection.rb:18:in bruteforce_attempt?’
7e26b9b1 | /usr/share/foreman/app/controllers/concerns/foreman/controller/bruteforce_protection.rb:9:in bruteforce_attempt?’ 7e26b9b1 | /usr/share/foreman/app/controllers/api/base_controller.rb:194:in authorize’
7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:427:in block in make_lambda’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:198:in block (2 levels) in halting’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/abstract_controller/callbacks.rb:34:in block (2 levels) in module:Callbacks’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:199:in block in halting’
7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:512:in block in invoke_before’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:512:in each’
7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:512:in invoke_before’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:115:in block in run_callbacks’
7e26b9b1 | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in clear_thread’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:126:in block in run_callbacks’
7e26b9b1 | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in set_topbar_sweeper_controller’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:126:in block in run_callbacks’
7e26b9b1 | /usr/share/gems/gems/audited-4.10.0/lib/audited/sweeper.rb:14:in around’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:126:in block in run_callbacks’
7e26b9b1 | /usr/share/gems/gems/audited-4.10.0/lib/audited/sweeper.rb:14:in around’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:126:in block in run_callbacks’
7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:137:in run_callbacks’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/abstract_controller/callbacks.rb:41:in process_action’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal/rescue.rb:22:in process_action’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal/instrumentation.rb:34:in block in process_action’
7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/notifications.rb:203:in block in instrument’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/notifications/instrumenter.rb:24:in instrument’
7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/notifications.rb:203:in instrument’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal/instrumentation.rb:33:in process_action’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal/params_wrapper.rb:249:in process_action’ 7e26b9b1 | /usr/share/gems/gems/activerecord-6.1.6.1/lib/active_record/railties/controller_runtime.rb:27:in process_action’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/abstract_controller/base.rb:165:in process’ 7e26b9b1 | /usr/share/gems/gems/actionview-6.1.6.1/lib/action_view/rendering.rb:39:in process’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal.rb:190:in dispatch’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_controller/metal.rb:254:in dispatch’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/route_set.rb:50:in dispatch’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/route_set.rb:33:in serve’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/mapper.rb:19:in block in class:Constraints’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/mapper.rb:49:in serve’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/journey/router.rb:50:in block in serve’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/journey/router.rb:32:in each’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/journey/router.rb:32:in serve’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/routing/route_set.rb:842:in call’
7e26b9b1 | /usr/share/gems/gems/katello-4.6.0.rc2/lib/katello/middleware/organization_created_enforcer.rb:18:in call’ 7e26b9b1 | /usr/share/gems/gems/katello-4.6.0.rc2/lib/katello/middleware/event_daemon.rb:10:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in call’
7e26b9b1 | /usr/share/gems/gems/apipie-dsl-2.5.0/lib/apipie_dsl/static_dispatcher.rb:67:in call’ 7e26b9b1 | /usr/share/gems/gems/apipie-rails-0.5.20/lib/apipie/static_dispatcher.rb:66:in call’
7e26b9b1 | /usr/share/gems/gems/apipie-rails-0.5.20/lib/apipie/extractor/recorder.rb:137:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in call’ 7e26b9b1 | /usr/share/foreman/lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in call’
7e26b9b1 | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in call’ 7e26b9b1 | /usr/share/gems/gems/apipie-rails-0.5.20/lib/apipie/middleware/checksum_in_headers.rb:27:in call’
7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/tempfile_reaper.rb:15:in call’ 7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/etag.rb:27:in call’
7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/conditional_get.rb:27:in call’ 7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/head.rb:12:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/http/permissions_policy.rb:22:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/http/content_security_policy.rb:19:in call’
7e26b9b1 | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in call’ 7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/session/abstract/id.rb:266:in context’
7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/session/abstract/id.rb:260:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/cookies.rb:689:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/callbacks.rb:27:in block in call’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/callbacks.rb:98:in run_callbacks’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/callbacks.rb:26:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/debug_exceptions.rb:29:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in call’
7e26b9b1 | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/rack/logger.rb:37:in call_app’ 7e26b9b1 | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/rack/logger.rb:28:in call’
7e26b9b1 | /usr/share/gems/gems/sprockets-rails-3.4.2/lib/sprockets/rails/quiet_assets.rb:13:in call’ 7e26b9b1 | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/remote_ip.rb:81:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/request_id.rb:26:in call’
7e26b9b1 | /usr/share/gems/gems/katello-4.6.0.rc2/lib/katello/prevent_json_parsing.rb:12:in call’ 7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/method_override.rb:24:in call’
7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/runtime.rb:22:in call’ 7e26b9b1 | /usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/executor.rb:14:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/static.rb:24:in call’
7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/sendfile.rb:110:in call’ 7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/ssl.rb:77:in call’
7e26b9b1 | /usr/share/gems/gems/actionpack-6.1.6.1/lib/action_dispatch/middleware/host_authorization.rb:142:in call’ 7e26b9b1 | /usr/share/gems/gems/secure_headers-6.3.4/lib/secure_headers/middleware.rb:11:in call’
7e26b9b1 | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/engine.rb:539:in call’ 7e26b9b1 | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/railtie.rb:207:in public_send’
7e26b9b1 | /usr/share/gems/gems/railties-6.1.6.1/lib/rails/railtie.rb:207:in method_missing’ 7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/urlmap.rb:74:in block in call’
7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/urlmap.rb:58:in each’ 7e26b9b1 | /usr/share/gems/gems/rack-2.2.4/lib/rack/urlmap.rb:58:in call’
7e26b9b1 | /usr/share/gems/gems/puma-5.6.4/lib/puma/configuration.rb:252:in call’ 7e26b9b1 | /usr/share/gems/gems/puma-5.6.4/lib/puma/request.rb:77:in block in handle_request’
7e26b9b1 | /usr/share/gems/gems/puma-5.6.4/lib/puma/thread_pool.rb:340:in with_force_shutdown’ 7e26b9b1 | /usr/share/gems/gems/puma-5.6.4/lib/puma/request.rb:76:in handle_request’
7e26b9b1 | /usr/share/gems/gems/puma-5.6.4/lib/puma/server.rb:441:in process_client’ 7e26b9b1 | /usr/share/gems/gems/puma-5.6.4/lib/puma/thread_pool.rb:147:in block in spawn_thread’
7e26b9b1 | /usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in block in create_with_logging_context’ 2022-09-13T15:51:25 [I|app|7e26b9b1] Rendered api/v2/errors/custom_error.json.rabl within api/v2/layouts/error_layout (Duration: 4.9ms | Allocations: 5370) 2022-09-13T15:51:25 [I|app|7e26b9b1] Rendered layout api/v2/layouts/error_layout.json.erb (Duration: 9.1ms | Allocations: 10721) 2022-09-13T15:51:25 [I|app|7e26b9b1] Completed 500 Internal Server Error in 45ms (Views: 9.6ms | ActiveRecord: 9.9ms | Allocations: 12627) 2022-09-13T15:51:25 [F|app|7e26b9b1] 7e26b9b1 | NoMethodError (undefined method dup_value!’ for #String:0x0000557c74d08b30):
7e26b9b1 |
7e26b9b1 | config/initializers/apipie.rb:64:in block (3 levels) in <top (required)>’ 7e26b9b1 | config/initializers/apipie.rb:73:in block (4 levels) in <top (required)>’
7e26b9b1 | config/initializers/apipie.rb:73:in each’ 7e26b9b1 | config/initializers/apipie.rb:73:in map’
7e26b9b1 | config/initializers/apipie.rb:73:in block (3 levels) in <top (required)>’ 7e26b9b1 | lib/foreman/middleware/logging_context_session.rb:22:in call’
7e26b9b1 | lib/foreman/middleware/logging_context_request.rb:11:in call’ 7e26b9b1 | katello (4.6.0.rc2) lib/katello/prevent_json_parsing.rb:12:in call’

bobbytables · September 14, 2022, 11:08am

Hello,

I’ll just quickly jump in and say that I experienced exactly the same problem after upgrading from Foreman 3.3.0 to 3.4.0 (on Debian 11).

In my case, the solution was to remove ruby-foreman-memcache package and set foreman::plugin::memcache: false in /etc/foreman-installer/scenarios.d/foreman-answers.yaml. Obviously, I re-ran foreman-installer afterwards.

SarathHewage · September 15, 2022, 2:04am

Thanks for the comment.
I checked it. I didn’t have ruby-foreman-memcache pakage install and already foreman::plugin::memcache: false set in /etc/foreman-installer/scenarios.d/foreman-answers.yaml.
However I had rubygem-foreman_memcache and I removed it and ran the foreman-installer.
Unfortunately still I get the same error.

lionking62 · September 16, 2022, 10:21am

I removed rubygem-dalli and rubygem-foreman_memcache, then all started working.

SarathHewage · September 20, 2022, 8:20am

This problem is now been fixed
Solution is:
sudo yum remove rubygem-foreman_memcache
sudo foreman-maintain service stop --exclude postgresql
sudo view /etc/foreman-installer/scenarios.d/katello-answers.yaml
foreman::plugin::memcache: false
sudo foreman-installer

Thanks for everyone who supported