Foreman Upgrade error NoMethodError: undefined method `to_sym' for nil:NilClass

Hello,

i’m trying to Update my Foreman instance from Version 2.5.3 to Version 3.0.
I followed the description from Foreman :: Manual

systemctl stop httpd foreman.service foreman.socket dynflow
dnf upgrade https://yum.theforeman.org/releases/3.0/el8/x86_64/foreman-release.rpm
dnf clean metadata
dnf module reset ruby
dnf module enable ruby:2.7
dnf upgrade ruby* foreman*
dnf install foreman-plugin-puppet

Next step Foreman will migrate Databases with foreman-rake db:migrate
There are some errors:

[root@foreman01 ~]# foreman-rake db:migrate
Rubocop not loaded.
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
rake aborted!
NoMethodError: undefined method to_sym' for nil:NilClass Did you mean? to_m /usr/share/foreman/app/services/setting_registry.rb:177:in block in _load_category_from_db’
/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in each' /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in each’
/usr/share/foreman/app/services/setting_registry.rb:175:in _load_category_from_db' /usr/share/foreman/app/services/setting_registry.rb:120:in block in load_definitions’
/usr/share/foreman/app/services/setting_registry.rb:117:in each' /usr/share/foreman/app/services/setting_registry.rb:117:in load_definitions’
/usr/share/foreman/config/initializers/foreman.rb:29:in <top (required)>' /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in load’
/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in block in load' /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:291:in load_dependency’
/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in load' /usr/share/foreman/config/initializers/0_print_time_spent.rb:38:in block in load’
/usr/share/foreman/config/initializers/0_print_time_spent.rb:17:in benchmark' /usr/share/foreman/config/initializers/0_print_time_spent.rb:38:in load’
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:666:in block in load_config_initializer' /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:182:in instrument’
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:665:in load_config_initializer' /usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:625:in block (2 levels) in class:Engine
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:624:in each' /usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:624:in block in class:Engine
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:32:in instance_exec' /usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:32:in run’
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:61:in block in run_initializers' /usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:50:in each’
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:50:in tsort_each_child' /usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:60:in run_initializers’
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application.rb:363:in initialize!' /usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in public_send’
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in method_missing' /usr/share/foreman/config/environment.rb:5:in <top (required)>’
/usr/share/gems/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require' /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in block in require’
/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:291:in load_dependency' /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in require’
/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application.rb:339:in require_environment!' /usr/share/gems/gems/railties-6.0.3.7/lib/rails/application.rb:523:in block in run_tasks_blocks’
/usr/share/gems/gems/rake-13.0.1/exe/rake:27:in `<top (required)>’
Tasks: TOP => db:migrate => db:load_config => environment
(See full trace by running task with --trace)

Has someone an idea? I tried “su - postgres -c ‘vacuumdb --full --dbname=foreman’” and “foreman-installer --noop --verbose”. There are always the same errors…

My OS is CentOS Linux release 8.4.2105.

Regars
nurweilesgeht

same issue if I try to upgrade to nightly

Seems to be related to the new settings registry, @ezr-ondrej have you seen this elsewhere? Any thoughts?

Hello, I disabled “24. [✗] Configure foreman_plugin_expire_hosts“ + removing the package via dnf and foreman works fine.

1 Like

Sounds like the plugin is using even older setting definitions than old.
I’ll get the setting loader to count with that as well, but not sure if we will get that into 3.0.1

hi,
is there a solution for that issue on the horizon ?

i tried to install foreman 3.0.1 and got still that error like with foreman 3.0.0

/usr/share/foreman/app/services/setting_registry.rb:190:in `block in _load_category_from_db': undefined method `to_sym' for nil:NilClass (NoMethodError)
Did you mean?  to_m
from /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in `each'
from /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in `each'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/app/services/setting_registry.rb:188:in `_load_category_from_db'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/app/services/setting_registry.rb:120:in `block in load_definitions'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/app/services/setting_registry.rb:117:in `each'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/app/services/setting_registry.rb:117:in `load_definitions'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/config/initializers/foreman.rb:29:in `<top (required)>'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in `load'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in `block in load'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:291:in `load_dependency'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in `load'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/config/initializers/0_print_time_spent.rb:38:in `block in load'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/config/initializers/0_print_time_spent.rb:17:in `benchmark'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/config/initializers/0_print_time_spent.rb:38:in `load'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:666:in `block in load_config_initializer'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:182:in `instrument'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:665:in `load_config_initializer'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:625:in `block (2 levels) in <class:Engine>'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:624:in `each'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:624:in `block in <class:Engine>'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:32:in `instance_exec'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:32:in `run'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:61:in `block in run_initializers'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:228:in `block in tsort_each'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:431:in `each_strongly_connected_component_from'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:421:in `block in each_strongly_connected_component_from'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:50:in `each'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:50:in `tsort_each_child'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:415:in `call'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:415:in `each_strongly_connected_component_from'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:349:in `block in each_strongly_connected_component'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in `each'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in `call'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:347:in `each_strongly_connected_component'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:226:in `tsort_each'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/ruby/tsort.rb:205:in `tsort_each'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/initializable.rb:60:in `run_initializers'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/application.rb:363:in `initialize!'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `public_send'
Nov 05 15:01:31  foreman[11920]: from /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `method_missing'
Nov 05 15:01:31  foreman[11920]: from /usr/share/foreman/config/environment.rb:5:in `<top (required)>'
Nov 05 15:01:31  foreman[11920]: from /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:146:in `require'

@ezr-ondrej @Marek_Hulan

Hi @cooler22,

So I’ve introduced workaround in https://github.com/theforeman/foreman/commit/5c3289f724 but that is only for 3.1 currently, you can try to patch your instance if it helps.

Tho it would be helpful to know what plugins you have on your instance to figure out what plugin might be causing it.

@ezr-ondrej,
i currently downgraded my Testsystem back to version 2.5.x. and there foreman starts without any problem.

we using in our infrastructure the normal “theforeman/foreman”-module (18.2.0 - theforeman/foreman · Foreman server configuration · Puppet Forge) without special plugins i guess.
therefore it seems there is a major change from version 2.5 to 3.x that causes the issue, if i update the repos to version 3.0 and install then the packages from that version range.

maybe i could try your development branch with the fix, but would be cooler to have that fix over a stable rpm-version package.

Yeah, I’ll try to get that fix into 3.0, but I doubt this is happening on Foreman without plugin. I’ve seen this issue in other scenarios, but only with special plugins, that people usually do not have in development setups.

Thant beling said, I’ve referenced the fix, just for you to have a way to quick test the patch, before we manage to propagate it into stable branches.
Propagating to the stable was also the reason I asked for the plugins list, to know how critical this is.

I’ve digged bit more and created a ticket Bug #33861: Foreman wont start with old settings in DB - Foreman and filed a proper fix.

as i can see we are currently installed:

foreman,
foreman-proxy
foreman-openscap / scap-client
foreman_proxy::tftp
foreman_proxy::bmc
foreman_proxy::realm
foreman-proxy-discovery
foreman-proxy-dynflow

@ezr-ondrej,
just informational,
added your fix under - /usr/share/foreman/app/services/setting_registry.rb

issue still persists

/usr/share/foreman/app/services/setting_registry.rb:191:in block in _load_category_from_db': undefined method to_sym’ for nil:NilClass (NoMethodError)

This should be fixed by https://github.com/theforeman/foreman/pull/8913
What will be included in 3.0.2.

Thanks for reporting @cooler22 !

@ezr-ondrej,
i tested the new Release Candidate - foreman-3.1.0-0.1.rc1.el7.noarch.rpm - and service finally starts again now :slight_smile:

thank you very much, hopefully the change is then still in the stable version :slight_smile:
You and the team have a nice weekend.

1 Like