Problem: After running the do-release-upgrade command in Bionic (upgrade to Focal) Foreman does not start anymore. Or, the web UI tells We're sorry, but something went wrong.
The API seems to work already since we are fetching data from it.
Foreman and Proxy versions: 2.5
Foreman and Proxy plugin versions: 2.5
Distribution and version: Ubuntu 20.04 LTS x64
Other relevant data:
I’ve run the foreman-installer in noop mode to see if it complains about something but it was fine. The normal run offered:
[2022-10-19 09:30:01] /usr/lib/ruby/2.7.0/rubygems.rb:275:in `find_spec_for_exe': Could not find 'bundler' (1.17.3) required by your /usr/share/foreman/Gemfile.lock. (Gem::GemNotFoundException)
[2022-10-19 09:30:01] To update to the latest version installed on your system, run `bundle update --bundler`.
[2022-10-19 09:30:01] To install the missing version, run `gem install bundler:1.17.3`
[2022-10-19 09:30:01] from /usr/lib/ruby/2.7.0/rubygems.rb:294:in `activate_bin_path'
[2022-10-19 09:30:01] from /usr/bin/bundle:23:in `<main>'
So I’ve updated:
09:32]root@rmvm069:/usr/share/foreman# bundle update --bundler
Fetching gem metadata from https://rubygems.org/.........
_snip_
Using bundler 2.1.4
_snip_
Bundle updated!
I ran the foreman-installer again and it finished successfully.
But the web UI issue is still there. Now I found this error in the logs:
Please don’t update Bundler outside of the packages.
Can you please ensure all Foreman (and plugins) packages are installed from the right (focal) repository?
Worst case, force-reinstall all of them using apt install --reinstall <list of pkgs> (you should be able to get a rather succinct list using dpkg -l |grep foreman)
I am pretty sure they were not. Foreman was running on 18.04 then we did the release upgrade. But I am not in panic since I have snapshots of this VM and go back easily.
@evgeni The bundler error message also appears when I try to reinstall from the focal repos.
[13:20]root@rmvm069:~# apt install --reinstall foreman foreman-cli foreman-debug foreman-dynflow-sidekiq foreman-installer foreman-postgresql foreman-proxy foreman-service ruby-foreman-deface ruby-foreman-discovery ruby-foreman-remote-execution ruby-foreman-remote-execution-core ruby-foreman-setup ruby-foreman-tasks ruby-foreman-tasks-core ruby-hammer-cli-foreman
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
python-certifi python-cffi-backend python-chardet python-cryptography python-enum34 python-idna python-ipaddress python-openssl
python-pkg-resources python-six
Use 'apt autoremove' to remove them.
The following packages will be upgraded:
ruby-hammer-cli-foreman
1 upgraded, 0 newly installed, 15 reinstalled, 0 to remove and 7 not upgraded.
Need to get 77,6 MB/87,8 MB of archives.
After this operation, 1.024 B disk space will be freed.
Get:1 http://deb.theforeman.org focal/2.5 amd64 ruby-hammer-cli-foreman all 2.5.0-1 [84,3 kB]
Get:2 http://deb.theforeman.org focal/2.5 amd64 ruby-foreman-remote-execution-core all 1.4.8-1 [13,1 kB]
Get:3 http://deb.theforeman.org focal/2.5 amd64 ruby-foreman-tasks-core all 0.3.6-1 [10,3 kB]
Get:4 http://deb.theforeman.org focal/2.5 amd64 foreman amd64 2.5.4-2 [77,3 MB]
Get:5 http://deb.theforeman.org focal/2.5 amd64 foreman-proxy all 2.5.4-1 [89,0 kB]
Get:6 http://deb.theforeman.org plugins/2.5 amd64 ruby-foreman-deface all 1.5.3-1 [69,3 kB]
Get:7 http://deb.theforeman.org plugins/2.5 amd64 ruby-foreman-setup all 6.0.0-1 [49,7 kB]
Fetched 77,6 MB in 5s (16,5 MB/s)
(Reading database ... 166222 files and directories currently installed.)
Preparing to unpack .../00-ruby-hammer-cli-foreman_2.5.0-1_all.deb ...
Unpacking ruby-hammer-cli-foreman (2.5.0-1) over (2.5.0-1) ...
Preparing to unpack .../01-foreman-cli_2.5.4-2_all.deb ...
Unpacking foreman-cli (2.5.4-2) over (2.5.4-2) ...
Preparing to unpack .../02-foreman-installer_2.5.4-1_all.deb ...
Unpacking foreman-installer (2.5.4-1) over (2.5.4-1) ...
Preparing to unpack .../03-ruby-foreman-remote-execution-core_1.4.8-1_all.deb ...
Unpacking ruby-foreman-remote-execution-core (1.4.8-1) over (1.4.8-1) ...
Preparing to unpack .../04-ruby-foreman-tasks-core_0.3.6-1_all.deb ...
Unpacking ruby-foreman-tasks-core (0.3.6-1) over (0.3.6-1) ...
Preparing to unpack .../05-foreman_2.5.4-2_amd64.deb ...
Failed to stop dynflowd.service: Unit dynflowd.service not loaded.
invoke-rc.d: initscript dynflowd, action "stop" failed.
Unpacking foreman (2.5.4-2) over (2.5.4-2) ...
Preparing to unpack .../06-foreman-debug_2.5.4-2_all.deb ...
Unpacking foreman-debug (2.5.4-2) over (2.5.4-2) ...
Preparing to unpack .../07-foreman-dynflow-sidekiq_2.5.4-2_all.deb ...
Unpacking foreman-dynflow-sidekiq (2.5.4-2) over (2.5.4-2) ...
Preparing to unpack .../08-foreman-postgresql_2.5.4-2_all.deb ...
Unpacking foreman-postgresql (2.5.4-2) over (2.5.4-2) ...
Preparing to unpack .../09-foreman-proxy_2.5.4-1_all.deb ...
Unpacking foreman-proxy (2.5.4-1) over (2.5.4-1) ...
Preparing to unpack .../10-foreman-service_2.5.4-2_all.deb ...
Unpacking foreman-service (2.5.4-2) over (2.5.4-2) ...
Preparing to unpack .../11-ruby-foreman-deface_1.5.3-1_all.deb ...
Unpacking ruby-foreman-deface (1.5.3-1) over (1.5.3-1) ...
Preparing to unpack .../12-ruby-foreman-discovery_17.0.1-1_all.deb ...
Unpacking ruby-foreman-discovery (17.0.1-1) over (17.0.1-1) ...
Preparing to unpack .../13-ruby-foreman-remote-execution_4.5.6-1_all.deb ...
Unpacking ruby-foreman-remote-execution (4.5.6-1) over (4.5.6-1) ...
Preparing to unpack .../14-ruby-foreman-setup_6.0.0-1_all.deb ...
Unpacking ruby-foreman-setup (6.0.0-1) over (6.0.0-1) ...
Preparing to unpack .../15-ruby-foreman-tasks_4.1.5-1_all.deb ...
Unpacking ruby-foreman-tasks (4.1.5-1) over (4.1.5-1) ...
Setting up foreman-debug (2.5.4-2) ...
Setting up ruby-hammer-cli-foreman (2.5.0-1) ...
Setting up foreman (2.5.4-2) ...
Traceback (most recent call last):
2: from /usr/bin/bundle:23:in `<main>'
1: from /usr/lib/ruby/2.7.0/rubygems.rb:294:in `activate_bin_path'
/usr/lib/ruby/2.7.0/rubygems.rb:275:in `find_spec_for_exe': Could not find 'bundler' (1.17.3) required by your /usr/share/foreman/Gemfile.lock. (Gem::G
emNotFoundException)
To update to the latest version installed on your system, run `bundle update --bundler`.
To install the missing version, run `gem install bundler:1.17.3`
dpkg: error processing package foreman (--configure):
installed foreman package post-installation script subprocess returned error exit status 1
Setting up ruby-foreman-tasks-core (0.3.6-1) ...
dpkg: dependency problems prevent configuration of foreman-service:
foreman-service depends on foreman (= 2.5.4-2); however:
Package foreman is not configured yet.
dpkg: error processing package foreman-service (--configure):
dependency problems - leaving unconfigured
Setting up foreman-installer (2.5.4-1) ...
dpkg: dependency problems prevent configuration of ruby-foreman-discovery:
ruby-foreman-discovery depends on foreman (>= 1.24.0~rc1); however:
Package foreman is not configured yet.
dpkg: error processing package ruby-foreman-discovery (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of ruby-foreman-tasks:
ruby-foreman-tasks depends on foreman (>= 2.4.0~rc1); however:
Package foreman is not configured yet.
dpkg: error processing package ruby-foreman-tasks (--configure):
dependency problems - leaving unconfigured
Setting up foreman-proxy (2.5.4-1) ...
Setting up foreman-cli (2.5.4-2) ...
dpkg: dependency problems prevent configuration of ruby-foreman-deface:
ruby-foreman-deface depends on foreman; however:
Package foreman is not configured yet.
dpkg: error processing package ruby-foreman-deface (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of ruby-foreman-setup:
ruby-foreman-setup depends on foreman (>= 1.13.0~rc1); however:
Package foreman is not configured yet.
dpkg: error processing package ruby-foreman-setup (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of ruby-foreman-remote-execution:
ruby-foreman-remote-execution depends on foreman (>= 1.25.0~rc1); however:
Package foreman is not configured yet.
ruby-foreman-remote-execution depends on ruby-foreman-tasks (>= 0.15.1); however:
Package ruby-foreman-tasks is not configured yet.
ruby-foreman-remote-execution depends on ruby-foreman-deface; however:
Package ruby-foreman-deface is not configured yet.
dpkg: error processing package ruby-foreman-remote-execution (--configure):
dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of foreman-postgresql:
foreman-postgresql depends on foreman (= 2.5.4-2); however:
Package foreman is not configured yet.
dpkg: error processing package foreman-postgresql (--configure):
dependency problems - leaving unconfigured
Setting up ruby-foreman-remote-execution-core (1.4.8-1) ...
dpkg: dependency problems prevent configuration of foreman-dynflow-sidekiq:
foreman-dynflow-sidekiq depends on foreman (= 2.5.4-2); however:
Package foreman is not configured yet.
dpkg: error processing package foreman-dynflow-sidekiq (--configure):
dependency problems - leaving unconfigured
Processing triggers for man-db (2.9.1-1) ...
Errors were encountered while processing:
foreman
foreman-service
ruby-foreman-discovery
ruby-foreman-tasks
ruby-foreman-deface
ruby-foreman-setup
ruby-foreman-remote-execution
foreman-postgresql
foreman-dynflow-sidekiq
E: Sub-process /usr/bin/dpkg returned an error code (1)
Here also the initial state before running apt commands and after the OS upgrade:
[13:19]root@rmvm069:~# dpkg -l | grep foreman
ii foreman 2.5.4-2 amd64 Systems management web interface
ii foreman-cli 2.5.4-2 all metapackage providing hammer CLI for Foreman
ii foreman-debug 2.5.4-2 all provides support utility foreman-debug.
ii foreman-dynflow-sidekiq 2.5.4-2 all package providing dynflow sidekiq services for Foreman
ii foreman-installer 2.5.4-1 all Automated puppet-based installer for The Foreman
ii foreman-postgresql 2.5.4-2 all metapackage providing PostgreSQL dependencies for Foreman
ii foreman-proxy 2.5.4-1 all RESTful proxies for DNS, DHCP, TFTP, and Puppet
ii foreman-service 2.5.4-2 all package providing Foreman services running via puma
ii ruby-foreman-deface 1.5.3-1 all Foreman Deface Plugin Dependency
ii ruby-foreman-discovery 17.0.1-1 all Foreman Discovery Plugin
ii ruby-foreman-remote-execution 4.5.6-1 all Foreman Remote Execution Plugin
ii ruby-foreman-remote-execution-core 1.4.8-1 all Foreman remote execution - core bits
ii ruby-foreman-setup 6.0.0-1 all Foreman Setup Plugin
ii ruby-foreman-tasks 4.1.5-1 all Tasks management engine for Foreman.
ii ruby-foreman-tasks-core 0.3.6-1 all Foreman tasks - core bits
ii ruby-hammer-cli-foreman 2.5.0-1 all Foreman commands for Hammer
Cool, if you have snapshots, we can try a few things out
For me, the following seems to have worked correctly around the Bundler issue:
# do-release-upgrade
# vi /etc/apt/sources.list.d/foreman.list (uncomment the repos again, as d-r-u has disabled them, check it reads "focal" not "bionic"
# apt update
# cd /usr/share/foreman/
# bundle update --local --bundler
# apt install --reinstall foreman <and any other foreman packages>
