Foreman 3.0 server with Katello 4.2 fails to install on Centos 7.9

I’ve been following the installation guide Installing Foreman 3.0 server with Katello 4.2 plugin on Enterprise Linux

The os is a Centos 7.9, vanilla installed.

From trial and error I firstly installed the following packages manually since they where updated in epel there version was not ok for the installer.

qpid-proton-c-0.34.0-1.el7.x86_64.rpm
tfm-rubygem-qpid_proton-0.34.0-2.el7.x86_64.rpm

The rest following the doc

yum localinstall https://yum.theforeman.org/releases/3.0/el7/x86_64/foreman-release.rpm
yum localinstall https://yum.theforeman.org/katello/4.2/katello/el7/x86_64/katello-repos-latest.rpm
yum localinstall https://yum.puppet.com/puppet6-release-el-7.noarch.rpm
yum install centos-release-scl-rh
yum install foreman-installer-katello

but when running

foreman-installer --scenario katello …

It ends up with a load of error related to the Sidekiq failing to start

2021-09-17 11:29:01 [ERROR ] [configure] Systemd start for dynflow-sidekiq@worker-hosts-queue-1 failed!
2021-09-17 11:29:01 [ERROR ] [configure] journalctl log for dynflow-sidekiq@worker-hosts-queue-1:
2021-09-17 11:29:01 [ERROR ] [configure] -- Logs begin at Fri 2021-09-17 12:20:02 CEST, end at Fri 2021-09-17 12:46:43 CEST. --
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:45 monforpoc2 systemd[1]: Starting Foreman jobs daemon - worker-hosts-queue-1 on sidekiq...
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:47 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:47.171Z 8749 TID-7g9 INFO: GitLab reliable fetch activated!
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:47 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:47.172Z 8749 TID-7et INFO: Booting Sidekiq 5.2.7 with redis options {:id=>"Sidekiq-server-PID-8749", :url=>"redis://localhost:6379/0"}
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:47 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:47.181Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:48 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:48.186Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:49 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:49.216Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:50 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:50.218Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:51 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:51.238Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:52 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:52.250Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:53 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:53.253Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:54 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:54.272Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:55 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:55.297Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:55 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_puppet-1.0.1/lib/foreman_puppet/register.rb:141: warning: already initialized constant Foreman::Plugin::RbacSupport::AUTO_EXTENDED_ROLES
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:55 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: /usr/share/foreman/app/registries/foreman/plugin/rbac_support.rb:5: warning: previous definition of AUTO_EXTENDED_ROLES was here
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:56 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:56.303Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:57 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:57.365Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:58 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:58.411Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:28:59 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:28:59.416Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:29:00 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: 2021-09-17T09:29:00.423Z 8749 TID-7et ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:29:00 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require': libruby.so.2.5: cannot open shared object file: No such file or directory - /opt/theforeman/tfm/root/usr/lib64/gems/ruby/qpid_proton-0.34.0/cproton.so (LoadError)
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:29:00 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: from /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:29:00 monforpoc2  dynflow-sidekiq@worker-hosts-queue-1[8749]: from /opt/theforeman/tfm/root/usr/share/gems/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'
2021-09-17 11:29:01 [ERROR ] [configure] Sep 17 11:29:00 monforpoc2 dynflow-sidekiq@worker-hosts-queue-1[8749]: from /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:324:in `block in require'

From this point on this service keeps on restarting, and the web interface isn’t responsive.

From the logs it complains about redis, but when checking this appears to be working

redis-cli -h localhost --latency-history localhost
min: 0, max: 7, avg: 0.10 (1395 samples) – 15.01 seconds range

so the `require’: libruby.so.2.5: cannot open shared object file: No such file or directory looks more relevant, but I tried installing ruby2.5 and make the system use it doing

update-alternatives --install /usr/bin/ruby ruby /opt/rh/rh-ruby25/root/bin/ruby 25 --slave /usr/bin/gem gem /opt/rh/rh-ruby25/root/bin/gem

but when I do that I get the following message

Sep 17 12:52:10 monforpoc dynflow-sidekiq@worker-hosts-queue-1[12744]: /opt/rh/rh-ruby27/root/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require': incompatible library version - /opt
/theforeman/tfm/root/usr/lib64/gems/ruby/qpid_proton-0.34.0/cproton.so (LoadError)

so at this point I’m not sure what to try next.

Expected outcome:
Successful Installation

Foreman and Proxy versions:
foreman-3.0.0-1.el7.noarch
foreman-installer-katello-3.0.0-1.el7.noarch
foreman-proxy-3.0.0-1.el7.noarch

I don’t know if @evgeni found your issue or independently, but there appears to be an ordering issue during installation:

That’s weird because it should be built against Ruby 2.7. @packaging?

My ordering issue was completely independent during some tests I did earlier this week.

Ended up retrying again, but now used the previous version of Foreman & Katello

Foreman 2.5
katello 4.1

Installed without any issue.

Which suits my purposes atm.

Cheers,

FTR, 3.0/4.2 should be also installable again, but it’s still in Release Candidate phase (even if close to GA)