CentOS 8 update to foreman 2.1.0-1.el8 issue rpm resolution

[rm-td]

Problem:
Mismatching rpm packages performing “yum update” on CentOS 8 for latest foreman.

# yum update
Last metadata expiration check: 2:45:55 ago on Mon 06 Jul 2020 15:58:04 UTC.
Error:
 Problem: package foreman-2.1.0-1.el8.noarch requires rubygem(net-ssh) = 4.2.0, but none of the providers can be installed
  - cannot install both rubygem-net-ssh-5.1.0-2.el8.noarch and rubygem-net-ssh-4.2.0-2.el8.noarch
  - cannot install both rubygem-net-ssh-4.2.0-2.el8.noarch and rubygem-net-ssh-5.1.0-2.el8.noarch
  - cannot install the best update candidate for package rubygem-net-ssh-4.2.0-2.el8.noarch
  - cannot install the best update candidate for package foreman-2.1.0-0.22.rc3.el8.noarch
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

On initial installation the foreman-installer script clearly deploys rubygem-net-ssh from the foreman repo and not later version seeminly available from epel repo. This does not appear to be an issue for initial deployment.

“yum update” today displays a mismatch in rpm versions as above.

The mismatch appears to come from differing versions in foreman and epel repo:

# yum list | grep rubygem-net-ssh
rubygem-net-ssh.noarch                                  4.2.0-2.el8                                       @foreman
rubygem-net-ssh.noarch                                  5.1.0-2.el8                                       epel
rubygem-net-ssh-doc.noarch                              5.1.0-2.el8                                       epel
rubygem-net-ssh-gateway.noarch                          1.2.0-6.el8                                       foreman-plugins
rubygem-net-ssh-gateway-doc.noarch                      1.2.0-6.el8                                       foreman-plugins
rubygem-net-ssh-krb.noarch                              0.4.0-3.el8                                       foreman-plugins
rubygem-net-ssh-krb-doc.noarch                          0.4.0-3.el8                                       foreman-plugins
rubygem-net-ssh-multi.noarch                            1.2.0-9.el8                                       foreman-plugins
rubygem-net-ssh-multi-doc.noarch                        1.2.0-9.el8                                       foreman-plugins

Expected outcome:
Update successfuly resolves rpm dependencies and matches successfully.

Foreman and Proxy versions:
Updating from foreman-2.1.0-0.22.rc3.el8.noarch to latest published foreman-2.1.0-1.el8

Foreman and Proxy plugin versions:
n/a

Distribution and version:
CentOS Linux release 8.2.2004

Other relevant data:

Conflicting package existing in EPEL repo. As no other packages are required from EPEL to complete update, a suitable workaround to permit update is to temporarily disable the epel 8 repo.
After editing /etc/yum/repos.d/epel.repo “enabled=0” a “yum update” elicits the following and completes correctly:

# yum update
Last metadata expiration check: 0:11:46 ago on Mon 06 Jul 2020 18:44:37 UTC.
Dependencies resolved.
=====================================================================================================================================================
 Package                                 Architecture          Version                                                Repository                Size
=====================================================================================================================================================
Upgrading:
 foreman                                 noarch                2.1.0-1.el8                                            foreman                   37 M
 foreman-cli                             noarch                2.1.0-1.el8                                            foreman                   25 k
 foreman-debug                           noarch                2.1.0-1.el8                                            foreman                   31 k
 foreman-dynflow-sidekiq                 noarch                2.1.0-1.el8                                            foreman                   27 k
 foreman-installer                       noarch                1:2.1.0-1.el8                                          foreman                  1.7 M
 foreman-postgresql                      noarch                2.1.0-1.el8                                            foreman                   26 k
 foreman-proxy                           noarch                2.1.0-1.el8                                            foreman                  156 k
 foreman-release                         noarch                2.1.0-1.el8                                            foreman                   12 k
 foreman-selinux                         noarch                2.1.0-1.el8                                            foreman                   54 k
 foreman-service                         noarch                2.1.0-1.el8                                            foreman                   27 k
 rubygem-dynflow                         noarch                1.4.6-1.fm2_1.el8                                      foreman                  372 k
 rubygem-foreman_maintain                noarch                1:0.6.6-1.el8                                          foreman                  155 k
Installing dependencies:
 cyrus-sasl                              x86_64                2.1.27-1.el8                                           BaseOS                    96 k
 cyrus-sasl-gssapi                       x86_64                2.1.27-1.el8                                           BaseOS                    49 k
 libvirt-libs                            x86_64                4.5.0-42.module_el8.2.0+320+13f867d7                   AppStream                4.1 M
 nmap-ncat                               x86_64                2:7.70-5.el8                                           AppStream                237 k
 rubygem-ruby-libvirt                    x86_64                0.7.1-1.el8                                            foreman                   91 k
 rubygem-xmlrpc                          noarch                0.3.0-105.module_el8.1.0+214+9be47fd7                  AppStream                 81 k
 yajl                                    x86_64                2.1.0-10.el8                                           AppStream                 41 k

Transaction Summary
=====================================================================================================================================================
Install   7 Packages
Upgrade  12 Packages

Per above, after temporarily disabling epel we see that rubygem-net-ssh is installed from the foreman repo and not epel and completes correctly.

[ehelms]

This is called out in the release post as an issue (due to how modules work in EL8):

Foreman 2.1.0 has been released!

Foreman 2.1.0 is now available! Big thanks to everyone who tested the release candidates, contributed fixes and helped with the technical side of building this release. This release contains a total of over 300 bug fixes and new features! Some highlights include initial support for EL8 distributions, external IPAM provider integration points, a change of the default webserver from Passenger to Puma, and upgrade to Ruby on Rails 6. There are a couple of known issues in this release: The vers…

@tbrisker I could find these known issues in the official release notes on the website.

[rm-td]

Hi, cheers for that didnt totally see the top section perhaps an addition in the upgrade section itself dunno.
https://theforeman.org/manuals/2.1/index.html#Releasenotesfor2.1

[tbrisker]

Thanks for bringing it up, i’ve opened https://github.com/theforeman/theforeman.org/pull/1643 to add the known issues in the release notes as well.

[rm-td]

Hello,
Could I ask for another pair of eyes on a potentially more elegant solution?
The workaround to upgrade is sufficient but as soon as the EPEL repo is subsequent re-enabled, continued errors on mismatched packages appear (and we know why):

# dnf update
Last metadata expiration check: 0:33:58 ago on Tue 07 Jul 2020 13:43:14 UTC.
Error:
 Problem: package foreman-2.1.0-1.el8.noarch requires rubygem(net-ssh) = 4.2.0, but none of the providers can be installed
  - cannot install both rubygem-net-ssh-5.1.0-2.el8.noarch and rubygem-net-ssh-4.2.0-2.el8.noarch
  - cannot install both rubygem-net-ssh-4.2.0-2.el8.noarch and rubygem-net-ssh-5.1.0-2.el8.noarch
  - cannot install the best update candidate for package rubygem-net-ssh-4.2.0-2.el8.noarch
  - cannot install the best update candidate for package foreman-2.1.0-1.el8.noarch
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Could I suggest that dnf-plugin-versionlock may be the answer?

# dnf install dnf-plugin-versionlock
Last metadata expiration check: 0:01:31 ago on Tue 07 Jul 2020 14:17:57 UTC.
Dependencies resolved.
=====================================================================================================================================================
 Package                                             Architecture                Version                           Repository                   Size
=====================================================================================================================================================
Installing:
 python3-dnf-plugin-versionlock                      noarch                      4.0.12-3.el8                      BaseOS                       58 k

Transaction Summary
=====================================================================================================================================================
Install  1 Package

followed by

# dnf versionlock rubygem-net-ssh-4.2.0-2.el8.noarch
Last metadata expiration check: 0:01:36 ago on Tue 07 Jul 2020 14:17:57 UTC.
Adding versionlock on: rubygem-net-ssh-0:4.2.0-2.el8.*

With EPEL repo enabled, a subsequent dnf update operates correctly with no errors:

# dnf update
Extra Packages for Enterprise Linux 8 - x86_64                                                                       9.3 MB/s | 7.2 MB     00:00
Last metadata expiration check: 0:00:02 ago on Tue 07 Jul 2020 14:20:04 UTC.
Dependencies resolved.
Nothing to do.
Complete!

Grateful for comments?

[ehelms]

That looks like a fine solution to work around it for htose that must have EPEL. May I ask, for your scenario, why you need EPEL?

[rm-td]

Yes, of course.
There’s a few packages in the EPEL repo very useful to those running virtual machines and/or having some interest in hardening techniques.

Off the top of my head two packages that spring to mind are haveged to guard against low-entropy conditions and rkhunter. The “layers of an onion” adding to overall security and stablitiy these are useful packages in the arsenal.

Of course, with the CentOS 8 foreman installer it now appears EPEL is no longer required as it was for CentOS 7, so yes, indeed, it only matters if somebody actually wants relevant packages from the repo.