Katello 4.8.4 doesn’t show available package updates anymore

dodo · October 5, 2023, 8:36am

In Katello Content Host GUI, it doesnt show updates for 1 Repo on 1 Host anymore
The Problem occurs just for this one Repo on one Content Host

Remi Repo PHP 8 for Centos 7
https://rpms.remirepo.net/enterprise/7/php80/x86_64/

affected Host: Centos 7 3.10.0-1160.99.1.el7.x86_64

affected Packages:
php.x86_64 8.0.30-1.el7.remi
php-bcmath.x86_64 8.0.30-1.el7.remi
php-cli.x86_64 8.0.30-1.el7.remi
php-common.x86_64 8.0.30-1.el7.remi
php-fpm.x86_64 8.0.30-1.el7.remi
php-gd.x86_64 8.0.30-1.el7.remi
php-gmp.x86_64 8.0.30-1.el7.remi
php-intl.x86_64 8.0.30-1.el7.remi
php-ldap.x86_64 8.0.30-1.el7.remi
php-mbstring.x86_64 8.0.30-1.el7.remi
php-mysqlnd.x86_64 8.0.30-1.el7.remi
php-opcache.x86_64 8.0.30-1.el7.remi
php-pdo.x86_64 8.0.30-1.el7.remi
php-pecl-zip.x86_64 1.22.2-1.el7.remi.8.0
php-process.x86_64 8.0.30-1.el7.remi
php-sodium.x86_64 8.0.30-1.el7.remi
php-xml.x86_64 8.0.30-1.el7.remi

currently installed packages:
php-mbstring-8.0.29-1.el7.remi.x86_64
php-pecl-imagick-im6-3.7.0-7.el7.remi.8.0.x86_64
php-sodium-8.0.29-1.el7.remi.x86_64
php-pecl-msgpack-2.2.0-1.el7.remi.8.0.x86_64
php-pecl-zip-1.22.1-1.el7.remi.8.0.x86_64
php-8.0.29-1.el7.remi.x86_64
php-gd-8.0.29-1.el7.remi.x86_64
php-pdo-8.0.29-1.el7.remi.x86_64
php-ldap-8.0.29-1.el7.remi.x86_64
php-opcache-8.0.29-1.el7.remi.x86_64
php-fpm-8.0.29-1.el7.remi.x86_64
php-pecl-apcu-5.1.22-1.el7.remi.8.0.x86_64
php-common-8.0.29-1.el7.remi.x86_64
php-xml-8.0.29-1.el7.remi.x86_64
php-process-8.0.29-1.el7.remi.x86_64
php-fedora-autoloader-1.0.1-2.el7.noarch
php-pear-1.10.13-5.el7.remi.noarch
php-pecl-igbinary-3.2.14-1.el7.remi.8.0.x86_64
php-mysqlnd-8.0.29-1.el7.remi.x86_64
php-intl-8.0.29-1.el7.remi.x86_64
php-cli-8.0.29-1.el7.remi.x86_64
php-gmp-8.0.29-1.el7.remi.x86_64
php-bcmath-8.0.29-1.el7.remi.x86_64
php-pecl-redis5-5.3.7-1.el7.remi.8.0.x86_64

When i login to Host via SSH, i can run yum update and it shows the available updates
But not in the Katello Content host GUI

What i tried so far:
katello-package-upload -f
katello-enabled-repos-upload -f
katello-tracer-upload -f
subscription-manager repos

I cant get Katello GUI to recognize these Updates

Can you please provide some help to troubleshoot this further?
Thank you

jeremylenz · October 5, 2023, 4:23pm

The .remi. in all the package names may be nonstandard; I wonder if that’s confusing the package profile uploader?

In foreman-rake console can you do some spot checks like this (check several of the packages in your list):

Katello::InstalledPackage.find_by(name: 'php')

And post the output here? The output will look something like this:

=> #<Katello::InstalledPackage:0x0000558703bde1a8
 id: 896,
 name: "newt",
 nvra: "newt-0.52.15-4.el7.x86_64",
 nvrea: "newt-0.52.15-4.el7.x86_64",
 epoch: "0",
 version: "0.52.15",
 release: "4.el7",
 arch: "x86_64",
 evr: "(0,\"{\"\"(0,)\"\",\"\"(52,)\"\",\"\"(15,)\"\"}\",\"{\"\"(4,)\"\",\"\"(0,el)\"\",\"\"(7,)\"\"}\")",
 vendor: "Red Hat, Inc.">

I’m looking to see if it’s recording the name, arch and evr correctly, since our SQL query for applicable packages sorts by that:

"SELECT DISTINCT ON (katello_installed_packages.name, katello_installed_packages.arch) katello_installed_packages.id " \
          "FROM katello_installed_packages INNER JOIN " \
          "katello_host_installed_packages ON " \
          "katello_installed_packages.id = " \
          "katello_host_installed_packages.installed_package_id " \
          "WHERE katello_host_installed_packages.host_id = " \
          "#{content_facet.host.id} ORDER BY katello_installed_packages.name, katello_installed_packages.arch, katello_installed_packages.evr DESC"

dodo · October 6, 2023, 6:50am

Hi,
thanks for taking your time

i’ve got the follwing

=> #<Katello::InstalledPackage id: 5078, name: "php-opcache", 
nvra: "php-opcache-7.1.32-1.el7.remi.x86_64", 
nvrea: "php-opcache-7.1.32-1.el7.remi.x86_64", 
epoch: "0", 
version: "7.1.32", 
release: "1.el7.remi", 
arch: "x86_64", 
evr: "(0,\"{\"\"(7,)\"\",\"\"(1,)\"\",\"\"(32,)\"\"}\",\"{\"\"(1,)\"\",\"\"(0...", 
vendor: nil>
irb(main):027:0>

Theres also php in Appstream, which is currently not affected as far as i know

=> #<Katello::InstalledPackage id: 2038, name: "php-fpm", 
nvra: "php-fpm-7.2.24-1.module_el8.3.0+2010+7c76a223.x86_...", 
nvrea: "php-fpm-7.2.24-1.module_el8.3.0+2010+7c76a223.x86_...", 
epoch: "0", 
version: "7.2.24", 
release: "1.module_el8.3.0+2010+7c76a223", 
arch: "x86_64", 
evr: "(0,\"{\"\"(7,)\"\",\"\"(2,)\"\",\"\"(24,)\"\"}\",\"{\"\"(1,)\"\",\"\"(0...", 
vendor: nil>
irb(main):028:0>

this is the only Host which consumes this repo

jeremylenz · October 9, 2023, 1:48pm

Do you have the repository url available? I’d like to try to sync it and see if I can reproduce the problem.

dodo · October 11, 2023, 9:27am

https://rpms.remirepo.net/enterprise/7/php80/x86_64/

jeremylenz · October 11, 2023, 4:44pm

Thanks!

I was able to get it to show the upgradable package without a problem:

We might try checking subscription-manager config --list

What does it show there under [rhsm] for package_profile_on_trans and report_package_profile? (Both should be 1)

I wonder who’s reporting package profiles for this host.

What subscription-manager version do you have on this host?

Is katello-host-tools installed on the host?

dodo · October 12, 2023, 8:17am

Hey
Thanks for helping to figure this out

i’ve got the following from subscription-manager config --list

[rhsm]
   auto_enable_yum_plugins = [1]
   baseurl = https://katello-01.example.com/pulp/content/
   ca_cert_dir = [/etc/rhsm/ca/]
   consumercertdir = [/etc/pki/consumer]
   entitlementcertdir = [/etc/pki/entitlement]
   full_refresh_on_yum = 1
   inotify = [1]
   manage_repos = [1]
   package_profile_on_trans = 1
   pluginconfdir = [/etc/rhsm/pluginconf.d]
   plugindir = [/usr/share/rhsm-plugins]
   productcertdir = [/etc/pki/product]
   repo_ca_cert = /etc/rhsm/ca/katello-server-ca.pem
   repomd_gpg_url = []
   report_package_profile = [1]

subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 4.2.13-${release}
subscription management rules: 5.43
subscription-manager: 1.24.52-2.el7.centos

rpm -qa katell\*
katello-host-tools-3.5.7-5.el7.noarch
katello-host-tools-tracer-3.5.7-5.el7.noarch

i’ve also tried to force reregister the Host
Problem still there

It seems it is just this single Repo, other Repos working just fine and recognize if there is an update available

dodo · October 12, 2023, 8:25am

i’ve also tried to recreate the repository, added it to my CCV

but Katello GUi still does not show applicable Updates for this Host

jeremylenz · October 12, 2023, 3:18pm

You may try removing katello-host-tools. This will cause subscription-manager to be the one that uploads the profiles.

Did you say it’s also only this host? Or is it this repo on any host?

dodo · December 27, 2023, 10:11am

Hello again

the issue i reported was just on 1 Host

But now i have the same Problem with REMI Repo with all Hosts

OS:
Centos 7
Almalinux 8 / 9
Rockylinux 8 / 9

 Katello::InstalledPackage.find_by(name: 'php-pecl-mcrypt')
=> #<Katello::InstalledPackage id: 5121, name: "php-pecl-mcrypt", nvra: "php-pecl-mcrypt-1.0.3-1.el7.remi.7.3.x86_64", nvrea: "php-pecl-mcrypt-1.0.3-1.el7.remi.7.3.x86_64", epoch: "0", version: "1.0.3", release: "1.el7.remi.7.3", arch: "x86_64", evr: "(0,\"{\"\"(1,)\"\",\"\"(0,)\"\",\"\"(3,)\"\"}\",\"{\"\"(1,)\"\",\"\"(0,...", vendor: nil>

But Update is available for this Package
The Host got following Version installed
rpm -qa php-pecl-mcrypt
php-pecl-mcrypt-1.0.6-1.el8.remi.8.0.x86_64

==========================================================================================================================================================================================
 Package                         Architecture           Version                              Repository                                                                              Size
==========================================================================================================================================================================================
Upgrading:
 php-pecl-mcrypt                 x86_64                 1.0.7-1.el8.remi.8.0                 provalida_Remi_Remi_Modular_repository_for_Enterprise_Linux_8_-_x86_64                  34 k

Transaction Summary
==========================================================================================================================================================================================

dodo · December 27, 2023, 11:38am

i think i fixed it

i’ve configured the REMI Repos with “additive” retain “5”
After i reconfigured the Repos with “content only” and did a “Complete Sync”
The Installable Packages show up in Content Host GUI

I did not Publish a new Content View for my Lifecycle Enviroment

The Problem fixed itself it seems

If you needmore information please let me know

dodo · December 27, 2023, 11:45am

i also did
Verify Content Checksum on every Repo and Publish Repository Metadata

dodo · March 13, 2024, 6:41pm

Hello again,

just want to update you guys
The Problem still exists, i still haven’t figuered it out, but haven’t spent time on it either
We are about to deprovision this CentOS 7 box soon

Just want to check in, if you guys are still interesting in troubleshooting this

This still occurs just on this Host in combination with the mentioned Repo

thank you

greetings
Dominik