Applicable rpms listed from inactive module streams

Problem:
Since migrating to pulp3 and upgrading to katello 4.0 all our centos8 server show a count of installable updates even though those packages are not available in the enabled module stream.

Currently, we have the perl 5.26 module stream enable on those server:

# dnf module list perl*
...
CentOS-8 - AppStream
Name                          Stream                 Profiles                      Summary                                              
perl                          5.24                   common [d], minimal           Practical Extraction and Report Language             
perl                          5.26 [d][e]            common [d], minimal           Practical Extraction and Report Language             
perl                          5.30                   common [d], minimal           Practical Extraction and Report Language             
perl-App-cpanminus            1.7044 [d]             common [d]                    Get, unpack, build and install CPAN modules          
perl-DBD-MySQL                4.046 [d][e]           common [d]                    A MySQL interface for Perl                           
perl-DBD-Pg                   3.7 [d]                common [d]                    A PostgreSQL interface for Perl                      
perl-DBD-SQLite               1.58 [d]               common [d]                    SQLite DBI driver                                    
perl-DBI                      1.641 [d][e]           common [d]                    A database access API for Perl                       
perl-FCGI                     0.78 [d]               common [d]                    FastCGI Perl bindings                                
perl-IO-Socket-SSL            2.066 [d][e]           common [d]                    Perl library for transparent TLS                     
perl-YAML                     1.24 [d]               common [d]                    Perl parser for YAML                                 
perl-libwww-perl              6.34 [d][e]            common [d]                    A Perl interface to the World-Wide Web               

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

For instance, we have the perl-libwww-perl installed:

# rpm -q perl-libwww-perl
perl-libwww-perl-6.34-1.module_el8.3.0+416+dee7bcef.noarch

With Katello 4 it suggest perl-libwww-perl-6.34-1.module_el8.3.0+416+9a1a0b3f.noarch as installable package. However that package requires ‘perl(:MODULE_COMPAT_5.30.1)’, i.e. it’s for the perl:5.30 module stream and not for the perl:5.26. It’s not offered as update on the server and it’s not possible to install it either because of that dependency.

With Katello 3.18.2 and pulp2 all those servers were correctly listed as up-to-date.

Expected outcome:
List only applicable/installable packages which are really available in the currently enabled module streams.

Foreman and Proxy versions:
Foreman 2.4, Katello 4.0

I see the exact same issue with katello 3.18.2 after upgrading to pulp3.

After a detour I just yesterday migrated and updated from Katello 3.18.2 with pulp2 to pulp3 to katello 4.0.1 again.

The applicability for packages from module streams is still broken. For instance, it shows perl-DBI-1.641-3.module_el8.1.0+199+249f9f29.x86_64 as applicable.

On the server, I have the perl-5.26 module enabled. I have the rpm perl-DBI-1.641-3.module_el8.1.0+199+8f0a6bbd.x86_64 installed which is for 5.26.

The suggested rpm perl-DBI-1.641-3.module_el8.1.0+199+249f9f29.x86_64 is not available on the server as that rpm is for the perl 5.24 module.

With 3.18/pulp2 the server did not show any applicable updates.

Created issue Bug #32739: Incorrect applicablity in katello 4 - Katello - Foreman

Bug #32739: Incorrect applicablity in katello 4 - Katello - Foreman has been set to “need more information”. I have asked a couple of days what is needed by got not response.

On issue Bug #32739: Incorrect applicablity in katello 4 - Katello - Foreman there is mention of @iballou and @Justin_Sherrill

@jjeffers originally set it to “need more information”.

Anyone has some insights?

We’ve had this on our immediate backlog for a while. I’m hoping we can get to it soon.

How to reproduce: with any CentOS 8, e.g. minimal installation. We always have perl-LDAP installed. This enables modules perl 5.26, perl-IO-Socket-SSL 2.066 and perl-libwww-perl 6.34 and installs the packages. But then I see 23 upgradable packages listed for the content host.

Upgradable Package
	perl-Data-Dump-1.23-7.module_el8.3.0+416+9a1a0b3f.noarch
	perl-Digest-HMAC-1.03-17.module_el8.3.0+416+9a1a0b3f.noarch
	perl-Encode-Locale-1.05-10.module_el8.3.0+416+9a1a0b3f.noarch
	perl-File-Listing-6.04-17.module_el8.3.0+416+9a1a0b3f.noarch
	perl-HTML-Parser-3.72-15.module_el8.3.0+416+9a1a0b3f.x86_64
	perl-HTML-Tagset-3.20-34.module_el8.3.0+416+9a1a0b3f.noarch
	perl-HTTP-Cookies-6.04-2.module_el8.3.0+416+9a1a0b3f.noarch
	perl-HTTP-Date-6.02-19.module_el8.3.0+416+9a1a0b3f.noarch
	perl-HTTP-Message-6.18-1.module_el8.3.0+416+9a1a0b3f.noarch
	perl-HTTP-Negotiate-6.01-19.module_el8.3.0+416+9a1a0b3f.noarch
	perl-IO-HTML-1.001-11.module_el8.3.0+416+9a1a0b3f.noarch
	perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+3b5aa49a.noarch
	perl-IO-Socket-SSL-2.066-4.module_el8.3.0+410+4cc2efa4.noarch
	perl-libwww-perl-6.34-1.module_el8.3.0+416+9a1a0b3f.noarch
	perl-LWP-MediaTypes-6.02-15.module_el8.3.0+416+9a1a0b3f.noarch
	perl-Mozilla-CA-20160104-7.module_el8.3.0+416+9a1a0b3f.noarch
	perl-Net-HTTP-6.17-2.module_el8.3.0+416+9a1a0b3f.noarch
	perl-Net-SSLeay-1.88-1.module_el8.3.0+410+3b5aa49a.x86_64
	perl-Net-SSLeay-1.88-1.module_el8.3.0+410+4cc2efa4.x86_64
	perl-NTLM-1.09-17.module_el8.3.0+416+9a1a0b3f.noarch
	perl-TimeDate-1:2.30-15.module_el8.3.0+416+9a1a0b3f.noarch
	perl-Try-Tiny-0.30-7.module_el8.3.0+416+9a1a0b3f.noarch
	perl-WWW-RobotRules-6.02-18.module_el8.3.0+416+9a1a0b3f.noarch

If you cannot reproduce it this way, then I guess it would be some bogus data from the migration in the database…

@gvde I have two PRs out that should fix this together. One of them is a patch to subscription-manager. Turns out certain module streams (like perl-IO-Socket-SSL) have multiple “enabled” sibling streams but only one is actually active. I thought this was a dnf bug until I talked to the RHEL 8 folks. Thankfully we can find the active one using the dnf library, so we’re having sub-man send over the “active” status to Katello.

Great. Am I right: this requires changes to the redhat subscription-manager? I guess this will take a while until it gets published then…

I do wonder, though, why it was working before with 3.18/pulp2? The active flag wasn’t available to Katello before, either, so how did it work then?

If I manually patch the profile.py on my CentOS 8 clients as well as the katello patch for testing, what should I do to verify it’s working? Is a dnf uploadprofile enough to get the information to the katello server and get the applicability recalculated?

Yep, you need to patch subscription-manager and Katello. The change is relatively small and harmless to sub-man, so I’m hoping it won’t take too long. If it does, we’ll have to solve it fully from Katello.

Pulp 2 performed all of the applicability calculations for Katello pre-Pulp 3. Now, the applicability code is in Katello. Pulp 2 didn’t match dnf perfectly (mostly with regards to modules I think), so we started the Katello applicability code with a blank slate back in Katello 3.16 for Pulp 3 users.

dnf uploadprofile should do it as long as you see the profile update and applicability regeneration tasks afterwards in the task browser.

I have tried to test your PR. But it doesn’t seem to work for me:

I have patched /usr/lib64/python3.6/site-packages/rhsm/profile.py on one of my test clients.

I have patched /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1/app/lib/actions/katello/host/upload_profiles.rb on my main foreman server. foreman-maintain service stop and start.

I ran uploadprofile on the client once foreman was up and running again.

# dnf uploadprofile  --force-upload
Updating Subscription Management repositories.
Package profile updates
        status: 1
        updates: []
        exceptions: 
        

But I still see those perl packages as Upgradable Package in foreman on that host.

The katello PR is only for the main server? The RPM isn’t installed on the content proxy thus I didn’t apply any patch on the content proxy. My test client is connected to the content proxy, if that matters.