CentOS 8 subscribers can only access one repo per subscription

caitslinux · July 29, 2020, 10:52pm

Problem:
RHEL 7 subscribers don’t appear to have any issues. CentOS 8 subscribers, however, only have access to the AppStream repo (BaseOS and PowerTools are not presented).

# subscription-manager repos --list
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   CrowdStrike_EL8
Repo Name: EL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CrowdStrike/EL8
Enabled:   1

Repo ID:   EPEL8
Repo Name: EPEL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/EPEL/EPEL8
Enabled:   1

Repo ID:   Inventory_EL8
Repo Name: EL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/OCS_Inventory/EL8
Enabled:   1

Repo ID:   CentOS_8_-_AppStream
Repo Name: 8 - AppStream
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CentOS/8_-_AppStream
Enabled:   1

Expected outcome:

Based on the activation key, the following should be available:

# subscription-manager repos --list
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   CrowdStrike_EL8
Repo Name: EL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CrowdStrike/EL8
Enabled:   1

Repo ID:   EPEL8
Repo Name: EPEL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/EPEL/EPEL8
Enabled:   1

Repo ID:   Inventory_EL8
Repo Name: EL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/OCS_Inventory/EL8
Enabled:   1

Repo ID:   CentOS_8_-_AppStream
Repo Name: 8 - AppStream
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CentOS/8_-_AppStream
Enabled:   1

Repo ID:   CentOS_8_-_BaseOS
Repo Name: 8 - BaseOS
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CentOS/8_-_BaseOS
Enabled:   1

Repo ID:   CentOS_8_-_PowerTools
Repo Name: 8 - PowerTools
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CentOS/8_-_PowerTools
Enabled:   1

Foreman and Proxy versions:

# hammer status
Version:           1.24.2
API Version:       v2
Database:          
    Status:          ok
    Server Response: Duration: 0ms
Plugins:           
 1) Name:    foreman-tasks
    Version: 0.17.5
 2) Name:    foreman_ansible
    Version: 4.0.5
 3) Name:    foreman_remote_execution
    Version: 2.0.8
 4) Name:    katello
    Version: 3.14.1
Smart Proxies:     
 1) Name:     foreman.server
    Version:  1.24.2
    Status:   ok
    Features: 
     1) Name:    pulp
        Version: 1.5.0
     2) Name:    templates
        Version: 1.24.2
     3) Name:    tftp
        Version: 1.24.2
     4) Name:    logs
        Version: 1.24.2
     5) Name:    httpboot
        Version: 1.24.2
Compute Resources: 

candlepin:         
    Status:          ok
    Server Response: Duration: 15ms
candlepin_auth:    
    Status:          ok
    Server Response: Duration: 13ms
pulp:              
    Status:          ok
    Server Response: Duration: 50ms
pulp_auth:         
    Status:          ok
    Server Response: Duration: 24ms
foreman_tasks:     
    Status:          ok
    Server Response: Duration: 3ms

Distribution and version:
RHEL 7

This issue does not seem related to: centos-host-cannot-subscribe-to-repositories

All subscriptions display an End Date of 2049-11-30 19:00:00 -0500.

Am I doing something wrong or missing a step?

gvde · July 30, 2020, 4:53am

Are you sure your content view “CentOS_8_Virtual_Servers” contains all repositories?

caitslinux · July 30, 2020, 5:17pm

Yes. I’ve confirmed the Content View contains all repositories. I’ve attempted to re-publish and re-promote the Content View, but to no avail.

caitslinux · July 30, 2020, 5:21pm

I also want to note that I have confirmed the data exists in the Content View:

[root@foreman ~]# ll /var/www/pub/yum/https/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CentOS/
8_-_AppStream/  8_-_BaseOS/     8_-_PowerTools/ listing         
[root@foreman ~]# ll /var/www/pub/yum/https/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CentOS/8_-_BaseOS/
Packages/ repodata/ 
[root@foreman ~]# ll /var/www/pub/yum/https/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CentOS/8_-_BaseOS/repodata/
0a459349b63fe6151b6f8e772992b57b4709ae7d8831286965390fa48c03e802-primary.xml.gz     925508479f430a33a16bb362b71d12b39566616853d2f435c026b3fcafccfc1a-updateinfo.xml.gz
1fee8e39b9e1a53ec29eaef4877c0eda56d833766879c796c1c2a57d2d5f6c24-comps.xml          eafcb0a64b23121a0c58962bd2b8882f200ebc8f4fa30628bd9e4afe605059da-other.xml.gz
5ac9c2865b7ad99d1c6f355d12761a099611d1dba4181c35a9e6382e4c1b712d-filelists.xml.gz   repomd.xml
[root@foreman ~]#

caitslinux · July 30, 2020, 6:12pm

I just re-re-published and promoted:

I verified the repos are present:

And the same result:

[root@centos8 ~]# subscription-manager refresh; subscription-manager repos
4 local certificates have been deleted.
All local data refreshed
+----------------------------------------------------------+
    Available Repositories in /etc/yum.repos.d/redhat.repo
+----------------------------------------------------------+
Repo ID:   CentOS_8_-_AppStream
Repo Name: 8 - AppStream
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CentOS/8_-_AppStream
Enabled:   1

Repo ID:   EPEL_EPEL8
Repo Name: EPEL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/EPEL/EPEL8
Enabled:   1

Repo ID:   OCS_Inventory_EL8
Repo Name: EL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/OCS_Inventory/EL8
Enabled:   1

Repo ID:   CrowdStrike_EL8
Repo Name: EL8
Repo URL:  https://foreman.server/pulp/repos/myorg/CentOS_8_Virtual_Servers/CentOS_8_Virtual_Servers/custom/CrowdStrike/EL8
Enabled:   1

[root@centos8 ~]#

caitslinux · July 30, 2020, 6:20pm

I should have titled this post better, but I was tired yesterday. The issue isn’t one repo per subscription, but only one repo per product is available. At least, that appears to be the behavior. I cannot see the other repos in other products (including CentOS). When I run subscription-manager --list only the repos shown above are displayed. --list-disabled results in an empty list.

caitslinux · August 3, 2020, 4:11pm

So, I installed updates today, and I noticed there was an update to Foreman. However, the issue still persists even with version 1.24.3.

[root@foreman ~]# hammer status
Version:           1.24.3
API Version:       v2
Database:          
    Status:          ok
    Server Response: Duration: 1ms
Plugins:           
 1) Name:    foreman-tasks
    Version: 0.17.5
 2) Name:    foreman_ansible
    Version: 4.0.5
 3) Name:    foreman_remote_execution
    Version: 2.0.8
 4) Name:    katello
    Version: 3.14.1
Smart Proxies:     
 1) Name:     foreman.server
    Version:  1.24.3
    Status:   ok
    Features: 
     1) Name:    pulp
        Version: 1.5.0
     2) Name:    templates
        Version: 1.24.3
     3) Name:    tftp
        Version: 1.24.3
     4) Name:    logs
        Version: 1.24.3
     5) Name:    httpboot
        Version: 1.24.3
Compute Resources: 

candlepin:         
    Status:          ok
    Server Response: Duration: 20ms
candlepin_auth:    
    Status:          ok
    Server Response: Duration: 19ms
pulp:              
    Status:          ok
    Server Response: Duration: 53ms
pulp_auth:         
    Status:          ok
    Server Response: Duration: 27ms
foreman_tasks:     
    Status:          ok
    Server Response: Duration: 4ms

[root@foreman ~]#

caitslinux · August 6, 2020, 5:37pm

So, I randomly came across this issue again … but it appears to be affecting a RHEL 7 Content Host. Except the issue is in reverse. I have a PostgreSQL Product with multiple Repos. The RHEL 7 Content Host (instead of only seeing one repo in a product) can see all BUT a few repos for the product. The PostgreSQL product has a repo for common, 9.4, 9.5, 9.6, 10, and 11. It can see 9.4, 9.5, 10, and common. I have confirmed that these repos ARE enabled for the Content View and that the content is actually there. The issue appears to be that, for whatever reason, subscription-manager is not receiving the correct entitlement data. This would suggest a deeper issue as CentOS 8 systems are having a similar issue.

caitslinux · August 6, 2020, 8:05pm

I’m wondering … should I maybe file a bug in the Foreman issue tracker about this? I don’t want to file a bug and bother others unless it is a problem worthy of reporting to the developers.

I’m just unsure of how to proceed from here. We have projects that are being put on hold because they can’t access the software they need.

gvde · August 7, 2020, 6:43am

I’d say you should upgrade to 3.15 and foreman 2.0.1. There were numerous issues with pulp, applicability, etc. even in the beginning in 3.15. I am not sure how many fixes make it into the older versions. 3.15/2.0.1 have a lot more recent version like pulp 2.0 etc., thus possibly your problems are already fixed in the latest version…

caitslinux · August 7, 2020, 3:20pm

Thanks, @gvde; I’ll investigate taking that path. I appreciate your insight!