https://<HOST>/pulp/repos/<ORG>/<ENV>/<CV>/content/dist/rhel/server/7/7Server/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found

Have 3.6.0.1.rc2 installed and configured with Red Hat subscriptions. All working fine except of below issue. I understand Katello content management subsystem very well - i.e. all necessary objects are created and setup. I can subscribe target systems to repositories.

Problem:
yum commands repolist|install|update throws below error message. This is the only repo I have troubles with but that’s the most important one. This happens for all content views

https:///pulp/repos////content/dist/rhel/server/7/7Server/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found

Expected outcome:
yum work normally

Foreman and Proxy versions: 1.17.0-RC2 / 3.6.0.1.rc2

Foreman and Proxy plugin versions: 1.17.0-RC2

Other relevant data:

If that’s an actual RHEL REPO, you can’t access it without loading the Katello TLS certificate into your browser if you’re trying to get to by browsing.

As to a system trying to yum install from it, you should be able to go into content->products->Redhat Enterprise Linux Server->Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server

You should see a “Published At:” URL (which LOOKS right above in your case). Make sure that matches what’s in your /etc/yum.repos.d file.

Other than that, do you show Packages / Eratta / etc. in that archive? It’s been synced already?

I know this is just general stuff, but sometimes it’s the silly stuff that gets you.

No it’s not about my browser - it’s the yum that fails to get metadata for RHEL7 repo (and ONLY for it) after a successful registration. All repos setup in my Katello are successfully synced, have errata. If disable this problematic RHEL7 repo in yum (config or cmdline) - I can install packages from any other Katello manged repos without any issues.

root@host:~/bin # subscription-manager list

+-------------------------------------------+
    Installed Product Status
+-------------------------------------------+
Product Name:   Red Hat Software Collections (for RHEL Server)
Product ID:     201
Version:        3
Arch:           x86_64
Status:         Subscribed
Status Details:
Starts:         07/11/2017
Ends:           07/10/2018

Product Name:   Red Hat Enterprise Linux Server
Product ID:     69
Version:        7.4
Arch:           x86_64
Status:         Subscribed
Status Details:
Starts:         07/11/2017
Ends:           07/10/2018

root@host:~/bin # subscription-manager status
+-------------------------------------------+
   System Status Details
+-------------------------------------------+
Overall Status: Current

root@host:~/bin # yum repolist
Loaded plugins: enabled_repos_upload, package_upload, product-id, search-disabled-repos, subscription-manager, tracer_upload
https://host/pulp/repos/ORGANIZATION/PROD/PROD_INFRA_RHEL7_SERVER/content/dist/rhel/server/7/7Server/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below knowledge base article

https://access.redhat.com/articles/1320623

If above article doesn't help to resolve this issue please open a ticket with Red Hat Support.

https://host/pulp/repos/ORGANIZATION/PROD/PROD_INFRA_RHEL7_SERVER/content/dist/rhel/server/7/7Server/x86_64/os/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
Trying other mirror.
repo id                                           repo name                                                                status
centos-sclo-rh/x86_64                             CentOS-7 - SCLo rh                                                        6,853
centos-sclo-sclo/x86_64                           CentOS-7 - SCLo sclo                                                        510
epel/x86_64                                       Extra Packages for Enterprise Linux 7 - x86_64                           12,439
foreman/x86_64                                    Foreman 1.17                                                                453
foreman-plugins/x86_64                            Foreman plugins 1.17                                                        253
group_theforeman-tfm-ror51/x86_64                 Copr repo for tfm-ror51 owned by @theforeman                                325
katello/x86_64                                    Katello 3.6                                                                  33
katello-candlepin/x86_64                          Candlepin: an open source entitlement management system.                      3
katello-client/x86_64                             Katello Client 3.6                                                           17
pulp/7Server/x86_64                               Pulp Community Release                                                       59
puppetlabs-deps/x86_64                            Puppet Labs Dependencies El 7 - x86_64                                       17
puppetlabs-pc1/x86_64                             Puppet Labs PC1 Repository el 7 - x86_64                                    138
puppetlabs-products/x86_64                        Puppet Labs Products El 7 - x86_64                                          225
rhel-7-server-rpms/7Server/x86_64                 Red Hat Enterprise Linux 7 Server (RPMs)                                      0
sensu/7Server/x86_64                              sensu                                                                       103
repolist: 21,428
Uploading Enabled Reposistories Report
Loaded plugins: product-id, subscription-manager

Yea, mostly what I was trying to validate was that the url line in your /etc/yum.repos.d/redhat.repo matched the “Published At” line in Katello’s Products section for that repository.

You could use the web interface to attempt to diagnose this: If you load the certificate from the Products section into your web browser you should be able to browse to that archive (Published At) through a browser to see if you get nothing returned or something returned.

On your Katello Master Host, you should also be able to go to:

/var/lib/pulp/published/yum/https/repos/< ORGANIZATION >/Library/< CONTENT VIEW >/content/dist/rhel/server/7/7Server/x86_64

Under that, you should see an “os” symbolic link which if followed should take you to the RHEL7 Packages and Repodata directory. Is there anything under that?

Well… the long story short.

When you setup your RH repos in Katello - always set checkboxes for all software releases you need (i.e. 7.3, 7.4) AND ALSO for $releasever for you RH OS. (see below)

RedHat (at least 7.x) resolves $releasever yum variable to “7Server”. Once repo(s) synced in Katello - don’t forget add them to your content views, publish new versions and promote to appropriate lifecycle environment(s).

Below command will print you the yum vars:

python -c ‘import yum, pprint; yb = yum.YumBase(); pprint.pprint(yb.conf.yumvar, width=1)’

Useful links:
https://access.redhat.com/solutions/1256473
https://unix.stackexchange.com/questions/19701/yum-how-can-i-view-variables-like-releasever-basearch-yum0
https://access.redhat.com/solutions/2802131