Digging a little deeper: I have checked the entitlement certificate on the clients for our private ORG product. This is how it looks like after the forced refresh on the clients where it’s already missing:
# rct cat-cert /etc/pki/entitlement/7348114868866713206.pem
+-------------------------------------------+
Entitlement Certificate
+-------------------------------------------+
Certificate:
Path: /etc/pki/entitlement/7348114868866713206.pem
Version: 3.4
Serial: 7348114868866713206
Start Date: 2020-07-03 14:26:57+00:00
End Date: 2049-12-01 00:00:00+00:00
Pool ID: 082c9981730a593d01731512c0320072
Subject:
CN: 98aa2d6239cb488b9d3bf6fd06b3b15f
O: ORG
Issuer:
C: DE
CN: foreman.example.com
...
Product:
ID: 963877616277
Name: ORG
Version:
Arch: ALL
Tags:
Brand Type:
Brand Name:
Order:
Name: ORG
Number:
SKU: 963877616277
Contract:
Account:
Service Type:
Roles:
Service Level:
Usage:
Add-ons:
Quantity: Unlimited
Quantity Used: 1
Socket Limit:
RAM Limit:
Core Limit:
Virt Only: False
Stacking ID:
Warning Period: 0
Provides Management: False
(Yes the name of the product is our ORG name). It’s missing the Authorized Content URLs
and Content
sections which I can see on the clients which still have the el8 repository.
I have enabled debug logging in candlepin on my main foreman server and capture the logs during creating of that entitlement certificate:
2022-06-02 16:52:01,278 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.audit.EventSinkImpl - Queuing event: Event [id: null, target: ENTITLEMENT, type: MODIFIED, time: 2022-06-02 16:52:01+0200, entity: 43c37940342e4ba69402e6a331151ffd]
2022-06-02 16:52:01,279 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Generating entitlement cert for entitlements
2022-06-02 16:52:01,280 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Persisting new certificate serials
2022-06-02 16:52:01,281 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] INFO org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Generating entitlement cert for pool: Pool [id: 082c9981730a593d01731512c0320072, type: NORMAL, product: 963877616277, productName: ORG, quantity: -1] quantity: 1 entitlement id: 98aa2d6239cb488b9d3bf6fd06b3b15f
2022-06-02 16:52:01,281 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Consumer has an environment; checking for promoted content in: Environment [id: 1809b04b0444be4df8aa435dd54cc11a, name: Production/el8-epel8, owner: Owner [id: 082c9981720833470172083368ac0001, key: ORG]]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cbbd01ff, id: 1589352123313, name: PostgreSQL 12 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg12-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cc25021f, id: 1589352127979, name: PostgreSQL 10 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg10-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c998180d04c0e0180e250083d019a, id: 1653064272568, name: AlmaLinux-8 - Real Time x86_64 os, label: ORG_almalinux8_plus_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f734f35017fbffed9e502e6, id: 1623228212554, name: CentOS-8 Stream - extras x86_64, label: ORG_centos8stream_extras_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cbf3020f, id: 1589352125594, name: PostgreSQL 11 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg11-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818069662a018074b7bed30320, id: 1589353644182, name: Extra Packages for Enterprise Linux Modular 8 - x86_64, label: ORG_epel8_epel-modular]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cfa2029a, id: 1592404126102, name: Backup Archiving Recovery Open Sourced 19.2 (CentOS_8), label: ORG_bareos_bareos-com_bareos-19_2-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c9981806407dd018065af8c8f04ab, id: 1625663429486, name: AlmaLinux-8 - AppStream x86_64 os, label: ORG_almalinux8_AppStream_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c6700108, id: 1623228210938, name: CentOS-8 Stream - Plus x86_64, label: ORG_centos8stream_centosplus_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6ca7f019f, id: 1589351695000, name: PostgreSQL common RPMs for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg-common-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c61700f8, id: 1623228205631, name: CentOS-8 Stream - AppStream x86_64, label: ORG_centos8stream_AppStream_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c69d0110, id: 1623228208320, name: CentOS-8 Stream - HighAvailability x86_64, label: ORG_centos8stream_HighAvailability_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cfd402a0, id: 1591882452614, name: Elasticsearch repository for 7.x packages, label: ORG_elastic_elasticsearch-7-x]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f734f35017fbffeb96002dd, id: 1648193484725, name: CentOS-8 Stream - extras common x86_64, label: ORG_centos8stream_extras_common_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80d3c4017a81176b5603f4, id: 1625663433430, name: AlmaLinux-8 - HighAvailability x86_64 os, label: ORG_almalinux8_HighAvailability_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80d3c4017a81177a8d0410, id: 1625663437330, name: AlmaLinux-8 - extras x86_64 os, label: ORG_almalinux8_extras_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817eca8d18017edd8317f400b1, id: 1644393805078, name: EPEL-Next 8 - x86_64, label: ORG_epel8_next]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f734f35017f968b7960016e, id: 1647498197222, name: PostgreSQL Supplementary ucommon RPMs for RHEL / Rocky 8 - x86_64, label: ORG_postgresql_pgdg-centos8-sysupdates]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818069662a018073ad42830094, id: 1650558752470, name: Foreman client 3.2 el8, label: ORG_foreman-client_3_2_el8_x86_64]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c6430100, id: 1623228209551, name: CentOS-8 Stream - PowerTools x86_64, label: ORG_centos8stream_PowerTools_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817eca8d18017f0848b1d30339, id: 1645111456085, name: CentOS 8 Stream - Ceph Pacific, label: ORG_centos8-stream-storage_ceph-pacific]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c2280083, id: 1601011894902, name: PostgreSQL 13 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg13-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f563e45017f64dee8cf0081, id: 1593787688196, name: ORG el8, label: ORG_org_el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817c362875017c3b4cf0da0066, id: 1633082470381, name: PostgreSQL 14 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg14-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c0bf0052, id: 1623308982040, name: stable EL8, label: ORG_docker-ce_stable_el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6d07802ae, id: 1592822449058, name: Shibboleth (CentOS_8), label: ORG_shibboleth_shibboleth-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c076004e, id: 1604314814110, name: Elasticsearch repository for 6.x packages, label: ORG_elastic_elasticsearch-6-x]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817eca8d18017f0c5fbe44038b, id: 1645180075449, name: CentOS 8 Stream - Ceph Quincy, label: ORG_centos8-stream-storage_ceph-quincy]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817c5e7528017c5e9725b80007, id: 1589353598586, name: Extra Packages for Enterprise Linux 8 - x86_64, label: ORG_epel8_epel]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817d32b20e017d33b4c325005f, id: 1637250024076, name: Foreman client 3.1 el8, label: ORG_foreman-client_3_1_el8_x86_64]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c998180d04c0e0180e24fbd530184, id: 1653064276294, name: AlmaLinux-8 - Plus x86_64 os, label: ORG_almalinux8_RT_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818069662a018073b029f700fa, id: 1650557669831, name: Puppet 7 Repository el 8 - x86_64, label: ORG_puppet_puppet7_el8_x86_64]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f734f35017f9688ff740122, id: 1647497907514, name: PostgreSQL extra packages to support some RPMs in the PostgreSQL RPM repo RHEL / Rocky 8 - x86_64, label: ORG_postgresql_pgdg-rhel8-extras]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818045e635018061a41de91538, id: 1589784079717, name: Puppet 6 Repository el 8 - x86_64, label: ORG_puppet_puppet6_el8_x86_64]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80d3c4017a811763c803e9, id: 1625663431505, name: AlmaLinux-8 - BaseOS x86_64 os, label: ORG_almalinux8_BaseOS_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80d3c4017a811773390401, id: 1625663435428, name: AlmaLinux-8 - PowerTools x86_64 os, label: ORG_almalinux8_PowerTools_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c998180d04c0e0180e2502b6f01a5, id: 1653064274495, name: AlmaLinux-8 - ResilientStorage x86_64 os, label: ORG_almalinux8_ResilientStorage_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c5e800f0, id: 1623228207122, name: CentOS-8 Stream - BaseOS x86_64, label: ORG_centos8stream_BaseOS_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c998180d04c0e0180e24fecab018f, id: 1653064270559, name: AlmaLinux-8 - Real Time for NFV x86_64 os, label: ORG_almalinux8_NFV_x86_64_os]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c4b400d6, id: 1621248407297, name: Backup Archiving Recovery Open Sourced 20 (CentOS 8), label: ORG_bareos_bareos-com_bareos_20-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818069662a018073af932000e9, id: 1650558367546, name: CentOS Configmanagement SIG - ansible-29 el8, label: ORG_centos-configmanagement_ansible_29_el8_x86_64]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cf770296, id: 1589982020034, name: ICINGA (stable release for epel) EL8, label: ORG_Icinga_icinga-stable-release-el8]
2022-06-02 16:52:01,282 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] INFO org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Creating X509 cert for product: Product [uuid: 082c99817f563e45017f64dee9240083, id: 963877616277, name: ORG]
2022-06-02 16:52:01,283 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Provided products: [Product [uuid: 082c99817f563e45017f64dee9240083, id: 963877616277, name: ORG]]
2022-06-02 16:52:01,283 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.util.X509Util - Skipping content not promoted to environment: Content [uuid: 082c99817f563e45017f64dee8cf0080, id: 1593787668737, name: ORG el7, label: ORG_org_el7]
2022-06-02 16:52:01,283 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Consumer has an environment; checking for promoted content in: Environment [id: 1809b04b0444be4df8aa435dd54cc11a, name: Production/el8-epel8, owner: Owner [id: 082c9981720833470172083368ac0001, key: ORG]]
2022-06-02 16:52:01,283 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cbbd01ff, id: 1589352123313, name: PostgreSQL 12 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg12-el8]
2022-06-02 16:52:01,283 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cc25021f, id: 1589352127979, name: PostgreSQL 10 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg10-el8]
2022-06-02 16:52:01,283 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c998180d04c0e0180e250083d019a, id: 1653064272568, name: AlmaLinux-8 - Real Time x86_64 os, label: ORG_almalinux8_plus_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f734f35017fbffed9e502e6, id: 1623228212554, name: CentOS-8 Stream - extras x86_64, label: ORG_centos8stream_extras_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cbf3020f, id: 1589352125594, name: PostgreSQL 11 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg11-el8]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818069662a018074b7bed30320, id: 1589353644182, name: Extra Packages for Enterprise Linux Modular 8 - x86_64, label: ORG_epel8_epel-modular]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cfa2029a, id: 1592404126102, name: Backup Archiving Recovery Open Sourced 19.2 (CentOS_8), label: ORG_bareos_bareos-com_bareos-19_2-el8]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c9981806407dd018065af8c8f04ab, id: 1625663429486, name: AlmaLinux-8 - AppStream x86_64 os, label: ORG_almalinux8_AppStream_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c6700108, id: 1623228210938, name: CentOS-8 Stream - Plus x86_64, label: ORG_centos8stream_centosplus_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6ca7f019f, id: 1589351695000, name: PostgreSQL common RPMs for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg-common-el8]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c61700f8, id: 1623228205631, name: CentOS-8 Stream - AppStream x86_64, label: ORG_centos8stream_AppStream_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c69d0110, id: 1623228208320, name: CentOS-8 Stream - HighAvailability x86_64, label: ORG_centos8stream_HighAvailability_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cfd402a0, id: 1591882452614, name: Elasticsearch repository for 7.x packages, label: ORG_elastic_elasticsearch-7-x]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f734f35017fbffeb96002dd, id: 1648193484725, name: CentOS-8 Stream - extras common x86_64, label: ORG_centos8stream_extras_common_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80d3c4017a81176b5603f4, id: 1625663433430, name: AlmaLinux-8 - HighAvailability x86_64 os, label: ORG_almalinux8_HighAvailability_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80d3c4017a81177a8d0410, id: 1625663437330, name: AlmaLinux-8 - extras x86_64 os, label: ORG_almalinux8_extras_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817eca8d18017edd8317f400b1, id: 1644393805078, name: EPEL-Next 8 - x86_64, label: ORG_epel8_next]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f734f35017f968b7960016e, id: 1647498197222, name: PostgreSQL Supplementary ucommon RPMs for RHEL / Rocky 8 - x86_64, label: ORG_postgresql_pgdg-centos8-sysupdates]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818069662a018073ad42830094, id: 1650558752470, name: Foreman client 3.2 el8, label: ORG_foreman-client_3_2_el8_x86_64]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c6430100, id: 1623228209551, name: CentOS-8 Stream - PowerTools x86_64, label: ORG_centos8stream_PowerTools_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817eca8d18017f0848b1d30339, id: 1645111456085, name: CentOS 8 Stream - Ceph Pacific, label: ORG_centos8-stream-storage_ceph-pacific]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c2280083, id: 1601011894902, name: PostgreSQL 13 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg13-el8]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f563e45017f64dee8cf0081, id: 1593787688196, name: ORG el8, label: ORG_org_el8]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817c362875017c3b4cf0da0066, id: 1633082470381, name: PostgreSQL 14 for RHEL/CentOS 8 - x86_64, label: ORG_postgresql_pgdg14-el8]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c0bf0052, id: 1623308982040, name: stable EL8, label: ORG_docker-ce_stable_el8]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6d07802ae, id: 1592822449058, name: Shibboleth (CentOS_8), label: ORG_shibboleth_shibboleth-el8]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c076004e, id: 1604314814110, name: Elasticsearch repository for 6.x packages, label: ORG_elastic_elasticsearch-6-x]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817eca8d18017f0c5fbe44038b, id: 1645180075449, name: CentOS 8 Stream - Ceph Quincy, label: ORG_centos8-stream-storage_ceph-quincy]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817c5e7528017c5e9725b80007, id: 1589353598586, name: Extra Packages for Enterprise Linux 8 - x86_64, label: ORG_epel8_epel]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817d32b20e017d33b4c325005f, id: 1637250024076, name: Foreman client 3.1 el8, label: ORG_foreman-client_3_1_el8_x86_64]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c998180d04c0e0180e24fbd530184, id: 1653064276294, name: AlmaLinux-8 - Plus x86_64 os, label: ORG_almalinux8_RT_x86_64_os]
2022-06-02 16:52:01,284 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818069662a018073b029f700fa, id: 1650557669831, name: Puppet 7 Repository el 8 - x86_64, label: ORG_puppet_puppet7_el8_x86_64]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817f734f35017f9688ff740122, id: 1647497907514, name: PostgreSQL extra packages to support some RPMs in the PostgreSQL RPM repo RHEL / Rocky 8 - x86_64, label: ORG_postgresql_pgdg-rhel8-extras]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818045e635018061a41de91538, id: 1589784079717, name: Puppet 6 Repository el 8 - x86_64, label: ORG_puppet_puppet6_el8_x86_64]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80d3c4017a811763c803e9, id: 1625663431505, name: AlmaLinux-8 - BaseOS x86_64 os, label: ORG_almalinux8_BaseOS_x86_64_os]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80d3c4017a811773390401, id: 1625663435428, name: AlmaLinux-8 - PowerTools x86_64 os, label: ORG_almalinux8_PowerTools_x86_64_os]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c998180d04c0e0180e2502b6f01a5, id: 1653064274495, name: AlmaLinux-8 - ResilientStorage x86_64 os, label: ORG_almalinux8_ResilientStorage_x86_64_os]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c5e800f0, id: 1623228207122, name: CentOS-8 Stream - BaseOS x86_64, label: ORG_centos8stream_BaseOS_x86_64_os]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c998180d04c0e0180e24fecab018f, id: 1653064270559, name: AlmaLinux-8 - Real Time for NFV x86_64 os, label: ORG_almalinux8_NFV_x86_64_os]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6c4b400d6, id: 1621248407297, name: Backup Archiving Recovery Open Sourced 20 (CentOS 8), label: ORG_bareos_bareos-com_bareos_20-el8]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99818069662a018073af932000e9, id: 1650558367546, name: CentOS Configmanagement SIG - ansible-29 el8, label: ORG_centos-configmanagement_ansible_29_el8_x86_64]
2022-06-02 16:52:01,285 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - promoted content: Content [uuid: 082c99817a80bc55017a80c6cf770296, id: 1589982020034, name: ICINGA (stable release for epel) EL8, label: ORG_Icinga_icinga-stable-release-el8]
2022-06-02 16:52:01,288 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Getting PEM encoded cert.
2022-06-02 16:52:01,288 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Generating v3 entitlement data
2022-06-02 16:52:01,291 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Generated cert serial number: 7348114868866713206
2022-06-02 16:52:01,291 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Key: -----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
2022-06-02 16:52:01,291 [thread=http-bio-127.0.0.1-23443-exec-6] [req=5c7a9bf4-4011-404d-b132-4659d6818e7d, org=ORG, csid=] DEBUG org.candlepin.service.impl.DefaultEntitlementCertServiceAdapter - Cert: -----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN ENTITLEMENT DATA-----
...
-----END ENTITLEMENT DATA-----
-----BEGIN RSA SIGNATURE-----
...
-----END RSA SIGNATURE-----
Which all looks good, too. It lists all repositories in the content view of the client including the missing repo:
promoted content: Content [uuid: 082c99817f563e45017f64dee8cf0081, id: 1593787688196, name: ORG el8, label: ORG_org_el8]
It skips the el7 repository which is belongs to the ORG product but is not in the content view:
Skipping content not promoted to environment: Content [uuid: 082c99817f563e45017f64dee8cf0080, id: 1593787668737, name: ORG el7, label: ORG_org_el7]
So those logs look good and I can see that logs follow the same logic for the other subscribed products of the client.
So question is: why doesn’t candlepin add the ORG_org_el8 repository to the entitlement certificate? And why now?