Certificate verify failed - self signed certificate in certificate chain for CN=Puppet Root CA: xxxxxx

Support
1

Problem:
Problem 1:
When trying to register a host, with Rocky Linyx 9.4, the prompt gets “stuck” at the stage below:

#
# Running registration
#
This system is currently not registered.
All local data removed

Problem 2:
When trying to manually apply the puppet agent through the command sudo puppet agent -tv --debug the prompt gets “stuck” at the stage below:

Debug: HTTP GET https://xxx.xxx.xxx:8140/puppet/v3/file_metadata/modules/basico_rocky/conf/ntp_server/chrony.conf_client?links=manage&checksum_type=sha256&source_permissions=ignore&environment=production returned 200 OK
Debug: Caching connection for https://xxx.xxx.xxx:8140
Debug: Executing: '/usr/bin/systemctl is-active -- chronyd'
Debug: Executing: '/usr/bin/systemctl is-enabled -- chronyd'
Debug: Prefetching yum resources for package
Debug: Executing: '/usr/bin/rpm --version'
Debug: Executing '/usr/bin/rpm -qa --nosignature --nodigest --qf '%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n' | sort'
Debug: Executing: '/usr/bin/yum check-update'

Problem 3:
When trying to list the repositories, the command gets “stuck” at the stage below:
$ sudo subscription-manager repos --list

Expected result:
I hope the command sudo puppet agent -tv can be executed successfully, applying the puppet manifests.
I hope the registration of new hosts can work correctly and subscription-manager works as expected.
Foreman and Proxy versions:
Foreman-3.10.1-1.el8.noarch
Foreman-proxy-3.10.1-1.el8.noarch
katello-4.12.1-1.el8.noarch
puppetserver version: 7.17.3
puppet agent version: 7.34.0
Foreman and Proxy plugin versions:
Foreman-3.10.1-1.el8.noarch
Foreman-proxy-3.10.1-1.el8.noarch
katello-4.12.1-1.el8.noarch
puppetserver version: 7.17.3
puppet agent version: 7.34.0
Distribution and version:
foreman-3.10.1-1.el8.noarch
foreman-proxy-3.10.1-1.el8.noarch
katello-4.12.1-1.el8.noarch
puppetserver version: 7.17.3
puppet agent version: 7.34.0
Other relevant data:
Rocky Linux release 8.10 (Green Obsidian)
Forem and Foreman-proxy run on the same server.

Evidence 1

On a Rocky linux 9.3 or 9.4 host, with puppet agent installed, I have the log below.
puppet-agent[1893909]: certificate verify failed [self signed certificate in certificate chain for CN=Puppet Root CA: abfba0e7744bxxx]
puppet agent service is down.
the host has connectivity to the puppet server and foreman server (services that are running on the same server).

Evidence 2

● puppet.service - Puppet agent
Loaded: loaded (/usr/lib/systemd/system/puppet.service; enabled; preset: disabled)
Active: active (running) since Wed 2024-11-27 11:23:31 -03; 2min 7s ago
 Docs: man:puppet-agent(8)
 Main PID: 16055 (puppet)
 Tasks: 7 (limit: 22958)
 Memory: 409.9M
 CPU: 5,086s
 CGroup: /system.slice/puppet.service
 ├─ 5987 "puppet agent: applying configuration"
 ├─ 6180 /usr/bin/python3.9 /usr/bin/dnf check-update
 ├─16055 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent --no-daemonize
 ├─16056 "puppet agent: applying configuration"
 └─16192 /usr/bin/python3.9 /usr/bin/yum check-update

Nov 27 11:23:31 xxx.xxx.xxx systemd[1]: Started Puppet agent.
Nov 27 11:23:33 xxx.xxx.xxx puppet-agent[16055]: Starting Puppet client version 7.31.0
Nov 27 11:23:45 xxx.xxx.xxx puppet-agent[16056]: Requesting catalog from xxx.xxx.xxx:8140 (xxx.xxx.0.99)
Nov 27 11:23:51 xxx.xxx.xxx puppet-agent[16056]: Catalog compiled by xxx.xxx.xxx

Some tests carried out:
Verify the certificate chain

sudo puppetserver ca list --all
sudo puppetserver ca revoke --certname <FQDN_DO_HOST>
sudo puppetserver ca clean --certname <FQDN_DO_HOST>

Clear certificates on the host
sudo rm -rf /etc/puppetlabs/puppet/ssl
Request a new certificate
sudo puppet agent -t

2

You have to make sure registration works first. All the other tests and things you have done after that are futile. It only says that the host isn’t properly registered.

Run the registration script with bash -x and check where exactly it’s “stuck”? It tells you the commands executed. What happens if you wait?

3

Thanks for the reply and sorry for the delay.
Here we go, after a little more investigation I found out what caused the problem:
puppet-agent[1893909]: certificate verify failed [self signed certificate in certificate chain for CN=Puppet Root CA: abfba0e7744bxxx].
See below how I solved it.
I just didn’t find out what caused the change in the /etc/rhsm/rhsm.conf file
Solution:

1. For some reason the `/etc/rhsm/rhsm.conf` file was changed:
In the [rhsm] session there was the configuration: repo_ca_cert = %(ca_cert_dir)skatello-server-ca.pem
It was adjusted to: repo_ca_cert = /etc/rhsm/ca/katello-server-ca.pem

2. Existence of "zombie" processes for yum and dnf
To find the PID: ps -el | grep defunct
To kill the PPID: sudo kill -9 <PPID number>
4

Well the ca_cert_dir variable should be set right before that line:

# Server CA certificate location:
ca_cert_dir = /etc/rhsm/ca/

# Default CA cert to use when generating yum repo configs:
repo_ca_cert = %(ca_cert_dir)skatello-server-ca.pem

That’s how it is on my clients and that works just fine.

1 Like