Problem:
Recently upgraded from puppet 1.12 to 1.15.6 (though all of the major versions), was planning on upgrading to 1.16 however 1.16 requires puppet 4. 1.15.6 has been stable and running successfully with puppet 3 communicating to the hosts.
When migrating from puppet3 to 4 (following this: Upgrading from Puppet 3 to 4 - Foreman ) I am encountering an error with certificates.
I am able to log into foreman, confirm the smart_proxy is working but when hosts try to connect via puppet to puppetserver - they give the error:
# puppet agent --test
...
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: {"message":"Server Error: Failed when searching for node proxy02.my.domain: Failed to find proxy02.my.domain via exec: Execution of '/etc/puppetlabs/puppet/node.rb proxy02.my.domain' returned 1: ","issue_kind":"RUNTIME_ERROR","stacktrace":["Warning: The 'stacktrace' property is deprecated and will be removed in a future version of Puppet. For security reasons, stacktraces are not returned with Puppet HTTP Error responses."]}
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
When running the node.rb manually on the foreman/puppetmaster server:
# /etc/puppetlabs/puppet/node.rb proxy02.my.domain
During fact upload occured an exception: SSL_connect returned=1 errno=0 state=SSLv3 read server session ticket A: tlsv1 alert unknown ca
Unable to read from Cache file: No such file or directory - /opt/puppetlabs/server/data/puppetserver/yaml/foreman/proxy02.my.domain.yaml
It seems like there is something funny going on with the certs in the foreman.yaml file…
# cat /etc/puppetlabs/puppet/foreman.yaml | grep "^:ssl"
:ssl_ca: "/etc/pki/katello/puppet/puppet_client_ca.crt"
:ssl_cert: "/etc/pki/katello/puppet/puppet_client.crt"
:ssl_key: "/etc/pki/katello/puppet/puppet_client.key"
But the cert is signed with the CA listed…
# openssl x509 -in /etc/pki/katello/puppet/puppet_client.crt -noout -text | egrep "Subject:|Issuer:"
Issuer: C=US, ST=North Carolina, L=Raleigh, O=Katello, OU=SomeOrgUnit, CN=foreman.my.domain
Subject: C=US, ST=North Carolina, O=FOREMAN, OU=PUPPET, CN=foreman.my.domain
# openssl x509 -in /etc/pki/katello/puppet/puppet_client_ca.crt -noout -text | egrep "Subject:|Issuer:"
Issuer: C=US, ST=North Carolina, L=Raleigh, O=Katello, OU=SomeOrgUnit, CN=foreman.my.domain
Subject: C=US, ST=North Carolina, L=Raleigh, O=Katello, OU=SomeOrgUnit, CN=foreman.my.domain
Foreman -> Administer -> About also shows the same error from the pulp_auth backend status:
| Component | Status | Message |
|---|---|---|
| candlepin | OK | |
| candlepin_auth | OK | |
| foreman_tasks | OK | |
| pulp | OK | |
| pulp_auth | FAIL | SSL_connect returned=1 errno=0 state=SSLv3 read server session ticket A: tlsv1 alert unknown ca |
Not sure where to start… I’ve restored back and tried the upgrade a couple of times - still getting the same result.
Expected outcome:
Hosts would be able to connect to the puppetserver
Foreman and Proxy versions:
Foreman 1.15.6
Foreman Proxy 1.15.6