Newly provisioned machines not doing Puppet runs


#1

Problem:
On my fresh 1.22.0 installation, newly provisioned machines successfully install but never generate a report and don’t appear to be doing Puppet runs successfully.

When running the agent manually on those machines, we see this error:

[root@demo7 ~]# puppet agent -t
Warning: Unable to fetch my node definition, but the agent run will continue:
Warning: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]
Info: Retrieving pluginfacts
Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]
Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not retrieve file metadata for puppet://fm-master3.riff.cc/pluginfacts: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]
Wrapped exception:
SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]
Info: Retrieving plugin
Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]
Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve file metadata for puppet://fm-master3.riff.cc/plugins: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]
Wrapped exception:
SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]
Error: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet Root CA: a3558aa6a0a245]

Expected outcome:
Puppet should run without issue.

Foreman and Proxy versions:
1.22.0

Foreman and Proxy plugin versions:
1.22.0


#2

This is on CentOS 7, fully patched, firewall off (for testing only).


#3

Can you tell me which Puppetserver version is installed on the master and Puppet version on the systems?

I think I saw similar errors when Puppet 3.7 from CentOS repositories was still used with Puppetserver 5 from Puppet repositories on the master. If this is the case you can install Puppet 5 during provisioning by setting the parameter enable-puppetlabs-puppet5-repo to true.


#4

On the master:

[wings@fm-master3 ~]$ puppetserver --version
puppetserver version: 6.4.0

On the client:

[root@demo7 ~]# puppet --version
3.6.2

Very weird. Is there a way to enable Puppet6 provisioning on these machines?


#5

At the moment no, but he templates could be easily extended similar to Puppet 5:




#6

I gathered some more info. My Ubuntu client to that server (which gives the same errors as in my first post):

root@demo8:~# puppet --version
5.4.0

Then my old master running 1.20.2:

wings@fm-master2:~$ puppetserver --version
puppetserver version: 5.3.8

And its CentOS client:

[root@demo1 ~]# puppet --version
3.6.2

And its Ubuntu client:

root@demo2:~# puppet --version
5.4.0

On my 1.20.2 master, things work without issue including reporting.


#7

Still having issues… 1.20.2 works perfectly, unfortunately this is our last major blocker before the next stage…


#8

I went ahead and deployed a fresh 1.22.0 master, but installed Puppet 5 instead of Puppet 6 at the very start. Doing this, it worked perfectly (nodes successfully checking in and completing runs) so it appears there’s been a regression with the Puppet 6 master.

I’ve got both working and ready for testing, so if there’s anything I can do to help get this fixed let me know. This does buy some time though, as we were likely to stick with Puppet 5 for at least a couple more months.


#9

I had a stab at this, but haven’t tested it yet. Would be great if anyone could have a look.