Problem:
The original Puppet CA server has been upgraded many times over the years and while updating to Forman 1.24 and Puppet 6 it has changed to being a DHCP server. It has never been this role before and I cannot not get it to re-enable back to its previous role.
Foreman and Proxy versions:
Foreman 1.24
Distribution and version:
CentOS 7
Other relevant data:
I have been trying to create a new replacement Puppet CA server, however; it will not register as a proxy to the Foreman servers.
Is there a process to be able to switch out Puppet CA servers or has anyone done this before?
Currently the whole environment is not able to run with this server down. Been working this for nearly 24 hours so my mind is pretty much gone at this point.
Thank you in advance for any advice.
Cassus
How did you upgrade the server and from what version? What error is the new CA server showing when you are trying to register it to foreman and how are you registering it?
@tbrisker the issues with the original might have been created by myself. I believe it was created about 3 years as version 1.17 using the foreman-installer. Over the years another team supported and upgraded it 1.21. I upgraded 3 Foreman servers, 1 PuppetDB, 1 Memache, and 6 Puppet masters to 1.24. Of course I upgraded to 1.22 and then 1.23 first, but these systems were not built using the foreman-installer so I did not do the documented foreman-installer process on them or the Puppet CA server. I did run into one issue with the PaaS Postgres database once upgrading from Puppet 5 to version 6, but I was able to get that resolved. It seems the trouble started once I upgraded to Puppet 6 on CA server. It’s only purpose was for the certificates, but role got changed to just DCHP for some reason after the Puppet update.
The new server was receiving a SSL message when trying to register to Foreman. I don’t have the message with me since I am going to rebuild that new machine and try again. If I receive the same issue I will post the error.
I’m actually okay with replacing the old machine with a new one since it will be built more to our standards we use for the rest of this environment even if I need to regenerate certs for the servers connecting through Puppet.
I know there wasn’t a lot of details there, but hoping with a little more rest I can be more successful today.
Thanks for your response.
Cassus
Here is some output from the older Puppet CA server when I try to disable the DHCP and enable Puppet CA. I also receive the same error when trying to disable the compute options as well since I don’t need those either.
foreman-installer -v \
–foreman-proxy-dhcp false
–foreman-proxy-dhcp-managed false
–foreman-proxy-puppetca true
–foreman-proxy-puppetca-cmd “/opt/puppetlabs/bin/puppetserver ca”
[ INFO 2020-04-26T16:15:16 verbose] Executing hooks in group pre_migrations
[ INFO 2020-04-26T16:15:16 verbose] All hooks in group pre_migrations finished
[ INFO 2020-04-26T16:15:16 verbose] Executing hooks in group boot
[ INFO 2020-04-26T16:15:16 verbose] All hooks in group boot finished
[ INFO 2020-04-26T16:15:16 verbose] Executing hooks in group init
[ INFO 2020-04-26T16:15:16 verbose] All hooks in group init finished
[ INFO 2020-04-26T16:15:16 verbose] Loading default values from puppet modules…
[ INFO 2020-04-26T16:15:16 verbose] … finished
[ INFO 2020-04-26T16:15:16 verbose] Executing hooks in group pre_values
[ INFO 2020-04-26T16:15:16 verbose] All hooks in group pre_values finished
[ INFO 2020-04-26T16:15:16 verbose] Running installer with args [[“-v”, “–foreman-proxy-dhcp”, “false”, “–foreman-proxy-dhcp-managed”, “false”, “–foreman-proxy-puppetca”, “true”, “–foreman-proxy-puppetca-cmd”, “/opt/puppetlabs/bin/puppetserver ca”]]
[ INFO 2020-04-26T16:15:16 verbose] Executing hooks in group pre_validations
[ INFO 2020-04-26T16:15:16 verbose] All hooks in group pre_validations finished
[ INFO 2020-04-26T16:15:16 verbose] Running validation checks
[ERROR 2020-04-26T16:15:16 verbose] Parameter foreman-compute-ec2-version invalid: nil is not a valid string
[ERROR 2020-04-26T16:15:16 verbose] Parameter foreman-compute-vmware-version invalid: nil is not a valid string
Error during configuration, exiting
[ INFO 2020-04-26T16:15:16 verbose] Installer finished in 2.296453525 seconds
Another thing in the mix so that is originally deployed at a RHEL 7 system which was converted to CentOS 7 earlier this year which is one reason I’d like to just build a fresh Puppet CA replacement that starts as CentOS 7.
Running the foreman-installer with several options, but receiving this error during the run while foreman-proxy is trying to start.
/etc/puppetlabs/puppet/ssl/private_keys/{hostname.domain}.pem
Should this certificate be created during the installation process?
Running the installer with these options:
foreman-installer -v
–no-enable-foreman
–no-enable-foreman-cli
–no-enable-puppet
–foreman-configure-epel-repo false
–foreman-configure-scl-repo false
–foreman-db-manage false
–foreman-foreman-url “https://{foremanfqdn}”
–foreman-oauth-consumer-key “{oath-key}”
–foreman-oauth-consumer-secret “{oath-secret}”
–foreman-proxy-dhcp false
–foreman-proxy-dns false
–foreman-proxy-puppet false
–foreman-proxy-puppetca true
–foreman-proxy-tftp false
–foreman-proxy-trusted-hosts [list-of-trusted-servers]
Thanks,
Cassius
It seems my copy and paste didn’t work correctly on the above message.
Error receiving:
No such file or directory - /etc/puppetlabs/puppet/ssl/private_keys/{hostname.domain}.pem
Getting a little further and I’m thinking that I will need to create a new private key and certificate that will go on all my Foreman and Puppet Masters that is issued by my new CA server. I have not done this before so any advice is much appreciated or please let me know if I’m not on the right path.
During foreman-installer:
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[hostname.domain]: Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate) in get request to: https://foreman.example.com/api/v2/smart_proxies?search=name="hostname.domain"
Thanks again.
Cassuis
Well I’m pretty much out of ideas. I have updated every SSL cert that I can think of and still can’t get registered. In addition all my other Puppet Masters at now receiving this message and can no longer connect to the Foreman servers as well. This is the message that I’m receiving so it is still some kind of certificate issue.
Failure: ERF50-5345 [Foreman::WrappedException]: Unable to connect ([ProxyAPI::ProxyException]: ERF12-7885 [ProxyAPI::ProxyException]: Unable to fetch logs ([OpenSSL::SSL::SSLError]: SSL_connect returned=1 errno=0 state=error: tlsv1 alert unknown ca) for proxy https://hostname.company.com:8443/logs)
Please if anyone has any thoughts I would appreciate hearing them.
Thanks,
Clay
