Problem:
Unable to replace/update certificates on 1 out of 3 remote foreman-proxy servers using the generated tar certificate bundle.
Expected outcome:
Update/replace relevant certificates (with custom CA) on the remote foreman-proxy server and get it registered to foreman.
Foreman and Proxy versions:
foreman-proxy-2.2.1-1.el7.noarch
foreman-release-2.2.1-1.el7.noarch
Foreman and Proxy plugin versions:
Distribution and version:
RHEL 7
Other relevant data:
We are using katello 3.17 and we’re trying to use a custom CA on our existing environment. The environment consists of 4 RHEL 7 servers.
1 main katello server
2 foreman-proxy compilers (puppetserver)
1 foreman-proxy CA
We managed to get the certificates updated easily on 3 servers.
1 main katello server
2 foreman-proxy compilers (puppetserver)
But not on the foreman-proxy CA.
The process that we’re doing to update the certificate at the remote foreman-proxy is as follows.
At the main katello server:
- Check the validity of the certificates
katello-certs-check -c /etc/ssl/fmanca01/cert/fmanca01.pem \ -k /etc/ssl/fmanca01/key/fmanca01.key \ -b /etc/ssl/net.ca-bundle
- Generate a certificate bundle tar file
foreman-proxy-certs-generate --foreman-proxy-fqdn "fmanca01.inet.net"\ --certs-tar /root/custom_cert/fmanca01.inet.net-certs.tar\ --server-cert /etc/ssl/fmanca01.inet.net/cert/fmanca01.pem\ --server-key /etc/ssl/fmanca01.inet.net/key/fmanca01.key\ --server-ca-cert /etc/ssl/net.ca-bundle\ --certs-update-all
At the remote foreman-proxy:
-
Install the katello-ca-consumer-latest.noarch.rpm
-
Run the command shown after running the foreman-proxy-certs-generate command.
foreman-installer \ --scenario foreman-proxy-content \ --certs-tar-file "/root/fmanca01.inet.net-certs.tar"\ --foreman-proxy-content-parent-fqdn "fmanp02.inet.net"\ --foreman-proxy-register-in-foreman "true"\ --foreman-proxy-foreman-base-url "https://fmanp02.inet.net"\ --foreman-proxy-trusted-hosts "fmanp02.inet.net"\ --foreman-proxy-trusted-hosts "fmanca01.inet.net"\ --foreman-proxy-oauth-consumer-key "ZuQX3XETDtBhboo4WuxqKufKEytkh5FQ"\ --foreman-proxy-oauth-consumer-secret "7UKsqvTcJ5DY4Lrxhqx3PQBFXoje34gR"\ --puppet-server-foreman-url "https://fmanp02.inet.net"
The output of the command
Wrapped exception:
SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)
Preparing installation Done
Something went wrong! Check the log for ERROR-level output
The full log is at /var/log/foreman-installer/foreman-proxy-content.log
Error from /var/log/foreman-installer/foreman-proxy-content.log
[ERROR 2021-05-19T07:47:18 main] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[fmanca01.inet.net]: Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain) in get request to: https://fmanp02.inet.net/api/v2/smart_proxies?search=name="fmanca01.inet.net"
Some observations:
I compared some of the certifcates with the other foreman-proxy that worked and found the following differences:
The following certificates were updated on the 2 remote foreman-proxy that worked but not on fmanca01.inet.net
/etc/pki/ca-trust/source/anchors/katello_server-host-cert.crt
/etc/foreman-proxy/ssl_cert.pem
/etc/foreman-proxy/foreman_ssl_ca.pem
The following packages are not being updated at the fmanca01.inet.net which I expected to be updated because when I extract the contents of the tar bundle, there are already fmanca01.inet.net-xxxx-xxxxx-xxxxx-1.0-3.noarch in it.
fmanca01.inet.net-foreman-proxy-client-1.0-1.noarch
fmanca01.inet.net-qpid-router-client-1.0-1.noarch
fmanca01.inet.net-puppet-client-1.0-1.noarch
fmanca01.inet.net-apache-1.0-1.noarch
fmanca01.inet.net-qpid-client-cert-1.0-1.noarch
fmanca01.inet.net-qpid-broker-1.0-1.noarch
fmanca01.inet.net-foreman-proxy-1.0-1.noarch
fmanca01.inet.net-qpid-router-server-1.0-1011.noarch
Any help is greatly appreciated.