Foreman-proxy-certs-generate uses wrong server certificate in certs-tar-file

Support
1

Problem:
I am not sure if I miss something here, or if there is a mixup in the parameters I am using.

I have a single foreman server foreman.local.lan (katello scenario) and want to add a second smart proxy smartproxy.local.lan. Before that, I replaced the foreman/katello SSL certificates for foreman.local.lan with a certificate from an external CA. All worked perfectly.

Now I generated a new private key/certificate for the new proxy and placed both files on the Forman server:

foreman:
/root/smartproxy.key
/root/smartproxy.crt
/root/ca.crt

The subject name (and also the SAN) of the smartproxy.crt is set to smartproxy.local.lan.

I run the following command:
katello-certs-check -t foreman-proxy -c smartproxy.crt -k smartproxy.key -b ca.crt
and all checks out “Validation succeeded”.

At the end of this run I get the command to run inside of the new smartproxy:

  To use them inside a NEW $FOREMAN_PROXY, run this command:

      foreman-proxy-certs-generate --foreman-proxy-fqdn "$FOREMAN_PROXY" \
                                   --certs-tar  "~/$FOREMAN_PROXY-certs.tar" \
                                   --server-cert "/root/smartproxy.crt" \
                                   --server-key "/root/smartproxy.key" \
                                   --server-ca-cert "/root/ca.crt" \

So I ran the aforementioned command, replacing $FOREMAN_PROXY with smartproxy.local.lan on the foreman host. I received the certs.tar file, which I copied to smartproxy.local.lan.

On smartproxy.local.lan I ran the foreman-installer as the certs-generate command told me:

  foreman-installer \
                    --scenario foreman-proxy-content \
                    --certs-tar-file                              "/root/smartproxy.local.lan-certs.tar"\
                    --foreman-proxy-content-parent-fqdn           "foreman.local.lan"\
                    --foreman-proxy-register-in-foreman           "true"\
                    --foreman-proxy-foreman-base-url              "https://foreman.local.lan"\
                    --foreman-proxy-trusted-hosts                 "foreman.local.lan"\
                    --foreman-proxy-trusted-hosts                 "smartproxy.local.lan"\
                    --foreman-proxy-oauth-consumer-key            "xxx"\
                    --foreman-proxy-oauth-consumer-secret         "yyy"\
                    --puppet-server-foreman-url                   "https://foreman.local.lan"

The installation seems to succeed, but reports this error:
[ERROR 2020-11-18T20:28:27 main] Proxy smartproxy.local.lan cannot be registered: Unable to communicate with the proxy: ERF12-2530 [ProxyAPI::ProxyException]: Unable to detect features ([Errno::EHOSTUNREACH]: Failed to open TCP connection to smartproxy.local.lan:9090 (No route to host - connect(2) for "smartproxy.local.lan" port 9090)) for proxy https://smartproxy.local.lan:9090/v2/features Please check the proxy is configured and running on the host.

After some web searching I found out, that this error is usually caused by invalid certificates, so I went to check. Found all the certs installed by the certs-tar’s rpm inside of the directory /etc/pki/katello-certs-tools/certs/ and checked each.
All are correct, except the one smartproxy.local.lan-apache.crt
When opened in openssl it shows the subject as: foreman.local.lan, which is clearly false. Therefore a quick test with openssl:
openssl s_client -connect smartproxy.local.lan:9090
returned the wrong certificate, having the subject foreman.local.lan

Expected outcome:
The certificate file smartproxy.local.lan-apache.crt inside of the certs-tar should be taken from the –server-cert parameter of the foreman-proxy-certs-generate call.

Foreman and Proxy versions:
Foreman 2.2.1
Katello 3.17

Foreman and Proxy plugin versions:
Puppet 6

Distribution and version:
CentOS 7.9

Other relevant data:

2

Try regenerating the certificate bundle for the smart proxy passing the regenerate flag:

      foreman-proxy-certs-generate --foreman-proxy-fqdn "$FOREMAN_PROXY" \
                                   --certs-tar  "~/$FOREMAN_PROXY-certs.tar" \
                                   --server-cert "/root/smartproxy.crt" \
                                   --server-key "/root/smartproxy.key" \
                                   --server-ca-cert "/root/ca.crt" \
                                   --certs-regenerate

If you had generated the certificate bundle for that smart-proxy previously, there is a bug where it will not regenerate it everytime (this is fixed in 3.18 – Bug #31234: Users have to delete ssl-build/<capsule> directory and regenerate the certificates to add a cname in capsule certificates - Installer - Foreman). See the attached patch if you’d like to have it set to always do this from here on out until you get to 3.18.

3

Thats interesting, thank you for the tip. Of course I tried it multiple times, so that might just be it. I will post the results.

4

Thanks again Eric, you were spot on. The parameter –certs-regenerate does not work (its not implemented in release 3.17), however, manually deleting the sub directory smartproxy.local.lan under /root/ssl-build and re-running the command to generate the certs.tar file worked perfectly.