Certificate setup failure with custom CA

I don’t think so. They both need to be in the same file. The order is also important. IIRC it must be first the intermediate and then the CA.

I don’t immediately know how to create chain bundle (suggestions/tips welcome), but ill give this a try and circle back.

You can just cat them together.

2 Likes

Y’alls suggestions worked. Simply creating a chain bundle resolved issue. Looks like all certs are good, AND smart proxy is up.

Obviously, im not very familiar with custom cert implementations. Really appreciate y’alls patience.

1 Like

Glad to hear that. SSL implementations often require some knowledge that isn’t really documented. Those who use it typically know these pitfalls and don’t write docs, those who don’t are doomed to learn it the hard way.

Well, generally, if you configure an SSL server using a certificate from a sub CA of a root or intermediate CA the server must be configured with the CA chain. Thus, you’ll have to specify a bundle which contains those sub/intermediate CAs (and usually you can omit the root CA because the client should have preinstalled trusted CAs). That is how it always works. It has nothing to do with custom certs…

When a client connects, the server presents its own certificate together with the CA chain. The client uses the chain and it’s builtin list of trusted root CAs to verify the chain and the server certificate.

If the server does not deliver all intermediate CAs together the client won’t be able to verify the certificate and will fail. Unfortunately, browsers like firefox cache intermediate CAs whenever they come across one. When you later browse to a site which does not present all intermediate CAs, Firefox is still able to verify the chain because it has already learned the CAs before.

2 Likes

Very helpful, thank you. I was defiantly making a very big mistake by trying the root and sub, but not creating a CAChain.

I think the reason i was so hung up and confused about the appropriate solution was because the websocket and internal apache server were working ok, and the Smart Proxy was the only thing that was broken. This predicament lead me to incorrectly assume the issue was with Smart Proxy, and not my Certs. Lessons learned.

Well, it’s definitively misleading if your first check command

katello-certs-check -c /etc/pki/tls/certs/myforemanserver.crt \
> -k /etc/pki/tls/certs/myforemanserverD.key \
> -b /etc/pki/tls/certs/rootca-CA_2019.crt

was successful. It basically runs

openssl verify -CAfile /etc/pki/tls/certs/rootca-CA_2019.crt -purpose sslserver -verbose /etc/pki/tls/certs/myforemanserver.crt

for “Checking CA bundle against the certificate file:”

However, the openssl verify includes installed certificates, e.g. in /etc/pki/tls/certs on CentOS if you created the necessary hash symlinks. Basically, you can probably omit the CAfile option in the openssl verify call and still it gets verified. So basically, that call doesn’t really do what it says it does. It checks against the installed certificates in the system directory /etc/pki/tls/certs and in addition the CAfile certfile.

So this call more or less tells you if the certificate gets verified on the system you are running. I guess it would be more helpful, if the katello-certs-check actually verifies the certificate using only the CA certificates from -b option ignore any system setting. Then the verification is actually independent from the system you are running the check.

2 Likes

Yup, spot on. That would have pointed me in the right direction from the beginning.

I am having the same problem with my certificates. I have a wildcard certificate generated by my ISP Ionos.co.uk:

test.com_ssl_certificate.cer
test.com_ssl_certificate_INTERMEDIATE.cer

When I try to use it with Foreman I get an error:

# openssl verify -CAfile /tmp/rootCA.pem -purpose sslserver -verbose /tmp/test.com_ssl_certificate.cer
/tmp/test.com_ssl_certificate.cer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Encryption Everywhere DV TLS CA - G1
error 2 at 1 depth lookup:unable to get issuer certificate

I have tried downloading the latest root certificate CybertrustGlobalRoot.pem and also concat’d it with my INTERMEDIATE cert but I always get the issuer error

If I run:

cp test.com_ssl_certificate_INTERMEDIATE.cer /etc/pki/ca-trust/source/anchors/
cp CybertrustGlobalRoot.pem /etc/pki/ca-trust/source/anchors/
update-ca-trust

This does not help?

# katello-certs-check -c test.com_ssl_certificate.cer -b /tmp/rootCA.pem -k test.com_private_key.key
Checking server certificate encoding:
[OK]

Checking expiration of certificate:
[OK]

Checking expiration of CA bundle:
[OK]

Checking if server certificate has CA:TRUE flag
[OK]

Checking for private key passphrase:
[OK]

Checking to see if the private key matches the certificate:
[OK]

Checking CA bundle against the certificate file:
[FAIL]

The /tmp/rootCA.pem does not verify the /tmp/test.com_ssl_certificate.cer
/tmp/test.com_ssl_certificate.cer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Encryption Everywhere DV TLS CA - G1
error 2 at 1 depth lookup:unable to get issuer certificate

Checking CA bundle size: 2
[OK]

Checking Subject Alt Name on certificate
[OK]

Checking if any Subject Alt Name on certificate matches the Subject CN
[OK]

Checking Key Usage extension on certificate for Key Encipherment
[OK]

Checking for use of shortname as CN
[OK]

Any ideas what I am doing wrong?

The bundle file for option -b must contain the intermediate and the root ca certificates to verify the server certificate.

You can first use openssl verify with CAfile rootCA only against intermediate to make sure that sticks, e.g.

$ openssl verify -verbose -CAfile /tmp/rootCA.pem test.com_ssl_certificate_INTERMEDIATE.cer

With rootCA.pem only containing the root CA (I suppose CybertrustGlobalRoot.pem) This should return OK.

If it does, concatenate the root and intermediate into a single file (both pem format of course) and then use the concatenated file as CA file against your server certificate.

If I run the first command I get:

# openssl verify -verbose -CAfile /tmp/CybertrustGlobalRoot.pem /tmp/test.com_ssl_certificate_INTERMEDIATE.cer
/tmp/test.com_ssl_certificate_INTERMEDIATE.cer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Encryption Everywhere DV TLS CA - G1
error 20 at 0 depth lookup:unable to get local issuer certificate

If I concat the CA and INTER files into a singe CA.pem and compare I still get the same error. I think there must be an issue with the root cert?

The biggest clue I can find is if I check my root cert I see:

# openssl x509 -in CybertrustGlobalRoot.pem -text -noout | egrep '(Issuer:|Subject:|CA:|DNS:|Digtal|Not Before|Not After)'
        Issuer: C=IE, O=Baltimore, OU=CyberTrust, CN=Baltimore CyberTrust Root
            Not Before: May 12 18:46:00 2000 GMT
            Not After : May 12 23:59:00 2025 GMT
        Subject: C=IE, O=Baltimore, OU=CyberTrust, CN=Baltimore CyberTrust Root
                CA:TRUE, pathlen:3

Whereas Katello-certs-check shows:

Checking CA bundle against the certificate file:
[FAIL]

The /root/certs/cloudboxservices.com_ssl_certificate_INTERMEDIATE.cer does not verify the /root/certs/cloudboxservices.com_ssl_certificate.cer
/root/certs/cloudboxservices.com_ssl_certificate.cer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Encryption Everywhere DV TLS CA - G1
error 2 at 1 depth lookup:unable to get issuer certificate

I was told that Cybertrust and Digicert were the same issuer but it looks as though I need to search the net for this issuer root cert?

Solved it!

This was a really stupid problem digicert has more than one “brand” of certificate and I had downloaded the wrong one (dooh!). Once I found the correct one at: DigiCert Root Certificates - Download & Test | DigiCert.com and concatenated the intermediate and root it all works as expected.

Now when I actually try to replace the Foreman-certs I get the following error:

2021-04-02 08:27:55 [ERROR ] [configure] 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]
2021-04-02 08:27:55 [ERROR ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: change from 'notrun' to ['0'] failed: 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]
2021-04-02 08:27:55 [NOTICE] [configure] 3100 out of 3210 done.
2021-04-02 08:27:55 [NOTICE] [configure] 3200 out of 3210 done.
2021-04-02 08:27:58 [NOTICE] [configure] System configuration has finished.
2021-04-02 08:27:58 [NOTICE] [post] Executing hooks in group post

  There were errors detected during install.

The strange thing is that it seems to work and I now see my certificate when I login?