Can't setup custom certificate

Problem:
I recently tried to use LetsEncrypt to setup custom certificates but I am running into problems I issued the certificate from letsencrypt and I have my Cert/Key and CA Cert. When I tried to install the check seemed to fail checking the CA Bundle against the certificate file. Both were provided by Letencrypt. What am I missing?

# katello-certs-check -t foreman -c server.mydomain.com.cer -k server.mydomain.com.key -b fullchain.cer
Checking server certificate encoding:
[OK]

Checking expiration of certificate:
[OK]

Checking expiration of CA bundle:
[OK]

Checking if server certificate has CA:TRUE flag
[OK]

Checking for private key passphrase:
[OK]

Checking to see if the private key matches the certificate:
[OK]

Checking CA bundle against the certificate file:
[FAIL]

The /root/.acme.sh/server.mydomain.com/fullchain.cer does not verify the /root/.acme.sh/server.mydomain.com/server.mydomain.com.cer
C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
error 2 at 2 depth lookup: unable to get issuer certificate
error /root/.acme.sh/server.mydomain.com/server.mydomain.com.cer: verification failed

Checking CA bundle size: 3
[OK]

Checking Subject Alt Name on certificate
[OK]

Checking if any Subject Alt Name on certificate matches the Subject CN
[OK]

Checking Key Usage extension on certificate for Key Encipherment
[OK]

Checking for use of shortname as CN
[OK]


Expected outcome:
I expect the check to pass as everything was provided by certificate creation process. Not sure what I am missing here

Foreman and Proxy versions:
foreman-3.1.2
katello-4.3.0

Distribution and version:
Rocky Linux 8

Other relevant data:

As the error messages says, your chain file does not contain all necessary files for the chain including the root. In your case, it seems to be missing the issuer ca certificate for “USERTrust RSA Certification Authority”. Although that’s odd if you are using letsencrypt because I think letsencrypt doesn’t use USERTrust. Are you sure you have set up to obtain the certificate from letsencrypt?

Yes its using letsencrypt. I used certbot to obtain the certificate and the logfile indicates it’s connecting to https://acme-v02.api.letsencrypt.org. I tried using the acme.sh script as well and got the same results.

If you are using certbot, how does the certificate get into /root/.acme.sh/ ? Doesn’t look like the standard certbot/letsencrypt paths in /etc/letsencrypt… Is it possible you are using acme.sh and certbot obtaining certificates from different sources?

I suggest you check the certs and chain you have:

$ openssl x509 -in /root/.acme.sh/server.mydomain.com/server.mydomain.com.cer -noout -issuer

that should be from

issuer=C = US, O = Let's Encrypt, CN = R3

You can check more details from the cert with

$ openssl x509 -in /root/.acme.sh/server.mydomain.com/server.mydomain.com.cer -noout -text

Then check all certs in the chain file:

$ openssl x509 -noout -issuer -subject
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----

View the whole content of /root/.acme.sh/server.mydomain.com/fullchain.cer and copy&paste each BEGIN/END CERTIFICATE block into openssl to see the issuer and subject of each chain cert…

Sorry I used acme.sh initially but thought it might be related the script. So I switched over to certbot as I saw more examples with regards to that. So my initial paste of the failed check was against the acme.sh script however when I switched over I got the same response with the certbot certificates [just different path]. I did the checks you sugestted and all come back as Lets Encrypt:

# openssl x509 -in fullchain.pem -noout -issuer
issuer=C = US, O = Let's Encrypt, CN = R3


# openssl x509 -in fullchain.pem -noout -text
Certificate:
<Redacted>
        Issuer: C = US, O = Let's Encrypt, CN = R3
        Validity
            Not Before: Apr 16 23:30:45 2022 GMT
            Not After : Jul 15 23:30:44 2022 GMT
        Subject: CN = server.mydoamin.com
<Redacted the rest>        




#  openssl x509 -noout -issuer -subject
-----BEGIN CERTIFICATE-----
<Redacted>
-----END CERTIFICATE-----
issuer=C = US, O = Let's Encrypt, CN = R3
subject=CN = server.mydomain.com
#  openssl x509 -noout -issuer -subject
-----BEGIN CERTIFICATE-----
<Redacted>
-----END CERTIFICATE-----
issuer=C = US, O = Let's Encrypt, CN = R3
subject=CN = server.mydomain.com
#  openssl x509 -noout -issuer -subject
-----BEGIN CERTIFICATE-----
<Redacted>
-----END CERTIFICATE-----
issuer=C = US, O = Let's Encrypt, CN = R3

So it is showing that its setup with Lets Encrypt. I can only thing that the CA bundle in the fullchain.pem [or .cer file for the acme.sh] isn’t being read correctly for the CA bundle. I was under the impression tha thte fullchain.pem included the intermediate certificate. Do I need to provide another certificate to validate this?

Also here is the output from the certbot certificates:

# katello-certs-check -c fullchain.pem -k privkey.pem  -b fullchain.pem
Checking server certificate encoding:
[OK]

Checking expiration of certificate:
[OK]

Checking expiration of CA bundle:
[OK]

Checking if server certificate has CA:TRUE flag
[OK]

Checking for private key passphrase:
[OK]

Checking to see if the private key matches the certificate:
[OK]

Checking CA bundle against the certificate file:
[FAIL]

The /etc/letsencrypt/archive/server.mydomain.com/fullchain1.pem does not verify the /etc/letsencrypt/archive/server.mydomain.com/fullchain1.pem
C = US, O = Internet Security Research Group, CN = ISRG Root X1
error 2 at 2 depth lookup: unable to get issuer certificate
error /etc/letsencrypt/archive/server.mydomain.com/fullchain1.pem: verification failed

Checking CA bundle size: 3
[OK]

Checking Subject Alt Name on certificate
[OK]

Checking if any Subject Alt Name on certificate matches the Subject CN
[OK]

Checking Key Usage extension on certificate for Key Encipherment
[OK]

Checking for use of shortname as CN
[OK]

OK weird. I am still getting the failure when I run the katello-certs-check but I ran the foreman-installer again and it seems to be working. Certs installed and site is now running secure. Not sure what’s different but I’ll take it.

You didn’t do it right. My first two commands where for the server certificate, not the fullchain file.

For the second part, where you have to copy&paste each block from the fullchain file, you also didn’t seem to do it right. It seems like you only copied the first certificate from the chain each time. Each certificate in the fullchain file should be different, starting with the server certificate itself…

On a sidenote: don’t invent fake domain names which properly belong to someone else. Always use example.com or example.org instead, which are reserved for this purpose…

I think you should use the /etc/letsencrypt/live path to access the current certs and chains. For -b you should properly use the chain.pem and not fullchain.pem because the latter includes the server cert. chain.pem is really only the chain.

For letencrypt, if you check the certs in fullchain.pem it should look something like this:

# openssl x509 -noout -subject -issuer
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
subject=CN = www.example.com
issuer=C = US, O = Let's Encrypt, CN = R3
# openssl x509 -noout -subject -issuer
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw
WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP
R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx
sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm
NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg
Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG
/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB
Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA
FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw
AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw
Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB
gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W
PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl
ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz
CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm
lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4
avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2
yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O
yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids
hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+
HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv
MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
nLRbwHOoq7hHwg==
-----END CERTIFICATE-----
subject=C = US, O = Let's Encrypt, CN = R3
issuer=C = US, O = Internet Security Research Group, CN = ISRG Root X1
# openssl x509 -noout -subject -issuer
-----BEGIN CERTIFICATE-----
MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1ow
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEB
AQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XC
ov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpL
wYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+D
LtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK
4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5
bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5y
sR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZ
Xmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4
FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBc
SLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2ql
PRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TND
TwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYw
SwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1
c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx
+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEB
ATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQu
b3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9E
U1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26Ztu
MA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC
5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW
9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuG
WCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9O
he8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFC
Dfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5
-----END CERTIFICATE-----
subject=C = US, O = Internet Security Research Group, CN = ISRG Root X1
issuer=O = Digital Signature Trust Co., CN = DST Root CA X3

Now that looks like you have not installed the trusted root ca from your distribution or you have configured your system or enviroment not to use the default root ca certs.

Yeah I realized this afterwards. The system is registered to my FreeIPA server which is likely what caused the issue. After running the foreman-installer though my letsencrypt certs were used and now I can access it via HTTPS without that stinky unsecure lock! Thanks for your guidance though Now to deal with the other certificate issue with my release.gpg signing for ubuntu repository sync.