SSL Errors on build

Support
1

Problem: When trying to build machines from Foreman using PXE booting to VMWare, they are suddenly failing with “curl: (60) SSL certificate problem: self-signed certificate in certificate chain”

Expected outcome: Hosts build as normal

Foreman and Proxy versions: Foreman: 3.15.0 / Proxy: 3.15.0 and 3.15.1

Foreman and Proxy plugin versions: katello: 4.17.1, foreman-tasks: 11.0.0, foreman_bootdisk: 23.0.0, foreman_puppet: 9.0.0 foreman_remote_execution: 16.0.3, foreman_templates: 10.0.8, foreman_webhooks: 4.0.1

Distribution and version: Rocky Linux 9.6

Other relevant data:

Screenshot of the error on the VMWare host when trying to build

Screenshot 2026-01-09 143209
Screenshot 2026-01-09 1432091266×218 51.9 KB

This is repeated many times before it eventually stops.

I believe the only change recently was to enable the webhooks plugin.

checking the “/etc/foreman-proxy” folder, the certificates don’t appear to have changed recently:

# ls -alth /etc/foreman-proxy/
total 68K
drwxr-xr-x. 127 root    root          8.0K Dec  2 03:00 ..
drwxr-xr-x.   3 root    root          4.0K Nov 28 12:22 .
-rw-------.   1 foreman foreman-proxy  399 Nov 28 12:22 fiveguys-key
drwxr-xr-x.   2 root    root          4.0K Oct 24 15:01 settings.d
-rw-r-----.   1 root    foreman-proxy 3.4K Oct 24 15:01 settings.yml
-r--r--r--.   1 root    foreman-proxy 8.2K Aug  4 10:05 foreman_ssl_cert.pem
-r--r-----.   1 root    foreman-proxy 3.2K Aug  4 10:05 foreman_ssl_key.pem
-r--r--r--.   1 root    foreman-proxy 8.2K Aug  4 10:05 ssl_cert.pem
-r--r-----.   1 root    foreman-proxy 3.2K Aug  4 10:05 ssl_key.pem
-r--r-----.   1 root    foreman-proxy 2.5K Aug  4 10:05 foreman_ssl_ca.pem
-r--r-----.   1 root    foreman-proxy 2.5K Aug  4 10:05 ssl_ca.pem
-rw-r--r--.   1 root    root             0 Jun  9  2025 migration_state
2

Foreman and the proxies have been updated to 3.16 / Katello 4.18, but the issue persists.

3

Additionally, on hosts that were previously built or registered to Foreman, the certificate has clearly been automatically added to the local store:

curl -vvI https://REDACTED-Proxy/pulp/content/CRUK/Library/custom/Rocky_9/Rocky_9_BaseOS/images/

  • Trying REDACTED-IP:443…
  • Connected to (REDACTED-IP) port 443 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • CAfile: /etc/pki/tls/certs/ca-bundle.crt
  • TLSv1.0 (OUT), TLS header, Certificate Status (22):
  • TLSv1.3 (OUT), TLS handshake, Client hello (1):
  • TLSv1.2 (IN), TLS header, Certificate Status (22):
  • TLSv1.3 (IN), TLS handshake, Server hello (2):
  • TLSv1.2 (IN), TLS header, Finished (20):
  • TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, Request CERT (13):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, Certificate (11):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, CERT verify (15):
  • TLSv1.2 (IN), TLS header, Unknown (23):
  • TLSv1.3 (IN), TLS handshake, Finished (20):
  • TLSv1.2 (OUT), TLS header, Finished (20):
  • TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
  • TLSv1.2 (OUT), TLS header, Unknown (23):
  • TLSv1.3 (OUT), TLS handshake, Certificate (11):
  • TLSv1.2 (OUT), TLS header, Unknown (23):
  • TLSv1.3 (OUT), TLS handshake, Finished (20):
  • SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
  • ALPN, server accepted to use h2
  • Server certificate:
  • subject: C=US; ST=North Carolina; O=Katello; OU=SomeOrgUnit; CN=REDACTED-Proxy
  • start date: Aug 26 13:12:05 2025 GMT
  • expire date: Jan 18 13:12:05 2038 GMT
  • subjectAltName: host “REDACTED-Proxy” matched cert’s “REDACTED-Proxy”
  • issuer: C=US; ST=North Carolina; L=Raleigh; O=Katello; OU=SomeOrgUnit; CN=REDACTED-Foreman-Server
  • SSL certificate verify ok.
  • Using HTTP2, server supports multi-use
  • Connection state changed (HTTP/2 confirmed)

On a host not managed or built by Foreman / a Proxy, Curl throws the same “SSL certificate problem: self-signed certificate in certificate chain” error.

4

Usually, the Foreman/Katello CA is only installed on hosts near the end of the provisioning process (somewhere in KS `%post`) during registration. Because of that, accessing installation media during KS installation via HTTPS cannot work out of the box.

The easy fix would be to switch to plain HTTP for the installation media. Maybe someone just happened to slip a https instead of a http into the IM URL while updating it for a new minor release or something like that?

If this has worked with HTTPS in the past and you need to have this working via HTTPS, I have no clue how this would need to be set up. You would probably need to somehow patch the initrd files to trust the Katello CA, but this is for sure a pretty non-default setup.

5

So I found a workaround of adding a Global Parameter for “kickstart_kernel_custom_options > String > ‘inst.noverifyssl’” which seems to allow the machines to build again. It’s not essential for us to use HTTPS for this, just as far as we’re aware it was working but then stopped!

As far as I’m aware the Installation Media hasn’t been updated recently, but looking at them they do all appear to be https paths for ours - as we point them to the local smart-proxy (which is essential for ones in the Border as they have no internet access)