Smart Proxy: Oops, we're sorry but something went wrong SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error

Support
1

Problem: Foreman → Smart Proxy: Oops, we’re sorry but something went wrong SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error

Expected outcome: Foreman Connect to Smart Proxy

Foreman and Proxy versions:
Foreman: 3.10
Katello: 4.12

Foreman and Proxy plugin versions:

Distribution and version:
RHEL: 9.3

Other relevant data:
Hi All,

Setting up a Foreman just for yum repository, no puppet no dhcp no dns services.

May i understand which certificate or CA is required for Foreman to connect to the Smart Proxy ?
I used a custom certificate to build scenario katello:

foreman-installer --scenario “katello”
–foreman-initial-organization “${ORG}”
–foreman-initial-location “Global”
–foreman-servername “${FOREMAN_HOSTNAME}.${FOREMAN_DOMAIN}”
–foreman-serveraliases “${FOREMAN_FQDN}”
–foreman-proxy-dhcp “false”
–foreman-proxy-dns “false”
–foreman-proxy-tftp “false”
–certs-server-ca-cert “/etc/pki/certs/internalcacerts.crt”
–certs-server-cert “/etc/pki/tls/certs/${FOREMAN_HOSTNAME}.${FOREMAN_DOMAIN}.pem”
–certs-server-key “/etc/pki/tls/private/${FOREMAN_HOSTNAME}.${FOREMAN_DOMAIN}.key”
–foreman-initial-admin-username $(user)
–foreman-initial-admin-password $(secret)
–enable-foreman-plugin-discovery --enable-foreman-proxy-plugin-discovery --foreman-proxy-bind-host ‘*’

Generated a Cert for the Smart Proxy:

foreman-proxy-certs-generate --foreman-proxy-fqdn ${SPROXY_HOSTNAME}
–foreman-proxy-cname ${SPROXY_FQDN}
–certs-tar /var/www/html/pub/${SPROXY_HOSTNAME}/${SPROXY_HOSTNAME}.tar
–server-cert ~/capsule/${SPROXY_HOSTNAME}/${SPROXY_HOSTNAME}.pem
–server-key ~/capsule/${SPROXY_HOSTNAME}/${SPROXY_HOSTNAME}.key
–server-ca-cert /etc/pki/certs/internalcacerts.crt

Then build Smart proxy:

foreman-installer
–scenario foreman-proxy-content
–certs-tar-file “/root/${SPROXY_HOSTNAME}.${SPROXY_DOMAIN}.tar”
–foreman-proxy-register-in-foreman “true”
–foreman-proxy-foreman-base-url “https://${FOREMAN_HOSTNAME}.${FOREMAN_DOMAIN}”
–foreman-proxy-trusted-hosts “${FOREMAN_HOSTNAME}.${FOREMAN_DOMAIN}”
–foreman-proxy-trusted-hosts “${SPROXY_HOSTNAME}.${SPROXY_DOMAIN}”
–foreman-proxy-oauth-consumer-key “${KEY}”
–foreman-proxy-oauth-consumer-secret “${SECRET}”
–foreman-proxy-registration-url “https://${SPROXY_FQDN}:9090”
–certs-cname “${SPROXY_FQDN}”

Installation successful but when I try to verify connection between Foreman to Smart Proxy “Oops, we’re sorry but something went wrong SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error”

I verified the certificate on smart proxy it requires the the Foreman generated CA and not my custom ones.
openssl s_client -connect ${SPROXY_HOSTNAME}.${SPROXY_DOMAIN}:443
CONNECTED(00000003)
802BF0000F65FA57:error:0A000438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:ssl/record/rec_layer_s3.c:1600:SSL alert number 80
no peer certificate available

Acceptable client certificate CA names
C = US, ST = North Carolina, L = Raleigh, O = Katello, OU = SomeOrgUnit, CN = ${SPROXY_HOSTNAME}.${SPROXY_DOMAIN}

Which CA do i need to add to default OS bundle to be able to accept connection?

2

Please put code in preformatted blocks. They are much better readable.

I highly recommend to follow the docs

https://docs.theforeman.org/3.10/Installing_Server/index-katello.html#Configuring_Server_with_a_Custom_SSL_Certificate_foreman

and

https://docs.theforeman.org/3.10/Installing_Proxy/index-katello.html#configuring-capsule-default-certificate_smart-proxy

and in particular check the certs using katello-certs-check for the main server as well as the proxy.

If your proxy doesn’t use the custom ones which you have put into the tar with foreman-proxy-certs-generate then this suggests that you have tried it before and already installed the foreman certs before. The procedure for certificate update/renewal requires additional parameters.

Also remember, that foreman-installer always remembers your selection from the previous runs.

None. Do not add them to the default bundle. Foreman uses the certs and CAs which you set with foreman-installer.

Does internalcacerts.crt contain the full chain including the root ca?

3

Hi,

Yes the bundle contains the Root and the Signing CA.
i always try fresh install as this is currently a testbox
Yes i did use katello-certs-check and all valid:

Validation succeeded
To install the Katello server with the custom certificates, run:
    foreman-installer --scenario katello \
                      --certs-server-cert "/etc/pki/tls/certs/foreman.fqdn.pem" \
                      --certs-server-key "/etc/pki/tls/private/foreman.fqdn.key" \
                      --certs-server-ca-cert "/etc/pki/my/internalcacertsbundleall.crt"

i run exactly that command without any addition just to test, but this time, during the install itself it is now failing:

2024-08-24 10:42:13 [ERROR ] [configure] /Stage[main]/Foreman::Register/Foreman_host[foreman-foreman.fqdn]: Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error in get request to: https://foreman.fqdn/api/v2/hosts?search=name%3D%22foreman.fqdn%22
2024-08-24 10:42:13 [ERROR ] [configure] Wrapped exception:
2024-08-24 10:42:13 [ERROR ] [configure] SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error
2024-08-24 10:42:13 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_host[foreman-proxy-foreman.fqdn]: Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error in get request to: https://foreman.fqdn/api/v2/hosts?search=name%3D%22foreman.fqdn%22
2024-08-24 10:42:13 [ERROR ] [configure] Wrapped exception:
2024-08-24 10:42:13 [ERROR ] [configure] SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error
2024-08-24 10:42:13 [ERROR ] [configure] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[foreman.fqdn]: Could not evaluate: Exception SSL_connect returned=1 errno=0 state=error: tlsv1 alert internal error in get request to: https://foreman.fqdn/api/v2/smart_proxies?search=name%3D%22foreman.fqdn%22

Under ssl-build i see these:

[ssl-build] # for a in `ls *.crt`; do echo $a; openssl x509 -in $a -subject -noout; done
katello-default-ca.crt
subject=C = US, ST = North Carolina, L = Raleigh, O = Katello, OU = SomeOrgUnit, CN = foreman.fqdn
katello-server-ca.crt
subject=DC = my, DC = correct, DC = CA, CN = Correct Issuing Certification Authority

[ssl-build][foreman.fqdn] # for a in `ls *.crt`; do echo $a; openssl x509 -in $a -subject -noout; done
foreman.fqdn-apache.crt
subject=O = my_org, OU = myorg_unit, CN = foreman.fqdn
foreman.fqdn-foreman-client.crt
subject=C = US, ST = North Carolina, O = FOREMAN, OU = PUPPET, CN = foreman.fqdn
foreman.fqdn-foreman-proxy-client.crt
subject=C = US, ST = North Carolina, O = FOREMAN, OU = FOREMAN_PROXY, CN = foreman.fqdn
foreman.fqdn-foreman-proxy.crt
subject=O = my_org, OU = myorg_unit, CN = foreman.fqdn

and i can see that it uses the self-signed generated by Foreman:

[ssl-build][foreman.fqdn] # for a in `ls *.crt`; do echo $a; openssl x509 -in $a -subject -noout; done
foreman.fqdn-apache.crt
subject=O = my_org, OU = myorg_unit, CN = foreman.fqdn
foreman.fqdn-foreman-client.crt
subject=C = US, ST = North Carolina, O = FOREMAN, OU = PUPPET, CN = foreman.fqdn
foreman.fqdn-foreman-proxy-client.crt
subject=C = US, ST = North Carolina, O = FOREMAN, OU = FOREMAN_PROXY, CN = foreman.fqdn
foreman.fqdn-foreman-proxy.crt
subject=O = my_org, OU = myorg_unit, CN = foreman.fqdn

it’s like the initial Foreman-Katello is fine, then the Foreman Smart Proxy kicks in and updates the certificates, i will try to skip the foreman-proxy install next

4

Just an update, the installation is now successful with custom certificate after I reduced the bundle to just the root and the signing CA.

From:

Checking CA bundle size: 30
[OK]

to

Checking CA bundle size: 2
[OK]

But i’m not sure how that fixed it.