Custom IPA Issued Certificates in Foreman Katello 3.10/3.11

User4871 · July 19, 2024, 1:37am

Problem:
I have a trust issue with actions related to my local smart proxy,
when syncing or attempting to add new content to my smart proxy it returns the following error

Could not create the repository:
  SSL_read: tlsv1 alert unknown ca

or

SSL_read: tlsv1 alert unknown caSSL_read: tlsv1 alert unknown caSSL_read: tlsv1 alert unknown ca

Expected outcome:
Ability to interact with pulp and create content and sync existing repositories.

Foreman and Proxy versions:
Replicated on 3.10 and 3.11, but will be providing examples on 3.10 only.

Foreman and Proxy plugin versions:

Version: 3.10.0
API Version: v2
Database:
Status: ok
Server Response: Duration: 1ms
Plugins:

Name: foreman-tasks
Version: 9.1.1
Name: foreman_ansible
Version: 14.0.0
Name: foreman_remote_execution
Version: 13.0.0
Name: katello
Version: 4.12.1
Smart Proxies:
Name: fm99.example.com
Version: 3.10.0
Status: ok
Features:
1. Name: dynflow
  Version: 0.9.2
2. Name: script
  Version: 0.10.6
3. Name: ansible
  Version: 3.5.5
4. Name: pulpcore
  Version: 3.3.0
5. Name: logs
  Version: 3.10.0

Distribution and version:
Operating System: Red Hat Enterprise Linux 8.7 (Ootpa)
CPE OS Name: cpe:/o:redhat:enterprise_linux:8::baseos
Kernel: Linux 4.18.0-425.19.2.el8_7.x86_64
Architecture: x86-64

Other relevant data:

I know the cause of my issues stems from the Foreman Proxy not using the IPA issued certificates, instead it is using the self generated Katello certificates.
I’d like to get some clarification on its requirement and the cleanest way to handle this
e.g. should i use a separate certificate for the smart proxy install on the foreman host?

Does Katello certificates have any role within a Foreman environment with an external CA?
My background with foreman is from 1.3 to 2.4 releases when i used it with puppet, so this is my first step into using Katello with custom certs and want to understand things better.

Configuring Server with a Custom SSL Certificate mentions “The same CA must sign certificates for Foreman Server and Smart Proxy server” and “You must not use the same SSL certificate for Foreman server and Smart Proxy server.”
Is this still relative for a single host?

The other question i have is where is the best place to update this for the smart proxy configuration
Deploying a custom SSL certificate to Smart Proxy server seems to be an example for a separate server. for a single host install is it only the following areas that need to have the certificate details provided?

--foreman-proxy-ssl-ca "/etc/pki/tls/certs/idx.bundle.pem" \
--foreman-proxy-ssl-cert "/etc/pki/tls/certs/foreman.crt" \
--foreman-proxy-ssl-key "/etc/pki/tls/private/foreman.key" \
--foreman-proxy-foreman-ssl-ca "/etc/pki/tls/certs/idx.bundle.pem" \
--foreman-proxy-foreman-ssl-cert "/etc/pki/tls/certs/foreman.crt" \
--foreman-proxy-foreman-ssl-key "/etc/pki/tls/private/foreman.key" \

currently my installation done using the following

sudo foreman-installer \
--scenario katello \
--foreman-initial-organization "Example Networks" \
--foreman-initial-location "Example" \
--foreman-server-ssl-cert "/etc/pki/tls/certs/foreman.crt" \
--foreman-server-ssl-key "/etc/pki/tls/private/foreman.key" \
--foreman-server-ssl-ca "/etc/pki/tls/certs/idx.bundle.pem" \
--foreman-server-ssl-chain "/etc/pki/tls/certs/idx.bundle.pem" \
--foreman-ipa-authentication=true \
--enable-foreman-plugin-ansible \
--enable-foreman-proxy-plugin-ansible --enable-foreman-cli-ansible

cps86 · November 15, 2024, 3:14pm

I have exactly the same problem, however, I am not using the katello version, nor puppet. I am running Foreman 3.11 and Foreman Smart-Proxy on the same machines. They’re running on HA mode in a cluster.

There are no errors in the Infrastructure/Smart Proxies tab in Foreman Web Console. Smart proxies are trusted. Everything seems to work as expected.
However, I installed foreman remote execution plugin and I can’t execute any tasks because I am getting:

/usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context’
2024-11-15T15:59:38 73121188 [E] OpenSSL::SSL::SSLError SSL_read: tlsv1 alert unknown ca

Which drives me crazy. Why this error generates if I don’t get any errors or warnings in the Infrastructure/Smart Proxies tab?

My configuration is as following:
Foreman

:ssl_certificate: /etc/certs/server1.crt
:ssl_ca_file: /etc/certs/Foreman_CA.crt
:ssl_priv_key: /etc/certs/server1.key

Foreman Smart Proxy
(they are pointing to the same files)

:ssl_ca_file: /etc/certs/Foreman_CA.crt
:ssl_certificate: /etc/certs/server1.crt
:ssl_private_key: /etc/certs/server1.key

Same certs are used for httpd.

Why ca is not trusted if I use the same CA for both?! It doesn’t make any sense. Can I use the same certs for smart proxy and foreman?

cps86 · November 18, 2024, 5:57am

Updated to version 3.12 and the problem remains.
Can someone tell me how to verify if Foreman can communicate with Foreman Smart Proxy? In my case I did the following checks:

Checked that Both smart proxies has green status in Infrastracture/ Smart proxies tab:

image1432×99 15.8 KB
Executed from Foreman server (and smart proxy server) following curl command:

curl -v --cacert /etc/certs/Foreman_CA.crt https://server1.infra.net:8443/features

* TLSv1.2 (OUT), TLS header, Unknown (23):
> GET /features HTTP/1.1
> User-Agent: curl/7.76.1
> Accept: */*
>
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Unknown (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 37
< X-Content-Type-Options: nosniff
< Server: foreman-proxy/3.12.0
< Date: Mon, 18 Nov 2024 05:48:10 GMT
< Connection: Keep-Alive
<
* TLSv1.2 (IN), TLS header, Unknown (23):
* Connection #0 to host server1.infra.net left intact
["ansible","dynflow","logs","script"]

SSL verification with OpenSSL OK:

CONNECTED(00000003)
(REMOVED)
verify return:1
(REMOVED)
verify return:1
(REMOVED)
verify return:1
SSL handshake has read 5136 bytes and written 440 bytes
Verification: OK
Verify return code: 0 (ok)

Any idea why I get Unknown CA error ?

Is there an another way to verify if Foreman can talk with Foreman SmartProxy?

cps86 · November 21, 2024, 6:45am

Ok, the only way to get it work on 3.12 is to have self-generated certs by Foreman’s puppet for Foremans Smart proxy & foreman.
I left only the web frontend to use custom certs.

rriemann · November 22, 2025, 5:47pm

Hi all,

In the context of https://eu-os.eu/poc/provisioning, we also look into this topic.

We use Foreman with Katello and FreeIPA. We bumbed into the following error when importing IPA certificates to Foreman:

~~~
sudo foreman-installer --scenario katello \
–certs-server-cert “/etc/pki/tls/certs/foreman.crt” \
–certs-server-key “/etc/ssl/private/foreman.key” \
–certs-server-ca-cert “/etc/ipa/ca.crt” \
–certs-update-server --certs-update-server-ca
[…]
Processing by Api::V2::SmartProxiesController#refresh as JSON
2025-11-22T15:01:29 [I|app|be7b8b20] Parameters: {“apiv”=>“v2”, “id”=>“1”, “smart_proxy”=>{}}
2025-11-22T15:01:29 [I|app|be7b8b20] Authorized user foreman_api_admin(API Admin)
2025-11-22T15:01:29 [W|app|be7b8b20] Action failed
2025-11-22T15:01:29 [I|app|be7b8b20] Backtrace for ‘Action failed’ error (ProxyAPI::ProxyException): ERF12-9411 [ProxyAPI::ProxyException]: Unable to fetch public key ([OpenSSL::SSL::SSLError]: SSL_read: tlsv1 alert unknown ca) for proxy https:/
/fleet.eu-os.internal:9090/ssh
~~~

The current understanding is:

Only when using katello, smart proxies are setup that authenticate with client-side (mTLS) certificates to the Foreman server.
The default certificate issued for foreman server by FreeIPA has a more limited certificate profile than what is required according to the Foreman docs (to be verified)
Without the proper extended Key use attributes, the Foreman server certificate cannot be used to issue mTLS certificates for the smart proxies.
Hence, those smart proxies cannot authenticate themselves towards the Foreman server. With their old certificate, you see some SSL errors.

Docs: https://docs.theforeman.org/3.17/Installing_Server/index-katello.html#creating-a-custom-ssl-certificate_foreman

[ req ]
req_extensions = v3_req
distinguished_name = req_distinguished_name
prompt = no

[ req_distinguished_name ]
commonName = foreman.example.com

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = foreman.example.com

So it is important that the certificate from FreeIPA also has support for extendedKeyUsage and keyUsage as shown above.

Currently, I do not know how to ensure that.

@iballou, in your experiments with Katello, did you check this integration step with Foreman?

User4871 · November 22, 2025, 7:31pm

I currently have foreman/katello running with FreeIPA without any issues,
what i ended up doing is essentially generating the certificates how i need them first, before the installation of foreman, although you could retroactively do this too.

Currently my process is to do the following, it handles generating certificates with the required extended key usage.

Join server to FreeIPA

```
sudo ipa-client-install --mkhomedir --force-join --server=id01.example.com --domain example.com --hostname foreman.example.com  --password '<one-time-pass>'
```

On FreeIPA create a foreman service, (OPTIONAL)
Identity => Services => "add +" foreman-core/foreman.example.com@EXAMPLE.COM

Generate the certificates for foreman to use, note it includes the extended key usage required.

sudo ipa-getcert request \
-f /etc/pki/tls/certs/foreman.crt \
-k /etc/pki/tls/private/foreman.key \
-F /etc/pki/tls/certs/idx.bundle.pem \
-K foreman-core/foreman.example.com \
-D foreman.example.com \
-u digitalSignature \
-u nonRepudiation \
-u keyEncipherment \
-u dataEncipherment \
-U id-kp-serverAuth \
-U id-kp-clientAuth \
-U id-kp-codeSigning \
-U id-kp-emailProtection

if you don’t want to use your own foreman service in FreeIPA then run the following, it uses the default HTTP service that should already be setup in FreeIPA.

sudo ipa-getcert request \
-f /etc/pki/tls/certs/foreman.crt \
-k /etc/pki/tls/private/foreman.key \
-F /etc/pki/tls/certs/idx.bundle.pem \
-K HTTP/foreman.example.com \
-D foreman.example.com \
-u digitalSignature \
-u nonRepudiation \
-u keyEncipherment \
-u dataEncipherment \
-U id-kp-serverAuth \
-U id-kp-clientAuth \
-U id-kp-codeSigning \
-U id-kp-emailProtection

this will request the required certificate from FreeIPA.

validate the certificates with katello-certs-check

sudo katello-certs-check \
-c /etc/pki/tls/certs/foreman.crt \
-k /etc/pki/tls/private/foreman.key \
-b /etc/pki/tls/certs/idx.bundle.pem

then proceed with the installation ensuring you have the following flags set in the foreman-installer

--foreman-server-ssl-cert "/etc/pki/tls/certs/foreman.crt" \
--foreman-server-ssl-key "/etc/pki/tls/private/foreman.key" \
--foreman-server-ssl-ca "/etc/pki/tls/certs/idx.bundle.pem" \
--foreman-server-ssl-chain "/etc/pki/tls/certs/idx.bundle.pem" \
--foreman-ipa-authentication=true \

Hope this helps.

gvde · November 23, 2025, 6:32am

Please don’t hijack an old thread. Open a new one and give all the information about your system, versions etc which the template tells you to.

gvde · November 23, 2025, 6:39am

User4871:

then proceed with the installation ensuring you have the following flags set in the foreman-installer

--foreman-server-ssl-cert "/etc/pki/tls/certs/foreman.crt" \
--foreman-server-ssl-key "/etc/pki/tls/private/foreman.key" \
--foreman-server-ssl-ca "/etc/pki/tls/certs/idx.bundle.pem" \
--foreman-server-ssl-chain "/etc/pki/tls/certs/idx.bundle.pem" \
--foreman-ipa-authentication=true \

This is exactly one of the ways how to break your systems. If you set the certificates for specific parts of the foreman system it won’t fit to other parts. DO NOT DO THAT!

The docs tell you how to do it and the docs tell you to use the certs module for a reason. One you have modified some paths manually, the certs module won’t change it. If you set foreman-server-ssl-cert manually it won’t fit with what the certs module does.

Follow the docs. If you are using katello-certs-check that actually tells you want to run. It tells you to use the certs module.

Basically, now you have broken your system and you are destined for further problems.

Either way. Anything else should go into a new thread, not an hijacked one.