Ansible callback not working for Foreman due to SSL Error

EU07 · October 13, 2021, 7:36pm

Hello,
I have a problem with Ansible callback plugin for Foreman. It cannot connect to Foreman.
I’m running Ansible commands on the Foreman machine not a different one.
When I try to run an ansible playbook to get facts to Foreman to provision new machines it shows the tasks are ok, but it cannot connect to Foreman itself with the following error:

HTTPSConnectionPool(host=‘formanek.k219.local’, port=443): Max retries exceeded with url: /api/v2/reports (Caused by SSLError(SSLError(1, ‘[SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:2354)’),))

I changed Foreman SSL using foreman-installer uploading my own CA and certs and I also made sure to add my Root and Intermediate CAs to system truststore but unfortunately that didn’t help.

I’m using the latest Foreman 3.0 and Proxy on CentOS Stream 8 x86_64.
The same goes for Ansible and Remote Execution plugins.
I also installed theforeman ansible collection.

Kindly please help,
Best Regards,
EU07

cps86 · October 11, 2024, 5:22am

Hello

I have exactly the same problem. My foreman installation is using custom certs and it works just fine. Certs are trusted, there are no issues with smart proxy either.
However, I can’t use foreman callback plugin. The error which I am getting is:

[WARNING]: Sending data to Foreman at https://myservername.net failed for myhosttoadd.net:
HTTPSConnectionPool(host='myservername.net', port=443): Max retries exceeded with url: /api/v2/hosts/facts (Caused by SSLError(SSLError(1, '[SSL:
TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:2633)')))
[WARNING]: Sending data to Foreman at https://myservername.net failed for myhosttoadd.net:
HTTPSConnectionPool(host='myservername.net', port=443): Max retries exceeded with url: /api/v2/config_reports (Caused by SSLError(SSLError(1, '[SSL:
TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:2633)')))

First of all, I have no idea why I am getting the unknown CA error as I am using the same CA as foreman and foreman smart-proxy and there are no problems.
Moreover, what’s the purpose of:
verify_certs=0
if it still complains about not trusted certificates?

Marek_Hulan · October 11, 2024, 1:32pm

That almost sounds like the configuration of the certificates is rather ignored. Where exactly did you set the verify_certs=0

cps86 · October 14, 2024, 6:48pm

In ansible.cfg.
When I execute ansible-playbook I get a warning message informing that cert verification is turned off:
[WARNING]: SSL verification of https://myserver.net disabled
So it corectly reads that setting from ansible.cfg and plugin is succesfully loaded.
However, it cannot send any data because of the unknown ca error…

Marek_Hulan · January 2, 2025, 9:36am

The callback should reflect that setting. Could you check what version of the callback you have installed?

cps86 · January 24, 2025, 12:33pm

Hello

I think it happens because I am using different certificates for the httpd server.
So foreman and foreman proxy are using self-signed certs generated by puppet (generated while installing foreman). To not receive website not trusted warning while accessing foreman I replaced certificates for the httpd service for those which are trusted by OS.

In the ansible.cfg I provided the same certs as are in httpd config:

[defaults]
callbacks_enabled=foreman

[callback_foreman]
url = https://myserver.net
report_type=foreman
client_cert=myserver.net.pem
client_key=myserver.net.pem
verify_certs=0

Result:

ansible-playbook -i hosts_to_parse_es_only_2.ini ans1-only-setup.yaml
[WARNING]: SSL verification of https://myserver.net disabled

PLAY [Test] *******************************************************************************************************************************************************************

TASK [debug inventory hostname] ***********************************************************************************************************************************************
[WARNING]: Platform linux on host 10.76.4.34 is using the discovered Python interpreter at /usr/bin/python2.7, but future installation of another Python interpreter could
change the meaning of that path. See Interpreter Discovery — Ansible Core Documentation for more information.
ok: [10.76.4.34]

PLAY RECAP ********************************************************************************************************************************************************************
10.76.4.34 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

[WARNING]: Sending data to Foreman at https://myserver.net failed for 10.76.4.34: HTTPSConnectionPool(host=‘myserver.net’, port=443): Max retries
exceeded with url: /api/v2/hosts/facts (Caused by SSLError(SSLError(1, ‘[SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:2637)’)))
[WARNING]: Sending data to Foreman at https://myserver.net failed for 10.76.4.34: HTTPSConnectionPool(host=‘myserver.net’, port=443): Max retries
exceeded with url: /api/v2/config_reports (Caused by SSLError(SSLError(1, ‘[SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:2637)’)))

As you can see, even though I disabled certificate verification it fails. Why is that?
I know it’s because I am using different certs for httpd server (I have to), after changing the certs in ansible.cfg to that one from foreman config it works fine (plus I had to change httpd certs too).

cps86 · January 24, 2025, 2:51pm

Hey I fixed this issue. It is totally possible to have diffeerent certs for httpd and foreman but you need to set up SSLCACertificateFile and SSLCARevocationFile in httpd config to point to the Foreman’s CA.

gvde · January 24, 2025, 3:17pm

You should not modify the configuration files directly, but only use foreman-installer options.

Next time you run foreman-installer it’ll change everything back.

cps86 · January 24, 2025, 3:49pm

I know about this, however, I haven’t found a setting which sets up different certs for HTTPD only.

gvde · January 24, 2025, 4:03pm

Generally, I would advise to follow the docs and use the same CA throughout the foreman server and proxies.

If it is really absolutely necessary in your setup only to change the ssl certificates for the httpd server on the foreman server, you should the respective options from the foreman module of the foreman-installer. See help:

    --foreman-server-ssl-ca            Defines Apache mod_ssl SSLCACertificateFile setting in Foreman vhost conf file. (current: "/etc/pki/katello/certs/katello-default-ca.crt")
    --reset-foreman-server-ssl-ca      Reset server_ssl_ca to the default value ("/etc/puppetlabs/puppet/ssl/certs/ca.pem")
...
    --foreman-server-ssl-crl           Defines the Apache mod_ssl SSLCARevocationFile setting in Foreman vhost conf file. (current: "")
    --reset-foreman-server-ssl-crl     Reset server_ssl_crl to the default value ("/etc/puppetlabs/puppet/ssl/crl.pem")

However, I wonder if that wouldn’t break the communication between the foreman proxy and server…