Unable to get Remote Execution plugin fully working when Foreman has different Web UI SSL certs

Problem:
My remote execution jobs fail, although they run successfully - exactly like this guy explains Support #15070: Task not completing, although job runs - Foreman Remote Execution - Foreman.

I learned from the debug logs that, for me, it was failing when the proxy was trying to report the status back to Foreman. The error was
[2018-04-24 16:34:29.495 #6175] ERROR -- SSL_connect returned=1 errno=0 state=error: certificate verify failed (RestClient::SSLCertificateNotVerified)

After digging for quite a while, I stumbled on http://projects.theforeman.org/issues/15530, and learned that my proxy client was using the wrong cert because the Foreman server is uses different certs (signed by our in-house CA) for it’s WebUI. This here helped me realize that:
SSL settings for client authentication against Foreman. If undefined, the values from general SSL options are used instead. Mainly useful when Foreman uses different certificates for its web UI and for smart-proxy requests. :foreman_ssl_ca: /etc/foreman-proxy/foreman_ssl_ca.pem :foreman_ssl_cert: /etc/foreman-proxy/foreman_ssl_cert.pem :foreman_ssl_key: /etc/foreman-proxy/foreman_ssl_key.pem

Great so I thought I figured it out, I just need to set :foreman_ssl_cert:, :foreman_ssl_key: to what Foreman uses for it’s Web UI - Nope, it still doesn’t work. Now Foreman can’t talk to the proxy. :confused:

My web certificates are configured as per this article: Foreman :: Replacing Foreman's web SSL certificate. and ~everything else in my set up is largely DEFAULTS (foreman-installer) for Foreman 1.17 (everything on the same box; Foreman, Foreman Proxy, Puppet Master).

Expected outcome:
Remote execution jobs should complete.

Foreman and Proxy versions:
1.17

Foreman and Proxy plugin versions:
1.17

Other relevant data:
[e.g. logs from Foreman and/or the Proxy, modified templates, commands issued, etc]

Eh, I made a mistake recalling what I did:

Edit and continue:

Great so I thought I figured it out, I just need to set :foreman_ssl_cert:, :foreman_ssl_key: to what Foreman uses for it’s Web UI - Nope, it still doesn’t work. Now my remote execution jobs fail immediately with SSL certificate with unexpected serial number supplied - :confused:.

My web certificates are configured as per this article: Foreman :: Replacing Foreman's web SSL certificate. and ~everything else in my set up is largely DEFAULTS (foreman-installer) for Foreman 1.17 (everything on the same box; Foreman, Foreman Proxy, Puppet Master).

Through more digging, I guess it appears the certificated must be signed by the same CA (Foreman :: Manual) but that’s where I no longer know what to do - it doesn’t make sense - my Puppet CA is for puppet and not Trusted in browsers, obviously I cannot use it for Foreman’s Web UI cert…

Is it really not possible to make this work when Foreman has different Web UI SSL certs?

So I realized that “both certs signed by the same CA” probably (?) means the Foreman -> Proxy certs should be the same as Proxy -> Foreman, thus I changed :foreman_ssl_cert: and :foreman_ssl_key: to the same certs my Foreman is seemingly able to talk to the proxy with. But alas, no luck still :frowning: I’m back to the original error:

[2018-04-24 17:39:46.382 #965] ERROR -- SSL_connect returned=1 errno=0 state=error: certificate verify failed (RestClient::SSLCertificateNotVerified)

So is this a supported feature?

Have you tried using the installer to configure your certs for you as per this RHEL doco?

https://access.redhat.com/documentation/en-us/red_hat_satellite/6.2/html-single/installation_guide/index#configuring_satellite_server_with_custom_server_certificate

I’ve been testing that process out and it’s looking promising. I’ve found the Foreman/Katello SSL setup pretty complex and a pain to deal with manually.

I always configure everything via the installer for just that reason.

Has this ever been resolved? I have the same issues on foreman 2.0, 2.1, 2.2, 2.3 and 2.4 (yes I tried all versions)