SSL Errors with proxy and installer

when I am running the foreman-installer for the smart-proxy I still get the error:

/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[FQDN/ensure: change from absent to present failed: ProxyFQDN cannot be registered: Unable to communicate with the proxy: ERF12-2530 [ProxyAPI::ProxyException]: Unable to detect features ([RestClient::SSLCertificateNotVerified]: SSL_connect returned=1 errno=0 state=error: certificate verify failed) for proxy https://FQDN:9090/features Please check the proxy is configured and running on the host.

I am using a centos7 image I did the troubleshooting and the solution you added here but still the same issue. Any idea?
Help Please.

1 Like

It’s not a great idea to reply to a topic that’s marked as solved - as anyone filtering on “unsolved topics” won’t see your question. I’ve moved it to a new topic for you :slight_smile:

Thanks, Any idea for this type of issue?? hint on the foreman-master I have generated the certs in .tar file using the below command:

foreman-proxy-certs-generate --foreman-proxy-fqdn “FQDN” --certs-tar “/root/FQDN.com-certs.tar” --certs-update-server

Sorry, no. As I said in the older topic, SSL is not something I’m good at. If the notes there didn’t help, then we’ll have to wait for someone with more clue to jump in :wink:

Are you installing Katello? If so there may be a self signed Katello CA cert and a Puppet CA cert that you will need to add to your hosts trusted certificate store. Look in /etc/pki/katello/certs.

I also found that using /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem rather than /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt worked. And that I had to clear out the certs from /etc/foreman-proxy/ after each failed install.

Katello is already installed the foreman-installer scenario is to install smart-proxy after generating the certs on foreman-server it generate a command to use to install smart-proxy:

foreman-installer --scenario foreman-proxy-content\

                --foreman-proxy-content-parent-fqdn           "foreman-serverFQDN.com"\
                --foreman-proxy-register-in-foreman           "true"\
                --foreman-proxy-foreman-base-url              "https://foreman-serverFQDN.com"\
                --foreman-proxy-trusted-hosts                 "foreman-serverFQDN.com"\
                --foreman-proxy-trusted-hosts                 "Smart-proxyFQDN.com"\
                --foreman-proxy-oauth-consumer-key            "Oauth Key provided here"\
                --foreman-proxy-oauth-consumer-secret         "Oauth Secret provided here"\
                --foreman-proxy-content-pulp-oauth-secret     "Pulp Oauth Secret provided here"\
                --foreman-proxy-content-certs-tar             "/root/Smart-proxy-certs.tar"\
                --puppet-server-foreman-url                   "https://foreman-serverFQDN.com"

After running the command above I get the error I have posted

Thanks for posting that. Unfortunately it didn’t help. I still get the same error:

/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[foreman.com]/ensure: change from absent to present failed: Proxy fqdn.com cannot be registered: Unable to communicate with the proxy: ERF12-2530 [ProxyAPI::ProxyException]: Unable to detect features ([RestClient::SSLCertificateNotVerified]: SSL_connect returned=1 errno=0 state=error: certificate verify failed) for proxy https:/foreman-proxy.com:9090/features Please check the proxy is configured and running on the host.

I had to add both katello-default-ca.crt and katello-server-ca.crt to the trusted certificate store on the Katello master and all Smart Proxy servers. Then configure the foreman-proxy SSL settings so that /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem was the CA certificate.

You may have a different issue but that was the solution when I was seeing this error message.

I have done the steps you have mentioned above and still facing the same issue with same error.

Any other ideas here please?

Can you attempt to join the Smart Proxy and provide the following logs please?

From your Katello Master Server:

/var/log/foreman/production.log
/var/log/httpd/foreman-ssl_access_ssl.log
/var/log/httpd/foreman-ssl_error_ssl.log 

From your Smart Proxy:

/var/log/foreman-proxy/proxy.log

Also the output of the following on your Katello Master server.

openssl verify -CAfile /etc/foreman/proxy_ca.pem /etc/foreman/client_cert.pem

2 Likes

for proxy logs:

cat /var/log/foreman-proxy/proxy.log

E, [2018-06-19T15:50:09.802321 ] ERROR -- : OpenSSL::SSL::SSLError: SSL_accept returned=1 errno=0 state=SSLv3 read client certificate A: tlsv1 alert unknown ca
	/usr/share/ruby/openssl/ssl.rb:280:in `accept'
E, [2018-06-19T15:50:28.730849 ] ERROR -- : OpenSSL::SSL::SSLError: SSL_accept SYSCALL returned=5 errno=0 state=SSLv3 read client certificate A
	/usr/share/ruby/openssl/ssl.rb:280:in `accept'
I, [2018-06-19T15:50:32.289672 ]  INFO -- : 10.200.10.52 - - [19/Jun/2018:15:50:32 -0500] "GET /features HTTP/1.1" 200 58 0.0005
I, [2018-06-19T15:50:32.762544 ]  INFO -- : 10.200.10.52 - - [19/Jun/2018:15:50:32 -0500] "GET /favicon.ico HTTP/1.1" 404 27 0.0004
I, [2018-06-19T15:50:53.708101 ]  INFO -- : 10.200.10.52 - - [19/Jun/2018:15:50:53 -0500] "GET /features HTTP/1.1" 200 58 0.0007
I, [2018-06-19T15:50:55.640197 ]  INFO -- : 10.200.10.52 - - [19/Jun/2018:15:50:55 -0500] "GET /features HTTP/1.1" 200 58 0.0007

/var/log/httpd/foreman-ssl_access_ssl.log:

10.7.2.5 - - [19/Jun/2018:15:36:09 -0500] "GET /rhsm/ HTTP/1.1" 200 1841 "-" "RHSM/1.0 (cmd=yum)"
10.7.2.5 - - [19/Jun/2018:15:36:10 -0500] "GET /rhsm/consumers/1ca42f5c-cb2c-4869-affa-11c58c802d07/content_overrides HTTP/1.1" 200 2 "-" "RHSM/1.0 (cmd=yum)"
10.7.2.5 - - [19/Jun/2018:15:36:53 -0500] "GET /rhsm/ HTTP/1.1" 200 1841 "-" "RHSM/1.0 (cmd=yum)"
10.7.2.5 - - [19/Jun/2018:15:36:53 -0500] "GET /rhsm/consumers/1ca42f5c-cb2c-4869-affa-11c58c802d07/content_overrides HTTP/1.1" 200 2 "-" "RHSM/1.0 (cmd=yum)"
10.7.2.5 - - [19/Jun/2018:15:36:55 -0500] "GET /rhsm/ HTTP/1.1" 200 1841 "-" "RHSM/1.0 (cmd=yum)"
10.7.2.5 - - [19/Jun/2018:15:36:55 -0500] "GET /rhsm/consumers/1ca42f5c-cb2c-4869-affa-11c58c802d07/content_overrides HTTP/1.1" 200 2 "-" "RHSM/1.0 (cmd=yum)"
10.7.2.7 - - [19/Jun/2018:15:40:56 -0500] "POST /api/hosts/facts HTTP/1.1" 403 109 "-" "Ruby"
10.7.2.7 - - [19/Jun/2018:15:41:01 -0500] "GET /node/fra3lns16p.1worldsync.com?format=yml HTTP/1.1" 403 - "-" "Ruby"
10.7.2.7 - - [19/Jun/2018:15:41:03 -0500] "POST /api/hosts/facts HTTP/1.1" 403 109 "-" "Ruby"
10.7.2.7 - - [19/Jun/2018:15:41:03 -0500] "GET /node/fra3lns16p.1worldsync.com?format=yml HTTP/1.1" 403 - "-" "Ruby"
10.7.2.7 - - [19/Jun/2018:15:41:04 -0500] "POST /api/config_reports HTTP/1.1" 403 118 "-" "Ruby"
10.4.2.3 - - [19/Jun/2018:15:42:01 -0500] "GET /rhsm/consumers/5e7a9169-c6d0-49b4-a430-4edca6edb82b HTTP/1.1" 200 18584 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.3 - - [19/Jun/2018:15:42:01 -0500] "GET /rhsm/consumers/5e7a9169-c6d0-49b4-a430-4edca6edb82b/certificates/serials HTTP/1.1" 200 186 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.3 - - [19/Jun/2018:15:42:03 -0500] "GET /rhsm/consumers/5e7a9169-c6d0-49b4-a430-4edca6edb82b/certificates/serials HTTP/1.1" 200 186 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.3 - - [19/Jun/2018:15:42:04 -0500] "GET /rhsm/consumers/5e7a9169-c6d0-49b4-a430-4edca6edb82b HTTP/1.1" 200 18584 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.3 - - [19/Jun/2018:15:42:04 -0500] "GET /rhsm/ HTTP/1.1" 200 1841 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.3 - - [19/Jun/2018:15:42:04 -0500] "GET /rhsm/consumers/5e7a9169-c6d0-49b4-a430-4edca6edb82b/content_overrides HTTP/1.1" 200 3976 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.7.2.7 - - [19/Jun/2018:15:50:08 -0500] "GET /api/v2/smart_proxies?search=name=%22fra3lns16p.1worldsync.com%22 HTTP/1.1" 200 145 "-" "OAuth gem v0.5.1"
10.7.2.7 - - [19/Jun/2018:15:50:09 -0500] "POST /api/v2/smart_proxies HTTP/1.1" 422 745 "-" "OAuth gem v0.5.1"
10.4.2.15 - - [19/Jun/2018:15:53:01 -0500] "GET /rhsm/consumers/81f74357-a9e3-44fc-9652-82f2bacc2696 HTTP/1.1" 200 19320 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.15 - - [19/Jun/2018:15:53:12 -0500] "GET /rhsm/consumers/81f74357-a9e3-44fc-9652-82f2bacc2696/compliance HTTP/1.1" 200 6959 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.3.2.52 - - [19/Jun/2018:15:53:17 -0500] "GET /node/ord2lns15p.1worldsync.com?format=yml HTTP/1.1" 200 819 "-" "Ruby"
10.3.2.52 - - [19/Jun/2018:15:53:18 -0500] "POST /api/hosts/facts HTTP/1.1" 201 911 "-" "Ruby"
10.3.2.52 - - [19/Jun/2018:15:53:19 -0500] "GET /node/ord2lns15p.1worldsync.com?format=yml HTTP/1.1" 200 819 "-" "Ruby"
10.3.2.52 - - [19/Jun/2018:15:53:20 -0500] "POST /api/config_reports HTTP/1.1" 201 718 "-" "Ruby"
10.4.2.15 - - [19/Jun/2018:15:53:22 -0500] "GET /rhsm/consumers/81f74357-a9e3-44fc-9652-82f2bacc2696/certificates/serials HTTP/1.1" 200 217 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.15 - - [19/Jun/2018:15:54:16 -0500] "GET /rhsm/consumers/81f74357-a9e3-44fc-9652-82f2bacc2696/certificates/serials HTTP/1.1" 200 217 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.15 - - [19/Jun/2018:15:54:26 -0500] "GET /rhsm/consumers/81f74357-a9e3-44fc-9652-82f2bacc2696 HTTP/1.1" 200 19320 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.15 - - [19/Jun/2018:15:54:37 -0500] "GET /rhsm/ HTTP/1.1" 200 1841 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.15 - - [19/Jun/2018:15:54:47 -0500] "GET /rhsm/consumers/81f74357-a9e3-44fc-9652-82f2bacc2696/content_overrides HTTP/1.1" 200 3976 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"
10.4.2.15 - - [19/Jun/2018:15:54:57 -0500] "GET /rhsm/consumers/81f74357-a9e3-44fc-9652-82f2bacc2696/release HTTP/1.1" 200 24 "-" "RHSM/1.0 (cmd=rhsmcertd-worker)"

/var/log/httpd/foreman-ssl_error_ssl.log:

[Tue Jun 19 14:26:30.891836 2018] [ssl:error] [pid 30484] [client 10.4.7.171:58297] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/35/repositories/511
[Tue Jun 19 14:26:31.031703 2018] [ssl:error] [pid 30480] [client 10.4.7.171:58292] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/35/repositories/511
[Tue Jun 19 14:26:31.277249 2018] [ssl:error] [pid 31033] [client 10.4.7.171:58304] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/35/repositories/511
[Tue Jun 19 14:26:31.412148 2018] [ssl:error] [pid 31875] [client 10.4.7.171:58299] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/35/repositories/511
[Tue Jun 19 14:26:31.514233 2018] [ssl:error] [pid 30479] [client 10.4.7.171:58301] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/35/repositories/511
[Tue Jun 19 14:26:31.618518 2018] [ssl:error] [pid 30483] [client 10.4.7.171:58306] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/35/repositories/511
[Tue Jun 19 14:26:34.977806 2018] [ssl:error] [pid 30477] [client 10.4.7.171:58310] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/content_views?page=1&per_page=20&sortBy=name&sortOrder=ASC
[Tue Jun 19 14:27:00.459276 2018] [ssl:error] [pid 30479] [client 10.4.7.171:58336] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:00.459838 2018] [ssl:error] [pid 30481] [client 10.4.7.171:58328] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:00.587073 2018] [ssl:error] [pid 30483] [client 10.4.7.171:58322] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:00.587525 2018] [ssl:error] [pid 30478] [client 10.4.7.171:58329] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:00.707318 2018] [ssl:error] [pid 30482] [client 10.4.7.171:58338] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:00.750396 2018] [ssl:error] [pid 30477] [client 10.4.7.171:58337] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:00.900782 2018] [ssl:error] [pid 39253] [client 10.4.7.171:58344] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:00.926594 2018] [ssl:error] [pid 39252] [client 10.4.7.171:58341] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:01.008033 2018] [ssl:error] [pid 30479] [client 10.4.7.171:58348] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 14:27:01.084609 2018] [ssl:error] [pid 30481] [client 10.4.7.171:58349] AH02261: Re-negotiation handshake failed, referer: https://foreman.1worldsync.com/products/26/repositories/455
[Tue Jun 19 15:12:37.811827 2018] [ssl:error] [pid 31875] [client 10.7.2.7:50772] AH02039: Certificate Verification: Error (20): unable to get local issuer certificate
[Tue Jun 19 15:49:57.658426 2018] [ssl:error] [pid 30483] [client 10.7.2.7:50794] AH02039: Certificate Verification: Error (20): unable to get local issuer certificate
[Tue Jun 19 15:50:13.217583 2018] [ssl:error] [pid 30477] [client 10.7.2.7:50830] AH02039: Certificate Verification: Error (20): unable to get local issuer certificate

openssl verify -CAfile /etc/foreman/proxy_ca.pem /etc/foreman/client_cert.pem:

etc/foreman/client_cert.pem: C = US, ST = North Carolina, O = FOREMAN, OU = PUPPET, CN = ord2lns15p.1worldsync.com
error 20 at 0 depth lookup:unable to get local issuer certificate

The CA certificate, /etc/foreman/proxy_ca.pem, did not sign the certificate, /etc/foreman/client_cert.pem, so Smart Proxy authentication is failing.

You can get more info about both with "openssl x509 -text -in "

I have seen this when using custom certificates, is that the case here?
What options are you passing to foreman-installer?

In most cases the certs in /etc/foreman should be the same as the puppet certs.

Can you see if /etc/puppetlabs/puppet/ssl/certs/ca.pem or /etc/pki/katello/certs/katello-default-ca.crt will verify /etc/foreman/client_cert.pem?

1 Like

for /etc/puppetlabs/puppet/ssl/certs/ca.pem on Foreman Master is different than the smart-proxy host.
for /etc/pki/katello/certs/katello-default-ca.crt is the same cert.
I didn’t use openssl x509 cert creation for the smart proxy. I use the command: foreman-proxy-certs-generate

and for the foreman-installer to deploy the smart proxy it is mentioned above in the comments.
I moved the ca.pem and created a new one matching the foreman master ca.pem but still ended up with the same error.