Foreman-proxy unable to import environments : name or service not known

Hi all,

I’m having an issue with Foreman smart-proxy. I have one configured. and able to use it using apis such as /features or /puppet/ca. However, when I try to import classes using UI, I am getting the error
Error: ERF12-2749 [ProxyAPI::ProxyException]: Unable to get environments from Puppet ([RestClient::NotAcceptable]: 406 Not Acceptable) for proxy https://elb-sydpubmg-forprx-760250653.ap-southeast-2.elb.amazonaws.com:8443/puppet

I tried doing curl command and getting:

* About to connect() to elb-sydpubmg-forprx-760250653.ap-southeast-2.elb.amazonaws.com port 8443 (#0)
*   Trying 52.63.99.64...
* Connected to elb-sydpubmg-forprx-760250653.ap-southeast-2.elb.amazonaws.com (52.63.99.64) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate from file
*       subject: CN=puppetmaster.ec2.internal
*       start date: Jan 25 09:26:31 2018 GMT
*       expire date: Jan 25 09:26:31 2023 GMT
*       common name: puppetmaster.ec2.internal
*       issuer: CN=Puppet CA: puppetmaster.ec2.internal
* SSL connection using TLS_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=syd-puppetslave.ap-southeast-2.compute.internal
*       start date: Dec 02 06:10:02 2017 GMT
*       expire date: Dec 02 06:10:02 2022 GMT
*       common name: syd-puppetslave.ap-southeast-2.compute.internal
*       issuer: CN=Puppet CA: puppetmaster.ec2.internal
> GET /puppet/environments HTTP/1.1
> User-Agent: curl/7.29.0
> Host: elb-sydpubmg-forprx-760250653.ap-southeast-2.elb.amazonaws.com:8443
> Accept:pplication/json
>
< HTTP/1.1 406 Not Acceptable
< Content-Type: application/json
< Content-Length: 74
< X-Content-Type-Options: nosniff
< Server:
< Date: Sat, 03 Feb 2018 17:39:51 GMT
< Connection: Keep-Alive
<
* Connection #0 to host elb-sydpubmg-forprx-760250653.ap-southeast-2.elb.amazonaws.com left intact
Failed to list puppet environments: getaddrinfo: Name or service not known

Same are shown in the proxy.log file. It seems everything I need works fine except when the API requests for /puppet/environments

I have in my auth.conf snippet

{
    match-request: {
    path: "/puppet/v3/environment_classes"
       type: path
       method: get
    }
    allow: "*"
    sort-order: 500
    name: "puppetlabs environment classes"
},

{
            match-request: {
                path: "/puppet/v3/environments"
                type: path
                method: get
            }
            allow: "*"
            sort-order: 500
            name: "puppetlabs environments"
        },

and also

path /puppet/v3/environment_classes
method find
allow *

Any help or direction in troubleshooting this would be much appreciated

Jubix

is typicaclly a DNS issue. From the host that the Smart Proxy is on, what does ping elb-sydpubmg-forprx-760250653.ap-southeast-2.elb.amazonaws.com return? If that fails due a DNS lookup, that would confirm. You may need a hosts file entry for it. It might also be an issue with the fact this is an AWS instance - AWS by default will change your hostname when you reboot a host unless and until you configure it not to so make sure you still have the same hostname as when you restarted and you did not reboot the instance and be sure to make sure the hostname is static going forward.

Hi James,
Thanks for the reply.

I did a telnet instead as i dont have ping enabled on this instance.

Trying 52.63.99.64...
Connected to elb-sydpubmg-forprx-760250653.ap-southeast-2.elb.amazonaws.com.
Escape character is '^]'.

What is puzzling to me is that doing a curl for /features and /puppet/ca is executed successfully such as below. Leading me to believe that the smart proxy is working, at least partly. Only when doing /environments and /environments/classes is showing the Name or service not known error. See sample below:

curl -k -X GET -H Accept:pplication/json https://elb-sydpubmg-forprx-760250653.ap-southeast-2.elb.amazonaws.com:8443/features --tlsv1 --cacert /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem --cert /etc/puppetlabs/puppet/ssl/certs/puppetmaster.ec2.internal.pem --key /etc/puppetlabs/puppet/ssl/private_keys/puppetmaster.ec2.internal.pem
["logs","puppet","puppetca","tftp"]

Yes, I’ve configured my instances to retain their hostnames upon reboot.

Could you post debug-level log from the smart-proxy please?

You still will want to try a hosts file entry and confirm that your hosts file is correct and that your hostname is set correctly with:

  • hostnamectl
  • in your hosts file
  • in /etc/sysconfig/network
  • with systemctl

There are numerous ways of getting a hostname and different tools and underlying languages determine this information differently, so you can sometimes get different answers depending on the tool you consult.

1 Like

Here’s snippet of logs when I tried restarting the smart-proxy service:

D, [2018-02-03T20:07:37.907010 ] DEBUG -- : Rack::Handler::WEBrick is mounted on /.
I, [2018-02-03T20:07:37.907141 ]  INFO -- : WEBrick::HTTPServer#start: pid=29946 port=8443
W, [2018-02-03T20:07:37.907957 ]  WARN -- : Failed to initialize puppet class cache, deferring initialization. Is puppetserver running?
E, [2018-02-03T20:07:41.313809 ] ERROR -- : OpenSSL::SSL::SSLError: SSL_accept SYSCALL returned=5 errno=0 state=SSLv2/v3 read client hello A
        /usr/share/ruby/openssl/ssl.rb:280:in `accept'
E, [2018-02-03T20:07:57.266983 ] ERROR -- : OpenSSL::SSL::SSLError: SSL_accept SYSCALL returned=5 errno=0 state=SSLv2/v3 read client hello A
        /usr/share/ruby/openssl/ssl.rb:280:in `accept'
E, [2018-02-03T20:08:11.313031 ] ERROR -- : OpenSSL::SSL::SSLError: SSL_accept SYSCALL returned=5 errno=0 state=SSLv2/v3 read client hello A

From foreman-ui, i tried importing classes a couple of times and getting the same error

D, [2018-02-03T20:30:49.264921 ] DEBUG -- : accept: 10.0.90.230:20531
D, [2018-02-03T20:30:49.473412 ] DEBUG -- : Rack::Handler::WEBrick is invoked.
E, [2018-02-03T20:30:49.593555 ] ERROR -- : Failed to list puppet environments: getaddrinfo: Name or service not known
D, [2018-02-03T20:30:49.593678 ] DEBUG -- : Failed to list puppet environments: getaddrinfo: Name or service not known
I, [2018-02-03T20:30:49.594906 ]  INFO -- : 10.0.90.230 - - [03/Feb/2018:20:30:49 +0000] "GET /puppet/environments HTTP/1.1" 406 74 0.1159

D, [2018-02-03T20:30:49.802806 ] DEBUG -- : close: 10.0.90.230:20531
E, [2018-02-03T20:30:57.267218 ] ERROR -- : OpenSSL::SSL::SSLError: SSL_accept SYSCALL returned=5 errno=0 state=SSLv2/v3 read client hello A
        /usr/share/ruby/openssl/ssl.rb:280:in `accept'
D, [2018-02-03T20:30:59.486626 ] DEBUG -- : accept: 10.0.92.190:33614
D, [2018-02-03T20:30:59.694011 ] DEBUG -- : Rack::Handler::WEBrick is invoked.
E, [2018-02-03T20:30:59.696911 ] ERROR -- : Failed to list puppet environments: getaddrinfo: Name or service not known
D, [2018-02-03T20:30:59.696963 ] DEBUG -- : Failed to list puppet environments: getaddrinfo: Name or service not known

Agents connected to my puppetserver are updating fine. calling /features /logs /puppet/ca are all working fine when invoked from smart-proxy. Only under environments and importing classes this is failing.

I have another puppetserver that is able to import classess successfully. The only difference I see based on proxy logs is the initialization message:

I, [2018-02-03T20:06:46.696808 ]  INFO -- : WEBrick::HTTPServer#start: pid=24062 port=8443
I, [2018-02-03T20:06:46.827286 ]  INFO -- : Finished puppet class cache initialization

If I could fix the warning regarding puppet class cache initialization, perhaps it will get fixed. I’m comparing configurations between the 2 servers but no luck so far.

Failed to initialize puppet class cache, deferring initialization. Is puppetserver running?

Is not a failure, rather a warning that cache initialization has been delayed. The root cause is " getaddrinfo: Name or service not known" – puppet server dns name can’t be resolved.

1 Like

Found the issue.
I tried a curl on the smart proxy on localhost:8443 to eliminate issues regarding the server DNS and it was still failing. I reviewed my the files under settings.d and finally saw a misconfiguration.

It was a careless copy paste when trying to configure this server. I looked closely and i pasted the pem file name instead of the DNS as shown.

$cat /etc/foreman-proxy/settings.d/puppet_proxy_puppet_api.yml
:puppet_url: https://<hostname>.pem:8140

Thanks for the help guys, for helping me focus on the DNS and puppetserver connectivity.

2 Likes