Foreman-proxy cannot get environments - url not found

We recently had an admin make some changes to foreman in updating certificates. One change that was made was to move foreman-proxy to listen on localhost:8000. Everything is working as expected with one exception and that involves importing puppet environments.

First, if you navigate to configure -> classes in foreman and then click on Import Environments, you will get an error message "Unable to get environments from Puppet ". You can get the same message from a couple of other places in the gui. From the foreman cli, running ‘foreman-rake puppet:import:puppet_classes’ will also give you the same error message.

Both of the above actually are doing a web call to foreman-proxy. This can be reproducedwith ‘curl http://127.0.0.1:8000/puppet/environments’, which returns a 404 Not Found error. The proxy works fine however when you run ‘curl http://127.0.0.1:8000/puppet/ca’ or ‘curl http://127.0.0.1:8000/version’.

HOSTNAME: fr-s-sag-forman.ncifcrf.gov
OS: redhat
RELEASE: CentOS Linux release 7.8.2003 (Core)
FOREMAN: 1.15.6
RUBY: ruby 2.0.0p648 (2015-12-16) [x86_64-linux]
PUPPET: 4.10.12
DENIALS: 0

In the proxy log file we get the following every time we try to import the environment

D, [2020-05-21T10:45:29.311591 ] DEBUG – : accept: 127.0.0.1:54400
D, [2020-05-21T10:45:29.313185 ] DEBUG – : Rack::Handler::WEBrick is invoked.
I, [2020-05-21T10:45:29.314537 ] INFO – : 127.0.0.1 - - [21/May/2020:10:45:29 -0400] “GET /puppet/environments HTTP/1.1” 404 27 0.0007
D, [2020-05-21T10:45:29.314974 ] DEBUG – : close: 127.0.0.1:54400

Hello and welcome here.

To me, it looks like you might not have puppet or puppetca smart-proxy modules disabled, thus 404.

# find /etc/foreman-proxy -name *yml | xargs grep enabled: | grep puppet
/etc/foreman-proxy/settings.d/puppet.yml::enabled: https
/etc/foreman-proxy/settings.d/puppetca.yml::enabled: https

In my case these endpoints are enabled on HTTPS which is port 8443 or 9090 depending on installation scenario. Then you can only connect to these ports. Make sure your proxy is registered via HTTPS port, the HTTP endpoint is only used for some services like kickstart or HTTP booting but it should not definitely be used to communicate puppet CA certs.

@ekohl will know more about this.

Both puppet and puppetca smart-proxy modules are enabled. Given that smart-proxy is listening only on localhost, is it really critical to use https instead of http for any service?