A little more detail. I tried removing the classes in dev, no good they don’t then show back up. I tried changing the puppet version in the foreman-proxy puppet.yaml and now get:
ERF12-4115 [ProxyAPI::ProxyException]: Unable to get classes from Puppet for dev ([RestClient::NotAcceptable]: 406 Not Acceptable) for proxy https://puppet:8443/puppet
I’m imaging it’s not finding our code directory anymore but I’m a little unsure yet again why that would go away in the config with the upgrade.
That looks like the logs from Foreman, I asked for the logs from the Proxy, which are generated when you click Import Classes (since it’s the Proxy’s job to generate the list of classes and return it to Foreman so you can choose what to accept).
That said, a 406 Unacceptable is almost always the Puppetserver rejecting the API request from the Proxy - did the Puppetserver get upgraded or changed in some way as part of the upgrade?
Puppetserver was upgraded to 5.1.4. I changed the puppet.yml back to 4.1.3 and now it works, but wants to remove all the classes again. The logs didn’t show anything that looked interesting to me, here: https://pastebin.com/z7Vk1pXM
does that help any?
This tells us all the settings it was initialized with. It looks sane, but check the addresses, certs etc are right.
So, assuming clients are still finding the classes Foreman wants to delete (that is, the puppetserver still finds them and can compile catalogs) then the next question is why the proxy can’t see them. It’s an API call to the puppetserver, so it ought to work the same way as for the puppetserver itself. I may have to defer to @Dmitri_Dolguikh for help on this one… Dmitri, is there any difference in the parser for Puppet 4 vs Puppet 5?
is there any difference in the parser for Puppet 4 vs Puppet 5?
Not sure – we now rely on puppet server and its api to retrieve (and parse) classes.
I would suggest to try manually retrieve classes for environments in questions using curl or similar. The api endpoint is at “puppet/v3/environment_classes?environment=<environment_name>”.
I’m seeing 403’s from the puppet server when Puppet 5 was used, please make sure either puppet recognizes certificate used by the smart-proxy for environment classes calls, or puppet authorization is configured to allow unauthenticated calls from smart-proxy.
As far as we’re able to understand, there’s no authorization issues. We added in
name: “puppetlabs environment classes”
and no change. As far as we can tell - the logs don’t show any errors, just multiple re-tries and no found classes.
Thanks, that was the issue. So it does look like a permission issue then. What I don’t get it what configuration changed here from puppetserver 4 to 5. We tried what’s shown here: http://thinlight.org/2017/07/29/foreman-and-puppet-5-the-hard-way/ - as I said I added into auth.conf the above. Still getting errors. Is there a document my google fu has failed to find or pointers as to what needs to be in auth.conf for the proxy to be allowed?
We had just followed the foreman manual upgrade instructions to get us in this mess. We’re on Scientific Linux 7.4 - just realized I missed that, though I doubt that will help much.
Ok, maybe not. I just did: curl -k https://puppet:8140/puppet/v3/environment_classes?environment=dev
and got a listing of what looks like classes and ?maybe? parameters. https://pastebin.com/jjhj9KF0
is some of it slightly sanitized.
For Auth.conf. we just switched over to the one shipped with the rpm, save the addition I listed. I’ll go ahead and pastebin it here: https://pastebin.com/KVf4zJiA
Ok, I think we had a bunch of moving parts, but it seems fixed now. The big need was the change to auth.conf listed above, AND the puppet version in the puppet.yml - neither of which were changed for us from the RPMs or following the upgrade instructions. At the very least, I’d suggest calling out the foreman-proxy puppet.yml config on upgrades to new versions of foreman… If only because that may well coincide with a puppet upgrade and that puppet won’t mention needing it for foreman integration as it has nothing to do with them specifically.
Out of interest, did you re-run the installer after the upgrade? If I recall, the installer will write the current puppet version to the proxy config (unless overridden of course), so if it didn’t do that we’d need to investigate…
In the katello context we’re stricter about advising rerunning the installer when upgrading. At the Foreman hackday (not formally announced yet but planned for 7th of February in Ghent) I plan to have a discussion around the installer and continuous management. The installer is an aspect of this so I’ll include it there.