Puppet->foreman: Certificates do not conform to algorithm constraints

Problem:
Puppet (server) log shows an error when trying to update foreman after an agent run. Foreman is not updated

Expected outcome:
Foreman is updated successfully

Foreman and Proxy versions:
Foreman 3.8.0-1
Proxy 3.8.0-1
Both on the same host

Distribution and version:
Rocky 8.9

Other relevant data:
Not using Katello

This was all installed on a clean system, and all dependent packages were installed by foreman-installer (puppetserver7, java-1.8.0-openjdk-headless.x86_64)

https port 433 works as expected, no ssl or cert issues

However when the report processor runs, it throws this error

The certificate is sha256withrsa (2048 bits) and is happily accepted by chrome/edge/safari

I tried tweaking the java.security file to make it more permissive (no change)
I tried turning on as many debugging options as I can find (also no extra info)

How can I get some more meaningful info to debug the issue?
Has anyone seen this before?

Check the system-wide crypto policies, and reenable the default Redhat policy if needed via update-crypto-policies --set DEFAULT.

I know in the little playbook we have to do regular foreman updates, we have to re-enable the default crypto policy, because we have a more restrictive setup of crypto’s we allow for Tomcat/Apache on Foreman, and then re-enable our custom policy and configs after foreman-installer runs.

OK, thanks for that. I updated the java.security file to make sure the settings are honored over the crypto policy.

Now it doesn’t give me any error message at all

[puppetserver] Puppet Report processor failed: Could not send report to Foreman at https://REDACTED.com/api/config_reports: Broken pipe
["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1546:in `transport_request'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/n|
et/http.rb:1490:in `request'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1483:in `block in request'", "uri:classloader:/META-INF|
/jruby.home/lib/ruby/stdlib/net/http.rb:924:in `start'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/http.rb:1481:in `request'", "/opt/pup|
petlabs/puppet/lib/ruby/vendor_ruby/puppet/reports/foreman.rb:69:in `process'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/pr|
ocessor.rb:37:in `block in process'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:54:in `block in processors'", "|
org/jruby/RubyArray.java:1865:in `each'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:51:in `processors'", "/opt/|
puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/report/processor.rb:30:in `process'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indir|
ector/report/processor.rb:14:in `save'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:316:in `save'", "/opt/puppetlabs/|
puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:181:in `do_save'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/|
http/api/indirected_routes.rb:53:in `block in call'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'", "/opt/puppetlabs|
/puppet/lib/ruby/vendor_ruby/puppet.rb:289:in `override'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:52:|
in `call'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/server/v3.rb:17:in `block in wrap'", "/opt/puppetlabs/puppet/lib/ruby/v|
endor_ruby/puppet/network/http/route.rb:82:in `block in process'", "org/jruby/RubyArray.java:1865:in `each'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ru|
by/puppet/network/http/route.rb:81:in `process'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'", "/opt/pupp|
etlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:88:in `process'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handl|
er.rb:86:in `block in process'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:69:in `block in with_request_profiling'", "|
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppe|
t/util/profiler.rb:51:in `profile'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:65:in `with_request_profiling'", "/opt/|
puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:85:in `block in process'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/net|
work/http/handler.rb:92:in `respond_to_errors'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:84:in `process'", "uri:clas|
sloader:/puppetserver-lib/puppet/server/master.rb:69:in `block in handleRequest'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in |
`override'", "/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:289:in `override'", "uri:classloader:/puppetserver-lib/puppet/server/master.rb:68:in|
 `handleRequest'"]                                                                                                                                      |

Just for completeness, this was still due to a cert error (just not the same one)

This is now resolved

Just wanted to add a note here. Having moved from a working instance of Foreman on Linux 8 to the same certs failing on Linux 9. I had the same issues with Puppet sending reports back to Foreman and getting the same error above.

It turns out the bundled/chain of certs that was sent to me from the SSL/TLS provider was including some older certs using sha1WithRSAEncryption. I had to break down each cert in /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem and scan them individually with openssl x509 -text -in test.pem to work out which cert was causing the issues. Having removed the older sha1 certs, reports started working again.

The policy seemed to be a bit more forgiving in Linux System 8. Linux 9 seemed to be a little stricter with how Java connects with httpd/Apache. Rather strangely, Apache didn’t have a problem working with the certs.