We recently added two new puppetmasters then facts and reports uploads fail with Error 500 randomly.
2022-02-15T10:44:37.372Z WARN [qtp345670706-37] [c.p.p.ShellUtils] Executed an external process which logged to STDERR: During the fact upload the server responded with: 500 Internal Server Error. Error is ignored and the execution continues. <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>500 Internal Server Error</title> </head><body> <h1>Internal Server Error</h1> <p>The server encountered an internal error or misconfiguration and was unable to complete your request.</p> <p>Please contact the server administrator at [no address given] to inform them of the time this error occurred, and the actions you performed just before this error.</p> <p>More information about this error may be available in the server error log.</p> </body></html>
Looking for help how to better debug this to find the root cause. This is only happening to the two new puppetmasters built on two different networks(firewalls seem to be fine).We already have like 10 more on other regions working without any issues. The errors are random, sometimes puppet runs sometimes it takes very long and foreman comes back with that Error 500. I ran simple curl to the foreman URL from them and it responds fine every time. Again I’m looking for help on how to know more what’s causing foreman to take that long an error so randomly.
puppet run without errors
Foreman and Proxy versions:
Foreman and Proxy plugin versions:
Distribution and version:
Other relevant data: