RestClient::RequestTimeout from proxy when foreman is under load(100% CPU)

The performance has never been great, but if we try to create more than 5
hosts at once (VMWare) we get into trouble. We are using python-foreman to
interact with foreman REST API. The host create POST throws
"HTTPConnectionPool(host=<foreman_address>, port=<foreman_port>): Read
timed out. (read timeout=600)". The foreman processes are chocking the CPU
cores at 100% and in the logs I can see many exception, like:
task failed with the following error: ERF12-6899
[ProxyAPI::ProxyException]: Unable to set DHCP entry
failed to detect boot server: ERF12-5139 [ProxyAPI::ProxyException]: Unable
to detect TFTP boot server ([RestClient::RequestTimeout]: RequestTimeout)
for proxy
task failed with the following error: ERF12-4681
[ProxyAPI::ProxyException]: Unable to delete PuppetCA autosign for
We run foreman 1.8.2 and upgrading it in the past proved a nightmare so I
was hoping for a patch.
Thanks

Hey,

> We run foreman 1.8.2 and upgrading it in the past proved a nightmare so I
> was hoping for a patch.

can you send TTIN signal to Foreman processes and acquire backtrace
information in the passenger log?

Also if you can upload foreman-debug via -u option that gives more
flexibility when investigating.

ยทยทยท -- Later, Lukas #lzap Zapletal