This post describes steps to to enable monitoring and controlling of the puma service in production environment on RPM based distribution. Currently it’s not working for various reasons, but with newer versions of puma, it should, so dumping here for the future use.
- enable the control socket for puma protected with the token
echo "activate_control_app\nstate_path '/run/foreman/puma.stats'" >> ~foreman/config/puma/production.rb
note in development environment, you can add this to config/puma.rb (new file)
- restart the puma server
systemctl restart foreman
- read the token and url from the stats file, find the private tmp directory for the Foreman service
cat /run/foreman/puma.stats ls -d /tmp/systemd-private-*foreman*
given the Foreman’s tmp is using private tmp directory, you must construct the unix socket URL manually, take the
unix:// + private_tmp + control_url without the unix:// prefix
so for example the final url could be
this URL with the token needs to be specified for each pumactl command
- You can now use the pumactl commandline utility, namely stats, gc-stats can be useful. On RPM-based production setups, one can run it by
scl enable tfm 'pumactl stats --control-url unix:///tmp/systemd-private-70696e8a3a6748d197fe09749efc0bc3-foreman.service-PjLEQU/tmp/puma-status-1618483682415-26762 --control-token 12ff5f4af3e28b509774ffeb4b7c7790'
Today, you’d receive
too long unix socket path (112bytes given but 108bytes max) error, but with newer Puma than we ship today, it seems to work fine.
puma-status, monitoring the load
There’s also a puma-status gem one can install, pointing it to the /run/foreman/puma.stats shows currect status and load of workers. It requires 2.4, so you’d need to install it into SCL (not recommended, only if you know what you’re doing). It may be watched like this
watch --interval 0.1 --color scl enable tfm 'puma-status /run/foreman/puma.stats'
but it fails to correctly find the private tmp directory and there’s no way to specify the control socket directly. The only workaround I found so far was to copy the
/run/foreman/puma.stats to another file and manually update the control_url there. Then again, it fails with
too long unix socket path (112bytes given but 108bytes max)
The output you’d normally see looks like this
1643714 (/tmp/puma.stats) Uptime: 0m16s | Load: 0[ ]16 | Req: 0 └ 1643714 CPU: 0.0% Mem: 344 MB Uptime: 0m16s | Load: 0[ ]16 | Req: 0
watching it can tell you if you’re using all 16 threads and whether increasing it’s maximum could help improving the performance.