Setting up Prometheus Metrics / Grafana for Foreman Performance Monitoring

[ikonia]

I think on their own they are quite useful, especially if you’re in a very modular environment where you separate the components onto individual nodes.

For me their main value is the performance of the ruby environment monitoring, but as lzap pointed out in this threads, it’s more stable to use statsd

Monitoring Foreman with Prometheus

Watch out, the Prometheus endpoint in Foreman suffers from a serious bug in Ruby prometheus library. As more and more worker processes gets recycled, more and more temporary files are created and this is leading to /metrics endpoint being slower and slower up to minutes or even hours. Use statsd setting and statsd_exporter from Prometheus to workaround the issue.

I"ve not used the co-pilot monitoring here so can’t comment on it’s value, I’m sure it’s good though
https://docs.theforeman.org/3.16/Monitoring_Project/index-foreman-el.html

I’ve found the best way to get a picture (because most of my environments are now invested in prometheus and time based stats) is to run the apache exporter, the postgress exporter, the puppet-operational-dashboard (and exporter) and use a few instance seperations to get a clear end to end view or foremans web components, it’s ruby internals, it’s database, and the cost and performance of puppet (using additional exporters like memcache etc) but pulling the individual components and summing them into a single grafana dashboard has worked well for me so far.