Performance issues Foreman Katello managing 99 hosts

Hi All

I am trying to improve the performance of our Katello deployment at work, We are finding that it is taking an excessively long time loading the hosts page, up to 20 seconds to load. the specs of the host we are running is, Vcpu cores, and 24gb of ram. I have run the installer with the options –foreman-foreman-service-puma-workers=6 –foreman-foreman-service-puma-threads-min=6 –foreman-foreman-service-puma-threads-max=16

this made a slight difference however it is still extremely slow. looking on the server itself running htop, there are 2 processes " puma: cluster Worker 4: 3367622" that sit for a while but cpu resources and memory resources don’t max out while this is happening.

Im not sure where we can go from here, and im not sure where to look next to troubleshoot.

Which information are you showing in your columns? How many host-entries are you showing?
For VMware-hosts (associated with the compute resource) I saw often that the API-calls of the power-status are slowing down the loading of the all hosts page.
Also check how many entries you are showing when opening the page - the loading gets slower if you load more hosts - in most cases it starts to get slow for more than 50 hosts.

First of all, which version are you on?

There have been some performance improvements made by reducing the number of database queries. You won’t see that in your CPU load or anything, but there’s a lot of overhead in performing database queries. Often referred to as the N+1 query problem. That’s something that really needs to be solved in the application itself. If there’s more latency between the application server and the database server, the problem will be worse.

@Bernhard_Suttner and @jeremylenz I think you two were involved with some UI / API performance improvements recently right? Maybe relating to host content views if I remember correctly? Wonder if that could be related.

What Katello version?

If you turn on the new All Hosts page via Settings > Show new host overview page, are the loading times any better?

showing from 1-50 hosts and the information displayed is
Power > Name > Operating System > Model > Host Group > Last Report > Actions.

Katello is getting on a little we are looking at refactoring it soon, however the latest version doesn’t have some of the plugins available we use if i remember correctly so i have to find a solution to that prior to performing an upgrade project. but the katello version we are currently on is 4.3.0.

For our deployment the database was kept on the existing katello server in an attempt to reduce latency issues

Yeah, that Katello version is about 2+ years old.

I can’t remember if Foreman 3.1 (which goes with Katello 4.3) has this feature, but if there is a ‘Manage columns’ button on the page, you could try removing the Power column. That column in particular takes a bit to load. (If you don’t see that, you can also check hammer user table-preference).

Other than that I can’t really say anything except, please upgrade to a supported version as soon as you can :slight_smile:

Foreman 3.1 did not have this feature, but there is a setting to disable power status on the hosts page:
Settings → General → Show host power status
We chose to disable this setting when we were encountering load time problems with the all hosts page and this helped a lot.

1 Like