How do you handle upgrading your Foreman+Katello server itself? Is it possible from its own repos?

Support
1

I have an Alma 8 based Foreman 3.6/Katello 4.8 installation through which I push out a regular monthly patching cycle to my Linux Estate.

I would also like to keep my Foreman+Katello boxes updated in line with the same monthly releases.

I have a PreProd instance of my Foreman+Katello box, which is registered to my Prod instance, which I (intend to) use for testing of updates (along with a couple of test boxes registered to PreProd to facilitate this testing). The Prod box has loal copies of all the repos required to update Foreman.

I can update my PreProd server without issues, but when when I come to try to upgrade my Prod instance, which is registered against itself, I encounter errors when trying to run foreman-installer / start services post upgrade (I’m not 100% sure if this needs to be run after a standard update or just after an update which involve a change of major version/to nightly releases? It seems like it shouldn’t cause any harm either way however).

I’ve tried a number of different workarounds, such as trying to start/stop some services prior to upgrade, and use dnf update --downloadonly but I’ve not found anything viable yet.

So my question is how do people handle upgrades of the Foreman server itself? Has anybody got the above working? It would seem a little strange to be pushing out controlled point releases to every other server I have and then only being able to update straight from external repos for the Foreman box itself.

One possible alternative is that I take my monthly Content Views on both Foreman servers at the same time, so versioning should be in sync between the two boxes. Should I be updating PreProd from Prod, and then updating my Prod box from the repos on PreProd?

I think from a couple of other historic forum posts using a smart proxy might be another option, I have no other current requirement for a proxy and I’m not sure if this would involve more or less work than my other idea above, but that may also be a possibility.

Thanks!

2

We are updating our Foreman server from it’s own repositories (at least for OS updates).
That said, having this kind of setup leads to some very special issues.
In general, OS updates should not cause you any harm. You should run foreman-installer after every package update since those can cause problems with some configuration otherwise, but you seem to be doing that already.
Where problems might start is when foreman-installer tries to install new packages (for example new ruby dependencies after a Foreman upgrade). Since Foreman/Katello is by design not running during a foreman-installer run, the repos where these packages come from are also not available, so you will have to find some solution for that. If you can verify this in pre-prod, you can usually install those packages before running dnf update / foreman-installer.

If you encounter any other problems with foreman-installer that are not “can not install package” related, those are in my experience probably separate issues that are worth addressing separately.

3

For all foreman/katello related updates we use the repos from yum.theforeman.org. First, I always stop foreman before running foreman-installer we had some problems in the past that foreman-installer wouldn’t always correctly restart services. Second, foreman-installer wants to install/update packages at times and obviously that doesn’t work that well if katello is not running or not correctly running.

So basically, for the main server we use the official repos and for anything else including all smart proxies we use the main server.
The problem is that foreman-installer wants to install/update packages