Update foreman from 3.2 to 3.11

Support
1

Problem:
system lcsm has foreman 3.2 running on it
there are several foreman proxies running in various environments handling requests to it from clients

My understanding is that lscm upgrade 3.2-3.11 has to be done in steps 3.2->3.3, 3.3->3.4, 3.4->3.5 and so on one level at a time until 3.11 is reached.
Upgrading and Updating Foreman

  1. do the proxies have to up be updated similarly at each stage, or can they be done after lcsm has reached 3.11 ie will 3.2 proxy communicate/work with 3.11 ?

  2. alternatively does a proxy of 3.11 communicate with a server at 3.2 ? ie do the upgrades the otherw ay around?

  3. do the clients need their foreman clients updating similarly at each stage, or can they done after lcsm has been upgraded to 3.11 and after their proxy servers have all been upgraded to 3.11 ?
    (TBH Im assumoin and saw clients hinted at elsewhere - Ive been dumped with all of this when somebody left without documenting much at all)

  4. of can the clienmts be upgraded to 3.11 and still work with foreman proxies/servers at lower levels ?

  5. and im really sorry to ask such a dumb qustion. Where are the use of foreman-proxies defined/configured on a client? Because all the baseurls on all clients - including thus the ones that allegedly use a foreman-proxy - are for the main server itself.

Basically it is trivial for me to clone lcsm to eg lcsm2 (different IP), upgrade lcsm2 and eventually flip lcsm2 to be lcsm (turn off lcsm, rebame lcsm2 and change IP) but that isnt really feasible for 5 foreman proxies and multiple clients

Expected outcome:
a smooth and easy transition process.

Foreman and Proxy versions:
all 3.2 currently

Foreman and Proxy plugin versions:
dont understand what this means - sorry (see above caveat)

Distribution and version:
dont understand what this means - sorry (see above caveat)

Other relevant data:
None known/appreciated

didds

2

  1. Proxies have to be updated as well, supported is n-1 (e.g. Foreman 3.11 and Foreman Proxy 3.10 are tested to work well with each other)

  2. Always update the Foreman first, then the Proxies.

  3. Client packages are independent, but plan to upgrade them too as a too big drift will likely contain some bigger and possibly breaking changes. As you are using Katello tools like subscription-manager, katello-host-tools and similar are the client part to make everything work.

  4. Should also work, but the bigger the drift the more likely something breaks.

  5. There is also a Foreman Proxy on the central system. Use of the Proxy depends really on the features used. I have some environments where multiple Proxies manage different subnets with different DNS and DHCP providers, in this case you will not see anything about the Proxies on the client. In others a second Proxy with Pulp so the content feature exists and on clients you will see different URLs in the yum/dnf configuration.

3

so in a nutshell, for

1 x server
4 x proxies
207 x clients (inc. the above)

Id have to upgrade the versions minor version by minor version each step?
3.2->3.3
3.3->3.4
etc

merely confirming.

tx!

4

and…

I cant find anything anywhere about upgrading foreman clients. Looking at the clients that are registered with foreman here, they have no packages installed with foreman in their names. does this mean in all likelihood there are no foreman clients installed on those systems 9whatever such a beast may look like :slight_smile: )

5

You can have a look into the client repository at Index of /client/3.11/el8/x86_64 to see what is provided for the clients. I would expect katello-host-tools(-tracer) to be installed. But updating this is not really critical, just do it if something does not work after updating the central systems.

And yes, step by step so migrations are run in an expected state. I would always do it to avoid problems that later occur from a migration missing or run incorrectly. Spending some time for the update progress is better then spending more time debugging a broken system later!

6

cheers - Im still struggling :frowning:
is there a simple way to find if a server even has a “client” package?

I have rhel 8 systems I have recently built that are basically vnilla ISO installs, hardened slightly with katello installed

curl -m 120 -O http:///pub/katello-ca-consumer-latest.noarch.rpm
which leaves us with
[root@fl-cel-bitgit ~]# rpm -qa | grep katello
katello-ca-consumer--1.0-1.noarch

and there is no indication/documentation to indicate where that came from sadly.

after that the server is registered using

subscription-manager register --insecure --org=“Default_Organization” --activationkey=“AK_Rhel8_non-prod”

(same subnet as our foreman server, and where that Key is defined in the foreman server etc )

so has that server even got a “foreman client” ? Ive googled for “foreman client” and cant see anything obvious

7

You haven’t said if you have Foreman only or Foreman + Katello, but in general we do “n-2” for Foreman > proxy. This means that a 3.11 Foreman would work well with a 3.9 smart proxy, but anything below that is not supported and may break.

This means a possible upgrade path (if you don’t want to upgrade everything at once) is:

Foreman 3.2 / Proxy 3.2
Foreman 3.3 / Proxy 3.2
Foreman 3.4 / Proxy 3.2
Foreman 3.4 / Proxy 3.3
Foreman 3.4 / Proxy 3.4
Foreman 3.5 / Proxy 3.4

…and so on. The proxy version must always be within 2 of the Foreman version.

That said, when you need to jump this many versions, many find it easier to simply spin up a brand new Foreman (and smart proxies) at the current version, and migrate the hosts. That comes with its own set of tradeoffs.

Not quite sure what you mean by “foreman clients updating.” When upgrading Foreman itself, there is nothing in particular to update on hosts managed by Foreman.

Hosts managed by Foreman don’t need to be upgraded. Only Foreman and smart proxies do.

If you’re using Foreman + Katello, you’ll see the smart proxy it’s registered to in your subscription-manager configuration (/etc/rhsm/rhsm.conf) in the [server] section. If you see the Foreman server here, your host is not using a smart proxy.

If you want a host to use a smart proxy to consume content, it must be properly configured on both the host (sub-man config) and on the Foreman server. That configuration must happen either (a) when the host is registered; or (b) you can use the ‘Change Content Source’ feature to migrate a host to a new smart proxy. I would recommend completing the Foreman upgrade before attempting this.

1 Like
8

Thank you so much for youe extremely well considered reply Jeremy. I appreciate the help!

You haven’t said if you have Foreman only or Foreman + Katello,

welcome to my problem.

I have … “inherited” … this all from somebody who appears to have done a poor job extremely badly and hadn’t documented anything about how this was installed, configured, etc . They have left, several months ago. Im not finding the issues left behind - and its just me to fix it. Nobody else here knows anything either.

Bottom line - EPEL repositories wont sync because of something that changed in EPEL, we are on 3.2 and need to upgrade to 3.11 (well we could go lower but whats the point?).

So I have instructions (as yet untried of course) to upgrade the server and the proxies.
https://docs.theforeman.org/3.3/Upgrading_and_Updating/index-katello.html

But I don’t know what those instructions mean
“Upgrade to Index of /client/3.3 on all content hosts. For more information, see Upgrading Content Hosts.”
Upgrading Content Hosts = Upgrading and Updating Foreman

picking a couple of clients randomly they do not have any rpms installed named “katello-agent”.
What they do have is
katello-ca-consumer--1.0-1.noarch

I don’t know where that came from, why it is named that, whether that needs upgrading.

Si I really do apologise for totally ignorant in these circumstances but I’m stuffed basically hence reaching out.

Frankly, I’m phuqqed and learning on the hoof one little piece of knowledge at a time (and documenting it so some poor b4rstard doesn’t end up in the same boat as me at some time!)

As far as I understand it we have one foreman server (name = lcsm) and four proxies - plus apparently lcsm itself according to the smart proxies listing in the UI, and that has all been working fine (leaving aside the inability to sync EPEL now).

I get the point about spinning up new servers and proxies - the issue is the proxies are on systems with multiple other services and the headache of migrating those services to the new system is not an efficient path 9sadly - and reflects my point made several years ago to management that having multiple services on one server creates headaches for upgrades/migrations to one or more services.

TL;DR? - So from what you have said

step by step/minor version by minor version

update foreman server

  • does this auto magically also update the smart proxy on lcsm?
    update the four/five proxies (these are all in docker containers. Not documented. Guy that did that has left also…)
    ignore all clients

Correct?

9

If you’re syncing repositories including EPEL, you have Katello.

In the web UI under Administer / About (?), you can see the version of Foreman, Katello and any other plugins that you have.

Ah, I see now. Katello-agent was a method for executing remote package actions on hosts. It was replaced by Remote Execution a few versions ago. If you don’t have katello-agent installed on hosts, don’t worry about installing or upgrading it.

katello-host-tools could still be useful, if you also have katello-host-tools-tracer installed on hosts. Tracer is really the only part of katello-host-tools still in use today. If you want to upgrade to the latest, no need to do so on each upgrade step - just wait until you’re on 3.11 and then sync the latest Client repository, and then update your hosts.

katello-ca-consumer should remain on hosts, but does not need to be updated manually. It serves to configure subscription-manager to point to your Foreman server, and get your hosts’ certificates in order before they were registered.

Yes, this is correct. Your main Foreman server is also itself a smart proxy. :slight_smile:

I know you may not be able to change this, but this is explicitly called out as unsupported in the documentation: Smart Proxy server must be installed on a freshly provisioned system that serves no other function except to run Smart Proxy server.

Yes, updating Foreman server will also update its built-in smart proxy. :+1:

other than that your summary looks good!

10

Jeremy - you are a hero!!!

" Smart Proxy server must be installed on a freshly provisioned system that serves no other function except to run Smart Proxy server."

I have spoken to my boss. He laughed, cried etc.
Agreed with me that we must therefore build standalone smart proxies.
However. We have to Dcs shutting down middle October and we dont want to waste effort building SPs in those environments for two months use.

So the whole upgrade is now put off until at least mid October, likely November when we have clarity and a simplified palate to mix.

We will just have to live with a foreman 3.2 implementation that was already months if not a year out of date when it was implemented and an EPEL repo that won’t sync.

clarity.

My razor is named Occam.

THANKS XXXXX

Didds

2 Likes
11

Your boss sounds cool and understanding. Just saying. :wink:

1 Like