Given we didn’t manage to get Debian 11 for 3.1, I am proposing to update the table to read:
- 3.2 adds Debian 11
- 3.4 drops Debian 10
1 Like
It looks like Foreman 3.2 can ship Debian 11.
Branching is very close. That makes me ask: do we still want to continue with the plan and drop support for running Foreman on EL7 by version 3.3?
In the release notes, we tell folks that they should start thinking about planning a migration but I think the current state is that they should be thinking about a reinstall and repointing clients, no?
Assuming no easy migration/upgrade path then: if we drop support for 7.x in 3.3, I would guess a lot of people are going to sit on the upgrade until Foreman is running well on 9.x and then make the effort to reinstall. That could potentially only be 12-15 months away.
It might mean lower uptake for 3.4 and 3.5 and therefore less testing for those versions.
That is a good point. Clear upgrade instructions should be provided. @evgeni has been working on an upgrade using leapp:
AFAIK he’s at least tested with CentOS 7 → CentOS 8 Stream once, but I’ll let him correct me if I’m wrong.
1 Like
That’s correct, the idea is that the upgrade should be as easy as
# yum install leapp
# leapp preupgrade
# leapp upgrade
# reboot
As I have pointed out before and also in EL7 deprecation it all may look much easier if you only have a foreman installation without katello. (although even there I have a couple of puppet parameters which I pass per host group or host which need to be recreated…)
But with katello, I see a lot of additional configuration until everything is set up. The initial setup includes lots of products and repositories to be set up. For each subscribed host I basically have to assume a unique set of repositories enabled. And as far as I can see I cannot simply point a client to a new katello server and it automatically subscribes to all the products it had and enables/disables all the repositories it has.
So to me, a working migration path is essential. Because otherwise, I really do have to start all other again, going through all the settings for all the hosts and recreate everything.
To me, there seem to be two possible ways:
-
backup and restore. Backup all configuration and restore them into a new foreman/katello installation. The synced pulp content doesn’t have to be migrated. I can live with that being downloaded again (even though it’s close to 1 TB by now). Of course, some intelligent migration for the pulp content would be nice (i.e. the new server doing the initial download from the old server or mounting the old pulp partition on the new server to copy the content).
-
Extraction of the complete configuration into an ansible playbook to install the new server. That would be the ideal solution to me, because that way I could get into the ansible installation of foreman/katello without starting at scratch.
I think, EL 7 should only be deprecated after the backup/restore process is fully and well tested, so that I can use that to do the migration. No. 2 is far away, if it ever gets there.
Pulp2-3 migration was a major operation for me which took me months until I had s state (with a couple of workaround) to get everything migrated.
Now with katello 4.3 officially out I have to do the puppet module extraction and hope everything will still work after the upgrade. I really do hope that it’s not as difficult as the pulp migration, but because of the problems with the pulp migration and waited with the upgrade from 4.1 as long as possible. I hope it goes smooth but I am worried it may take me again a while to get it properly working again.
If migration is again another pain like the pulp migration, I really would defer any further upgrades until I set up a new server on EL9, because I don’t really don’t want to migrate now from EL7 to EL8 and in two years again from EL8 to EL9. When EL9 is out I would set up a new server on EL9 and start learning how to set up foreman/katello with ansible (which I have never used before).
And I assume many others will do so, too. After all the work and pain the past upgrades have caused you’d rather postpone the updates.
So please, please, give a well working migration plan on how to backup/restore everything before deprecating EL7. And generally, I always think it would be much better, if life time of a product follows the life time of the supported operating system. I wonder what RedHat does with satellite? They have to support satellite until EL7 EOL, don’t they?
1 Like
- backup and restore is tested and supported. if it doesn’t work for you it’s a bug we have to fix.
- the migration using leapp (which you didn’t list) upgrades your system in-place. no data movement whatsoever.
- having everything in ansible is great, but extracting that data after the fact is way to complicated.
1 Like
I’m also in the distressed to see EL7 support being removed camp.
CentOS 8 not being a thing has left loads of people stuck on CentOS 7 until either the Rocky/AlmaLinux situation settles or their companies stump up the money for RHEL licensing.
Red Hat Satellite 7.0 will be the last version to support RHEL7. EL7 is currently in maintenance support 2, which means no new features.
From a developer perspective it’s more and more common to run into limitations. I run into systemd limitations most often and those won’t get fixed, but also more fundamental issues where rpm is too old or various packaging macros. The Apache is too old to support the event MPM, which is supposed to be more efficient. Packaging go on EL7 is also painful and software is starting to show up which uses this.
So the cost of supporting EL7 is going up and at least from the Red Hat perspective there’s no need for it anymore. That means a serious community of people needs to stand up to make sure it continues to work. Time that IMHO is better spent on making sure future versions work well.
I don’t know if this has been mentioned, but apparently more recent versions of ansible do not work on RHEL7. No plan to fix that either, AFAIK.
Various dependencies just don’t build.
So, I think RHEL7 is pretty much as dead as it can be, even from the RedHat perspective.
And the only reason people are sour about this is the CentOS 8 “situation”. Else everybody would have moved on.
1 Like
O.K. That’s probably what I have to do then. I really do hope it works. It just took me 2 months, various upgrade attempts, countless debugging hours, to upgrade from 4.2 to 4.3, just discovering at least two more issues/bugs.
So considering that, I am worried that the backup/restore project takes me a year until it actually works. Even the docs are extremely limited on how to restore on a different system:
Restoring Foreman server or Smart Proxy server from a Backup
If the original system is unavailable, provision a system with the same configuration settings and host name.
So what are those “same configuration settings”? And using the same hostname is kind of difficult as I cannot really shutdown everything for a longer time…
Just thinking about how I can set up the new system in parallel to the old one in production causes me headache, e.g. hostnames and names in certificates etc. I also have 1 TB for /var/lib/pulp. As far as I understand the backup process, it will backup the content of that and then restore it. So I have 1 TB on the old server, 1 TB in the backup, and 1 TB on the destination. And everything is going to be copied twice. For content which is 99,9% available online anyway. --skip-pulp-content is only for debugging purposes, so it seems copying the pulp content manually before or even simply using cloning the vdi and attaching it to the new server isn’t an option.
So I am really worried that this is going to be a major operation and in the end I will only wonder if it hadn’t be better simply to start all over again. I have 31 products, ~120 repositories, 4 content views, 275 hosts with parameters set, individual subscriptions and enabled repositories sets. Manually setting this up on a new server takes a lot of time, but if backup/restore would take longer in the end, causing much more problems it might be worth it… Of course, it’s futile to assume how long something will really take…
I don’t really like that. Over the time foreman collected so much garbage on the system that I really don’t want to convert the system in-place to CentOS 8. If I run yum autoremote it wants to remove more than 200 packages. I don’t want to keep all that garbage and I don’t feel comfortable what is going to happen with this after the conversion…
Well, extracting all the data manually is more complicated… Using hammer to extract the information about my 120 repositories, converting it into a form that ansible get take. I mean it’s only a limited amount of information that there is.
And if I simply set up a new system I have to get this information out of the old system anyway, even if it would be a simple click/copy/paste…
So either way, this is already causing me headache and sleepless nights, and I really don’t know what’s the best way to go without wasting too much time…
Another note on migration options: using ansible isn’t really fully working at the moment either, at least if you want to setup a katello server: it seems even the latest version of the ansible collection is still wants to configure the mirror_on_sync option and also has problems with empty http_proxy_id and ssl_*_id options. During each run it wants to remove the latter even though they are set to null anyway.
So my original plan with running the same playbook against my old and new foreman server, verifying it doesn’t make any changes on the old server and then deploying everything on the new server, doesn’t run so smoothly. Each run I have to manually check what it says needs to be changed and check that it’s only the repositories which are incorrectly flagged…
So now I need to patch the theforeman.foreman collection to make it properly work to be used in a migration…
For developers, users of our nightly packages and our Puppet modules, on June 6th we will begin the process of removing EL7 support across the ecosystem for nightly and thus developer setups.
1 Like
Is there a documented in-place upgrade path for RHEL7? The leapp method here mentions it won’t work on RHEL7.
If not, is there a tested and working way to export the data (without pulp/packages) from Foreman/Katello installed on RHEL7 to a fresh install on RHEL8 or even RHEL9? I have the same concerns as post #18 above.
There is a recent community demo that shows how to use Leapp to perform this upgrade:
Official documentation is in process along with a blog post.
1 Like
I’ve updated Created module for RHEL 7 to RHEL 8 upgrade by bangelic · Pull Request #1248 · theforeman/foreman-documentation · GitHub to have RHEL steps too now.
And no, you can’t export data w/o packages.
1 Like
Thanks for this, I’ll watch it and plan to test the upgrade once there is more complete documentation around it.
Would this be worth supporting in the future? From the sysadmin side having brought a production system from Foreman 2.1 to 3.1 with the usual issues during almost every y-stream upgrade, it would be a boon to be able to quickly throw a streamlined backup onto a fresh install and let it sync the packages after.
Given most repos don’t serve historical packages, you wouldn’t be able to sync and get a broken setup.
(Or one that absolutely doesn’t match the one you “backed up”)
Yes, that would definitely be an issue for installs with the need to keep older versions. However, more options and robustness in general for backup surrounding these kind of cases, such as whether to export only packages that are marked such or no longer exist in the source repo, would really help. I can’t speak to the feasibility of these options, but if they existed now it would make upgrade testing and planning much quicker and easier for organizations with larger setups.
Why is it a broken setup then? Technically, it should simply clean it up and fine.
And if I don’t use backup but set up a complete new server with the foreman ansible module it won’t serve historical packages either.
And either way: why must the backup include the complete pulp directory? For me, it would be much more efficient, if the backup only included everything else needed and I take care of getting a complete copy of the pulp directory to the target server (which would be pretty easy as it’s on a separate virtual disk in our virtual environment…) That would safe me a lot of time and terabytes of data copied around by the backup/restore process…
1 Like