I am currently on my second attempt to get from katello 3.18 with pulp2 to katello 4.0. My first attempt a few weeks ago left me with a system which wasn’t usable as repo syncs failed, packages were missing, cv publish extremely slow… So I have reverted to my 3.18 snapshot…
Now I am at my second attempt and I am still having issues with failing or hanging repo syncs. Find the errors and applying patches manually fixes a few issues but that’s not nice and I am always worried that the next foreman/katello update may overwrite the patch again because it’s not yet in the rpms. So basically, I have to keep track of all the patches and workarounds to verify it’s still working after the update. So I am again thinking about abandoning this attempt again and go back to 3.18 and hope it will get better next time.
Now I see the upcoming foreman 2.5/katello 4.1 release and it makes me wonder if there will be every a fully working upgrade of a production system from 3.18/pulp2 to 4.x/pulp3. If it’s still the same I can skip a minor version for upgrade, i.e. I could go from 3.18 to 4.1 hoping that upgrade is going more smoothly.
But if the main development is moving on, eventually, people are stuck on 3.18 because they may not get migration fully working or it isn’t working with 4.x…
I’m sure there should be a stable upgrade path from previous releases. If you manage to give it another try, please report issues you see. I encountered few issues myself, but those are already reported if not fixed.
Apologies if you have reported this in other threads, if you could give us a list of the issues you’ve encountered (with as much detail as you can) trying to upgrade we’ll prioritize taking a look at them to see about a path forward.
Well, by know I am on 4.1. Eventually, with a couple of manual patches for .treeinfo and some more, my own patch to handle the problem with multiple repositories with identical label, and removing all almalinux and centos 8 stream repositories from the server I was able to switchover with 3.18.3. Luckily, I don’t use either only for evaluation this removing the repositories didn’t hurt and I had no issue adding them back later in katello 4.0.
And as I already knew the most prominent bugs and issues of 4.0 (applicablity, extremely slow content view publication, broken sync with grafana/postgresql repositories, broken sync with password protected repositories, no automatic sync of pulp3 content proxy…) from my previous attempts I finally felt confident enough to assume that, at least for the meantime, I can use the system and clients are delivered the right content. I was also afraid that waiting any longer wouldn’t really help. I wasn’t sure how much attention 3.18 would still get with the release of 4.1. And as I wasn’t sure how many bug fixes in 4.1 would be backported to 4.0 I have basically upgraded from 3.18 to 4.0 right after the switchover and then immediately upgraded to 4.1. That of course, increased the downtime of the server…
And this time I did not try to upgrade my content proxy but instead installed a completely new content proxy server and switched all clients over to the new proxy, which fortunately wasn’t too hard. Although, I still have a very bad feeling about those initial sync errors I got for the new content proxy. But at least with 4.1 and pulpcore 3.14 I finally was able to get a complete sync to the content proxy without errors. And so far, the (manually started) optimized syncs didn’t have any errors and clients seem to pick up on updates. I seems like it’s working and I hope there won’t be any surprises breaking it…
And all those issues and fixes to get it switch over and all those new issues with 4 in combination always made me wonder how stable the upgraded system will be or how many issues and bugs I have migrated hidden into the upgraded system which will cause serious problems later.
And timing for the switch to pulp3, at least for me, couldn’t be any worse: when I started going productive with 3.14 or 3.15 I had issues like with applicability which step by step all got sorted out and fixed and now with 3.18 and pulp2 I had a smoothly running system and no complaints. Going to 4 and pulp3 feels like moving back in time a year or two…
Anyhow, all of my issues with during the process should be in all those topics I have started in the last couple of months. I think I started new topics each time. I hope all those issues and bugs get sorted out soon…
EDIT: and just after I finished this I got some sync error with my content proxy…
And now even rpm requires are somewhere broken, breaking the latest puppet-agent update on my foreman servers. Funny enough, clients connected to the content proxy (which showed some errors during the last optimized sync) work. Only the servers connected to the main server don’t. You’ll see what I mean why it doesn’t feel good at the moment to run this system?
