Problem:
I spend the day yesterday updating from Foreman 2.2 (+katello) through:
- 2.3 (No issues beyond the python2 qpid dependency mess on centos 7)
- 2.4 (No issues. Pulp 3 migration steps failed with pulp 3 not installed errors)
- 2.5 (No issues during upgrade, but foreman about shows pulp: 404 ; pulp3: "not all necessary pulp workers running)
While doing the upgrades, I skipped the pulp 2 → 3 upgrade migration on 2.4 as it wasn’t working (It was complaining about the lack of pulp 3. I didn’t have any pulp 3 rpm’s installed so I assumed the upgrade to 2.5 would install them so I could do the migration.)
Looking through the documentation I feel like I “missed” the window to migrate to pulp 3 even though it wasn’t installed? Not sure what to do next. tried the normal foreman-service restart without any luck
a little more info. running foreman-maintain content prepare results in…
=========================
Prepare content for Pulp 3:
'ErbParser' is ignored
'RubyParser' is ignored
rake aborted!
Don't know how to build task 'katello:pulp3_migration' (See the list of available tasks with `rake --tasks`)
Did you mean katello:import_subscriptions
I feel like leaving the katello:pulp3_migration tasks in the code for a few versions with a notice of things to try would have been a bit more friendly.
My pulp3 install seems to have the 2->3 migration plugin installed… maybe I should try a direct migration within pulp?
https://docs.pulpproject.org/pulp_2to3_migration/installation.html
now our goferd’s are going nuts and consuming CPU on every attached node. Why is Foreman so difficult to upgrade 
Hi @kallisti5,
Can you confirm your Katello version?
Thanks!
Katello 4.1.4
Here’s a (relevant) full package list of the server:
Thanks!
Unfortunately there’s no way to run the Pulp 2 to 3 migration from Katello versions past 3.18.5, as that’s the last version that includes both Pulp 2 and 3. 
If you’re able to restore from a 3.18 backup, you may be able to try again.
This feels like it should be something mentioned in the documentation since not doing so is pretty catastrophic to your environment and potentially hundreds of man hours of work if you don’t have a backup.
Looking at the The Foreman Documentation | Foreman 2.4 and Katello 4.0 documentation document, under “features”
“* It is very important that when upgrading to Katello 4.0 that content is migrated from Pulp 2 to Pulp 3.”
It doesn’t mention why it’s important… or that moving on to 4.0 without migrating is potentially catastrophic.
However, the upgrade guide doesn’t mention this at all . The one time it mentions Pulp upgrades, it uses “optional” language…
“If you have migrated content from Pulp 2 to Pulp 3, remove all Pulp 2 content.”
https://docs.theforeman.org/2.4/Upgrading_and_Updating/index-katello.html
I’m not throwing shade, I know it’s as-is open source… but that seems like a major omission.
I thought we had made this more obvious on the 4.0 upgrade, but i could not find it and agree it is a major omission. we’ll be adding it!
2 Likes
no backups because of the size of the server
Is there any way to salvage the install? (Start clean in pulp 3, rebuild repos, etc?)
I’ve opened a pr here for the documentation: https://github.com/theforeman/foreman-documentation/pull/919
You likely can delete all of the Content Views And Repositories and then re-create them in Katello. If you hit any issues with the deletion we can probably work through it.
My biggest issue at the moment is while pulp 3 is running, (/pulp/api/v3) we’re seeing “pulp3: not all necessary pulp workers are running” in Foreman / about
If it isn’t an easy “click through the ui and recreate some stuff” I think we’re going to seek out other solutions. We migrated to Foreman from Spacewalk… but the amount of work to keep Foreman upgraded and running has grown exponentially over the years beyond our desire to have consistent lifecycle updates + patching.
We’re going to have a meeting to discuss what comes next. Thanks for your help.
