Katello - Saying goodbye to Pulp 2

Hello All!

With the release of Katello 3.15 we have introduced Pulp 3 as a backend service to manage content. Over the next couple of releases we will phase in Pulp 3 and phase out Pulp 2:


This is a companion discussion topic for the original entry at https://theforeman.org/2020/05/bye-bye-pulp2.html
4 Likes

How the heck I do headline in this markup.

Goodbye Mongo! And thanks.

1 Like

Fedora CoreOS and RHEL CoreOS are both OSTree-based systems. Without Pulp+Katello supporting them, what are we supposed to do for mirroring that content? There are no options without this.

As for Puppet stuff, it seems kind of bad to remove support for it in Katello+Pulp when so much currently supported projects and products rely on the functionality…?

this is one of my biggest concerns - Red Hat seems to be moving all puppet work out of projects in favour of Ansible, both have a place but Red Hat projects (pulp/foreman) seem to be not investing in puppet, while foreman makes puppet an equal option in the future, pulp seems to have dropped support with no plans to support it.

Hi @Conan_Kudo and @ikonia

Thanks for taking the time to write. I want to provide a bit of background.

Pulp 3 addresses major problems that were not possible to implement within the Pulp 2 architecture. This overhaul meant that every content plugin had to be written from scratch. For more information about the changes between Pulp 2 and Pulp 3, see About Pulp 3 on the Pulp project site.

Creating these new plugins was prioritized based on what the community used most and was asking for.

From the preliminary data gathered in the 2020 Foreman community survey, as well as the Puppet mini survey, we learned that the Pulp & Katello workflows for Puppet content were rarely used. For Puppet, the Katello functionality did not match the capabilities that were found in tools such as the R10k project. For example, the Puppet Forge v3 API was never implemented in Pulp, and therefore Puppet content management with Katello was limited to the built-in flows. Replacing the deployment script with existing tools like r10k was impossible. R10k has example hooks to flush the Puppetserver cache but Pulp didn’t. Feature-wise, it was very limited.

While currently implementing a Puppet plugin is not planned by the Pulp core team, If there is enough interest in the community in managing Puppet using Pulp & Katello, it is possible to write one. If you’re interested in writing a Puppet plugin, see the Pulp plugin writer’s guide. This then can be incorporated into future Katello releases.

Depending on how you are managing Puppet content, you might find some necessary functionality in the Pulp File plugin.

With regards to OSTree content, there is no plugin currently available for Pulp 3, but there are discussions happening about creating this plugin. Again, if you’re interested in contributing, see the Pulp plugin writer’s guide or get in touch.

2 Likes

this is a really good response, and appreciated, and I think you’ve highlighted some of the key circular situations (which doesn’t rest at your feet).

Is the reason that people where not using pulp 2 with katello on foreman due to the fact that tools essential for enterprise adoption such as R10k (or something with that version locking capability) wasn’t possible with pulp 2 in katello, I know personally for some of the larger deployments I worked on not being able to integrate puppet module and deployment in environments with r10k was a problem as I couldn’t “ensure” that versions and module where deployed to environments with a standard SDLC / gitflow style approach, which means when you poll “who’s using puppet with pulp and katello” the numbers may have been low compared to the question “who would use it if it was more feature rich”.

Now we have a similar situation where “would you like to use puppet with pulp 3 - the ability to implement a full forge capability to be able to use workflows like gitflow with r10k - or other similar approaches” - I suspect the answer will be yes, however if you say "do you plan to use puppet with pulp 3’ due to the lack of availability of functionality, or perception of what pulp 2 could do being carried over to 3 - I suspect you’ll get a “no” response

As you say people can build it, but I think it needs leading from the core project, such as other projects if it’s to gain traction, up take.

Love the work on pulp 3 - much simpler than pulp 2, so please don’t take the ciritique as against pulp 3, perhaps just a little disappointment that a core usecase for a tool that is currently using puppet a lot and providing a huge puppet use case (and puppet itself is a huge adoption by the world) was dropped rather than taken forward to the next better / full feature version.

2 Likes

Does Katello 3.18 with Pulp3 upgrade deletes the existing pulp2 and mongodb installation or we have to delete it manually

after you upgrade to 4.0 you can remove it with the ‘foreman-maintain content remove-pulp2’ command.

Keep in mind the discussion here: 'foreman-maintain content remove-pulp2' is evil

we’re in the process of improving that utility. If you use rpm to remove things by hand you can avoid that issue though (choose no when prompted to have the tool remove it for you)