Yeah, isn’t that also what our “Existing DB” CI check is about too?
It is! That’s what the second link enabled for Katello 
That answers it, thanks, all good.
I was using the term “monolith” lightly, I was more referring to the fact that Katello totally owns Pulp. I think Foreman should have an ability to use Pulp itself, which could mean moving some light Pulp functionality into Foreman, for example, rather than splitting it off into a new plugin. To your question, I don’t really think the most important thing is splitting Katello into pieces, but rather making those pieces easier for Foreman to use. In fact, introducing more plugins does give me some pause due to the new maintenance factors that come with it.
Content and subscriptions management feels very Katello-core to me. It’s more Pulp functionality outside of the Katello “content experience” that I think could benefit from being used by other plugins (like foreman-ansible-director).
Entitlements and Candlepin go hand in hand, so sub-man registration in Foreman alone would probably need to see Foreman proxying RHSM endpoints to the RH CDN, unless people don’t want their hosts to have any repos. Foreman being in the middle could mean the inventory is kept up to date while letting the host receive content.
Awesome to see, so we’re already testing Katello migrations after Foreman? It sounds like we’re over one major hurdle for making Katello installable after Foreman.
You are more proposing that Foreman core has facilities to interact with and use Pulp rather than some new plugin that provides that? How would you see Katello layering on top when present?
To answer this I think we need to consider what is core to using Pulp separately from what is core to Katello.
- Core to using Pulp
- Ensuring Pulp is present and configuring it
- Talking to Pulp’s API reliably (think: smart use of Pulp’s OpenAPI spec)
- Helpers for managing Pulp entities (think: Katello Pulp service classes)
- Core to Katello
- Higher-order task orchestration (and its connection to Dynflow)
- Indexing information from Pulp for performance reasons
- Katello-isms, like lifecycle management
If, say, IoP wanted to use Pulp to store some special advisory metadata, it may not need to know how to index every file’s checksum into Foreman’s database. It may just need to know that the repo is synced and perhaps have the option to query what is inside infrequently.
Each plugin could be responsible for orchestration and data. Do you want to save lots of info about your Pulp content? Do you want to create complex orchestrations that rely on Dynflow? Or, do you just want to create a pull-through cache and forget about it? Levels of complexity that are up to the plugin.
Katello itself I imagine would be loaning out its service classes that know how to do helpful things that have accumulated over the years. For example, how to fetch general items from the Pulp API cleanly with pagination.
I don’t have enough information to recommend having this lightweight Pulp functionality in Foreman or in a separate plugin. Stuffing Pulp bits into Foreman ‘feels’ strange to me, but that’s just a hunch.
On a more technical level, each plugin that uses Pulp could have its own Domain to ensure that conflicts do not occur.
I strongly support the idea of having a unified interface for talking to Pulp.
AFAIK, foreman_ansible_director is the only plugin (outside of Katello) that talks to Pulp directly at the moment, but as you have already said, there are definitely usecases for other plugins aswell.
Whether this integration comes in the form of a new plugin or a module, I’d keep it rather simple and only worry about the “how” and not the “why” when it comes to management of Pulp objects in there.
What I’d want from such an integration is:
- A set of DynFlow actions which wrap the needed Pulp*Client functions, which make composition a breeze
- Some initializer similar to require_dynflow, which takes care of some prerequisites, such as creation of a domain.
For foreman_ansible_director, I had to essentially re-create a lot of actions and a light version of orphan cleanup. That’s the type of thing I think could be abstracted and shared between “Pulp users”. Perhaps it would also make the migration of Katello to Pulp 4 a bit easier too, since dealing with domains will apply to every user of Pulp.
I’d be very happy to contribute to this.
Based on your examples, and how you’ve been thinking about this, I get the sense there might end up being stored in Pulp different scopes of data:
- User content (what Katello stores today)
- Admin / infra content (e.g. storing the Foreman containers)
- Behind the scenes content (things that are stored but not seen or managed directly by the user)
Does that sound correct? Am I missing a type?
Based on trying to classify the content in those ways, how would you see managing the content inside Pulp to ensure data doesn’t clash and that there is no breach of secure boundaries?
Those 3 categories cover everything I can imagine.
The best way to separate this would be via Pulp Domains: Enable and create Domains - Pulp Project.
Domains are Pulp’s answer to multi-tenancy where each tenant lives in its own bubble and content is not shared at all. In fact, the APIs are prefixed with different routes. It’s as close as you can get to having multiple Pulp instances on the same machine without actually doing that.