[RFC] Making things easier when working with custom products & Simple Content Access (SCA)

Development RFCs
#9

So with the current solution without SCA and a setup, where multiple OS and OS-Versions need to be maintained, often Composite Content Views are used to have different sets of custom repos in separate Content Views to update them without having to update all repositories.
In this case I often use Content Views to combine multiple products offering the same repo for different operating systems. With the SCA approach that would not work any more, meaning to have the same functionality I would end up with around 10 times more content views like currently.

To have an example:
Maintaining different OS:

  • RedHat 7
  • RedHat 8
  • RedHat 9
  • Ubuntu 18
  • Ubuntu 20
  • Ubuntu 22
  • Rockylinux 8
  • Rockylinux 9

And to simplify the amount of Custom Repositories, I have separate Content Views for:

  • Base-Repositories (OS, subscription-manager) - one for each OS major-release
  • EPEL - one CV combining EPEL for 7 and 8 (which are separate products)
  • Puppet 7 - one CV combining all Puppet 7 - repos (which are separate products)
  • Puppet 4 (because there is an legacy environment, where the modules are not yet updated to work with Puppet 7) - one CV combining all Puppet 7 - repos (which are separate products)
  • Docker-Repositories - one CV combining all Puppet 7 - repos (which are separate products)

Now I combine one composite Content View for each operating system with each CV → 8 composite Content Views and 12 “normal” content views.
Since I do not add the subscription for all the unwanted products in my activation keys, this is easy to maintain.
If SCA now means I have to separate those Content Views, I would still have 8 composite content views but to achieve the same functionality I end up with around 40 normal content views - and in this example there are not very many custom repos included.
So in the end it will be difficult to maintain all content views to have the correct patching states - which will only work, when automating it with Ansible or something.

So I can image that the overhead is somehow “acceptable” if I only have to use RHEL in 2-3 major releases but in the end I have the impression that SCA will make everything much more complex for every repository not provided by RedHat.

3 Likes
#10

To comment also the suggestions:
I could disable/enable all unwanted repositories in the activationkeys but since products consist of multiple repositories, this will make it complex to keep activation keys up to date. If everything is enabled as default I need to adjust all activationkeys when a new OS joins my CVs.
If everything is disabled, this is probably better. But I will end up with repo-files containing very many repositories, which are disabled. Since then I only need to activate repos in new activationkeys, which will be very “crowded” if I do not have the filter mechanism to show only repos included by subscriptions. So this would be similar to the current workflow but still less user-friendly due to having many repositories to go through.

2 Likes
#11

Remember, that activation keys only affect the repository during registration of a new host. It does not change the repository set of existing clients. Thus, if you add a new repository to a product (e.g. you add postgresql 15 to your postgresql product), this will be enabled by default on all your existing clients if they have access through the content view.

The redhat.repo always contains all repositories which are part of the content view. Thus with SCA you will always have a lot of disabled repositories in your redhat.repo. The only way to reduce the number of repositories would be to remove them from the content view…

#12

Ok so that would mean, that I have to split everything up into much more Content Views - which is not nice.

So maybe this would be an idea, where I could restrict the repositories to work only on certain OS. That would add the connection of a certain repository to a given operating system. Then there might be custom repositories, which are used in more than one OS but in this case I might need adjust Content Views.

#13

Another special case, which just came to my mind:

What will be the future of the possibility to connect to the latest synced repositories?
So an activation key pointing to Library & Default Organization View → this is often used for development machines.
But this would mean, that these hosts would get ALL repositories set up in this organization (which could be a mix of RPM-repos of redhat based-distros and suse-repos). I think a mix of rpm and deb will not work anyway but that could mean that this kind of use case needs something like a content-view which is published daily or similar.

#14

It seems like the two proposals in the original post may be useful for some people, but probably not enough. So I also want to add a third. (This third proposal was actually discussed previously internally, but it was decided that the first two were less complicated and more elegant…)

  1. During a future Katello upgrade, Katello flips custom products from “enabled by default” to “disabled by default.” As part of this process, there is a migration:

Before: Custom repositories are enabled by default. Let’s say a host has 6 custom repositories. 4 are overridden to disabled, leaving 2 repositories enabled.

Migration: The 2 repositories (that were previously enabled by default) have content overrides automatically added (“override to enabled”). The 4 repositories that were previously disabled via content overrides are not touched.

After: Of the 6 custom repositories, 2 are overridden to enabled, and 4 are overridden to disabled. The same 2 repositories are enabled as before, while the same 4 repositories as before are disabled. Going forward, new custom repositories will be disabled by default.

This proposal meets the criteria of

  1. No new configuration options for users; and
  2. No change for a given host’s enabled repositories - no surprises.

Also, it’s not mutually exclusive with the other two proposals above. We could do all three, or just this one.

Would this be useful? More useful than the status quo?

#15

I have mentioned this „migration“ above in here also pointing out some additional options if you wanted to give users some more control.

1 Like
#16

Yes, that would be reasonable place to be.

However, I still think that there needs to be a way to filter what a repositories a hosts can see/use. Yes, it can be done by content views, but then you are forced to managed content views. If we have multiple OS like Fedora, CentOS Stream, AlmaLinux with many rpm repositories you will just get everything at once without a content views.

2 Likes
#17

To be clear, the status quo (which controls repo visibility with products) works decently well for the degenerate case of community distro content.

It’s not at all clear how users are benefitting from this change - a default is changing, and tools (subscriptions) we’ve had to make content determinations are now being “nerfed”. So for the case of managing multiple community distros, content views will now become mandatory, and the complexity of defining activation keys to use them also increases (since you have to explicitly enable every repo you want to use).

It can be done, of course, I just don’t see the advantage. Meanwhile, we’re being told that a setting which is clearly an attribute of the repository (it’s “enabled” status) can’t be set on the object itself, but has to be manipulated through other means (actions on host collections, AKs etc). I recognize the limitation of candlepin in setting something to “enabled”, but as a user I think that’s something I can handle.

1 Like
#18

This is because of the way the ‘Restrict to OS’ feature works. Candlepin compares the “required tags” of the entitlement certificate in /etc/pki/entitlement

rct cat-cert 1795965736968636904.pem

...
Content:
	Type: yum
	Name: Red Hat Satellite Client 6 for RHEL 8 x86_64 (RPMs)
	Label: satellite-client-6-for-rhel-8-x86_64-rpms
	Vendor: Red Hat
	URL: /Default_Organization/Library/content/dist/layered/rhel8/x86_64/sat-client/6/os
	GPG: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
	Enabled: False
	Expires: 1
	Required Tags: rhel-8-x86_64
	Arches: x86_64

with the “provided tags” of the product certificate in /etc/pki/product or /etc/pki/product-default:

rct cat-cert 479.pem

...
Product:
	ID: 479
	Name: Red Hat Enterprise Linux for x86_64
	Version: 8.7
	Arch: x86_64
	Tags: rhel-8,rhel-8-x86_64
	Brand Type: 
	Brand Name:

In this case, the entitlement certificate “requires” the tag rhel-8-x86_64. Since the product certificate “provides” all of the “required tags” (in this case, just the one), the Red Hat Satellite Client 6 for RHEL 8 x86_64 (RPMs) repository is available.

Note that I say “available” here as opposed to “enabled.” If the required tags were not provided, the repo would simply not be listed at all under subscription-manager repos.

Since CentOS and other non-RHEL Enterprise Linux distros do not include a product certificate, none of the required tags will ever be provided. This is why the feature only works with RHEL currently.

I’m not sure exactly what it would take to make this feature work with other OSes. I plan to ask the Candlepin team about it.

3 Likes
"Restrict to OS version" does not filter repo
#19

I don’t really have to add much to this as most things I’m also very much scared of are already mentioned.

One little addition, which is most likely a extreme edge case of my own is:
I’m using Katello in the release process of any new Rocky Linux release, which means I have added all and every repo in one of my Katello instances.
This means prod and staging for 2 major releases and all in all I end up there already with 137 repos.
Plus application repos from many different distributors and for testing 2 Ubuntu releases.
All in all I end up with 195 repos.
Currently managing everything with 2 lifecycle environments over 2 content views.

Beside that I have some machines assigned to no CV because they are the only machine leveraging the repo (or are unable to switch the CV because they are a Kubernetes node, and the networks there make Katello bug out on switching).
Also heavily using adding subscriptions for systems which need repos that most other machines don’t need (like Remi’s RPM repos for PHP)

Moving to SCA would mean to rethink everything, creating like at least 50 CVs with several CCVs on top to make it more manageable (or I will end up with a insane list of repos on every system repository sets view and having 200 repos in every systems redhat.repo file), touching literally every system because of the all repos being disabled by default (with having to generate a report, what is previously enabled where, and has to be enabled afterwards)

Tbh right now as the subscription-manager for Ubuntu is not ready for this, it’s not possible to move, but as soon as it is, it would be doable, will just take me days to reconfigure everything.

And everything because SCA is the new great thing… for RHEL
I really understand the move, but tbh there aren’t really many benefits for distros beside RHEL.

#20

Can you elaborate why it’s a concern to have lots of repos in redhat.repo?

Can you elaborate why it’s a concern to have lots of items listed in Repository sets?

You either start with all custom products enabled, and must disable thru overrides (the status quo), or you start with custom products disabled, and must enable thru overrides (proposal #3 above.) I’m not sure I understand what concern you outlined here? Are you in favor of one or the other?

1 Like
#21

This is not really a technical problem, just a look and feel thing, even a very minor one. (In case you start looking for repos on a client machine via either dnf or subscription-manager, which nearly never happens)

As well very much a keeping the overview thing, a search and a filter for currently enabled repos would improve that of course, but still that is slower than just having the repos there, you enabled the subscriptions of and being able to see what is enabled or what needs to be enabled to get to the wanted result.

Correct, currently the systems get some repos disabled on the bootstrap via the Activation Key, after changing all repos to being visible to a machine all will be either enabled or disabled by default, the currently already overridden repos will stay the way they are, but the rest… Lots of manual work involved.

Or am I reading this wrong in your #3 proposal? Does it mean that all repos that previously were available but not changed to overridden disabled (so either default or enabled overridden), will be enabled overridden and everything else disabled? (looks like I didn’t think of a automatic migration then :roll_eyes:)

It wouldn’t remove the need of creating lots of CVs to keep the overview though, but at least it doesn’t have to be done while migrating to SCA, which sounds a lot better than I previously thought!

1 Like
#22

Yes. So on a given host you’d still have the same repos available before and after the Katello upgrade / migration.

1 Like
#23

Thank you for the explanation then! And sorry for throwing the dust…

So, this means it’s basically a manual migration of subscription+CV based housekeeping to a CV-only based housekeeping, but no need for a manual host migration, which a lot less time consuming :slight_smile:

1 Like
#24

Both searching and filtering repository sets are available in the new host details page :slight_smile:

image
image3336×1108 283 KB

1 Like
#25

I really should use the new UI a lot more :upside_down_face: (it’s just the bigger font that keeps me going back to the old more compact view)

I know I’m complaining on a very high level, but it’s of course more clicks :sweat_smile:
And maybe I’m very inefficient but sometimes I’m also just going through the list to enable repos like CRB without searching because, well more mouse movement and more typing.

1 Like
#26

I have created a (draft) PR to be able to use “Restrict to OS” for non-RHEL systems: Add os product tag based on /etc/os-release by sbernhard · Pull Request #3188 · candlepin/subscription-manager · GitHub Pino from candlepin team reached out to me and mentioned, that its possible to create a certificate for every candlepin product which would then be used from subscription-manager. Unfortunately, the certificate does not contain product tags which are later on used to match repository OS-tag. Maybe it would be possible to extend katello so that individual product tags can be written from katello to candlepin.

Is this really already planed and someone is working on this? I think this would be a huge mess. First, the plan was to make everything easier with SCA. Then, multiple Content Views can be assigned to a Content Host. This would also affect activation keys, APIs, etc. This would make a lot more complicated as before - and was not the idea of SCA I guess.

#27

Thanks for the effort! Would be great to have the option to restrict it manually as not all repositories will have helpful information. As some vendors build separate packages and others only one for all different EL versions, it could also be needed to have this as some kind of array.

The certificate mentioned is the productid metadata used by the dnf/yum to have the information from which repositories/products something is actually installed and afterwards reported to Katello? Would be great to have this specified somewhere openly so other vendors can use it. Even if I think it is no solution for this problem.

#28

Another proposal:

  • all repos disabled by default
  • Add quick-actions to the activation key / repository page for
    • Overwrite all Repos to enable / Disable
    • "Activate repos from selected product: "
2 Likes