Ubuntu Suites and Components

Support
,
1

Katello 3.18.1 (deb is using pulp2)

Ubuntu 20.04 using subscription-manager from Atix

When I run apt update -I see this

katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-universe default/all all Packages Err:126 katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-universe default/all amd64 Packages 404 Not Found Ign:127 katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-universe default/all Translation-en_US Err:131 katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-multiverse default/all amd64 Packages 404 Not Found Ign:132 katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-multiverse default/all all Packages Reading package lists... Done E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-universe/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-main/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-security-restricted/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-updates-multiverse/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-updates-restricted/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-updates-universe/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-security-multiverse/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-updates-main/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-restricted/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/subscription-manager-2004/subscription-manager-focal/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-security-main/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-multiverse/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-security-universe/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-restricted/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-main/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-universe/dists/default/all/binary-amd64/Packages 404 Not Found E: Failed to fetch katello://foreman.lab.site/pulp/deb/SBS/Development_Publishing/PubImageFocal/custom/Ubuntu_2004/focal-backports-multiverse/dists/default/all/binary-amd64/Packages 404 Not Found E: Some index files failed to download. They have been ignored, or old ones used instead. user@foremanclient:~$

If I browse the repos via HTTP - I don’t see paths like focal-main/dists/default/all/binary-amd64/ - instead I see focal-main/dists/focal/main/binary-amd64/

That appears to be the source of the 404 errors.

I’m not sure how the Suites and Components are supposed to work here.

2

This means your underlying pulp instance is not configured to publish under the default/all distribution/component combination.

Background: Right now there is no way of passing distributions (aka releases, aka suite, aka codename) and components all the way to the managed systems, so the ATIX published subscription manager has them hard coded as distribution=default and component=all. Which works fine, so long as Pulp is also configured to publish using the same. (The upload package feature also relies on the default/all combination, since manually uploaded packages will not be associated with any upstream distribution/component combination).

What you need to do to fix it:

  1. Make sure the json config file /etc/pulp/server/plugins.conf.d/deb_distributor.json exists on your server, and contains at least the value "publish_default_release": true. The minimal config file you need should look like this:
    $ cat /etc/pulp/server/plugins.conf.d/deb_distributor.json 
{
  "publish_default_release": true
}
  2. I am not sure if this is required, but to be safe restart your foreman (or at least your pulp services) to ensure the changes take effect:
    foreman-maintain service restart
  3. You now need to republish the metadata of all repositories and content views that were published before the change. There are “Repubish repository metadata” actions throughout the Katello GUI, but the easiest way to ensure this is probably just to publish new content view versions.
  4. If you now browse to those repos via HTTP you should be able to see “paths like” focal-main/dists/default/all/binary-amd64/ (your example) in addition to all the paths you previously had.
  5. If this was all successfull you should now be able to use your content via the subscription manager.

Important: May I ask what guide/documentation you were following to get to this point (or where you would expect to find such documentation in case you did it on your own)? If you, the user, are walking into this, it means we did not document the required Pulp configuration properly and I would like to see that fixed. :wink:

2 Likes
3

Thank you very, very much. This is working (with one limitation - see below)

I missed this instruction when I was looking at the Community post from Feb 3.

I used that post and a whole pile of other resources when I was troubleshooting this yesterday. Here’s a quick list

https://apt.atix.de/

https://theforeman.org/plugins/katello/3.18/
https://theforeman.org/manuals/2.3/index.html
https://community.theforeman.org/
Foreman Community Demo #85 - YouTube (Maximilian’s talk/demo during the Community Demo in October)

Is there any other place where this tweak is documented besides the Community post from Feb 3rd? There is only a single valid looking Google result for “publish_default_release” and it is that post. I guess this post will be the second hit.

One other important question. This tweak doesn’t seem to work when I use a composite content view. The distributor publish doesn’t seem to respect the setting. Is that a known limitation? It does work if I rearchitect it to use a standard Content View.

4

That is strange, we use composite content views all the time, and I have never observed this.
So long as the underlying Pulp repository is republished, I don’t see how CCVs could evade the changed configuration, which suggests that your CCVs were not republished since your configuration has changed.

If you have performed the “Regenerate Repository Metadata” Action (after your configuration change) for the CCV version you tested, and that did not solve the problem, then that is a bug.

Since we have never had this problem I am guessing it does not manifest if the publish_default_release parameter is set from the start. (For our internal installation method this is always the case, which is also why we did not have it on our radar to document it prominently.) We will update the documentation on apt.atix.de (and possibly elsewhere) to make clear that this configuration is needed. (@maximilian)

Ideally the foreman-installer should set this configuration automatically if deb content is enabled, I will look into opening a ticket for this.

5

FYI: There’s a new article on how to serve content and deploy hosts running Debian 10. You should be able to adapt this to your Ubuntu use case.

6

This guide only shows how to create new hosts, not join existing ones to the Foreman + Katello ecosystem.

I’m getting a mix of 404 not avaialble, and “file name too long” when trying to update my apt repo catalogue on a test Ubuntu 20.04 VM. It appears to have correctly registered with the Foreman server (that was an adventure), and I see it under Content Hosts.

However, I cannot remotely install stuff just yet.

I’ve made the change by creating the earlier mentioned json file for deb stuff, and I’m currently waiting for my content view to publish a new version, then I’ll need to promote it and retry.

While I GREATLY appreciate all the efforts around getting Debian and Ubuntu content management in Foreman + Katello to a working state, it really is still a rocky road.

Firstly, it would be nice if we could get a deb package generated for the ca and other aspects on the Foreman/Katello side, not just rpm (is that already a WIP?).

Secondly, having the Foreman documentation updated with debian/ubuntu centric content would be fabulous, as it is horribly lacking right now (ocharhino is doing a great job of filling this gap for now, but it’s also a bit spotty/unclear in some areas too).

Thirdly, hoping this whole process for deb/ub content can get streamlined to the same degree that it is for CentOS/RHEL/Fedora.

I am really excited to get Foreman managing all the content on my Debian and Ubuntu hosts! And I may get that going today in my lab, but it’s a landmine field filled with slight gaps in documentation, tools not quite behaving as one would expect, and the like. Literally live broadcasting me writing this post, and the whole process.

Yay! Thanks for the help! Almost there :wink:

7

Oh, one other thing. Not sure if this is the best place to put it, but sure would be nice to have “download policies” for deb type, so we can do “on-demand” or others. Looks like that’s not yet in the deb repo type for Foreman as of this writing. :frowning:

8

I am only seeing this now! I took a few days off! Fantastic @maximilian

1 Like