Debian/Ubuntu full mirror with Pulp verbatim publish

Community
1

Hello everyone

I tried to create a mirror of the Ubuntu repositories, but when publishing a content view I saw that the signed metadata is not forwarded and there seems to be no option in Katello to do so, at least I found none.

I was eventually able to publish the repositories manually with the pulp CLI with this commands:
https://pulpproject.org/pulp_deb/docs/user/guides/publish/#verbatim-publications

Now my question is, if I setup some cron script on the Foreman server that does the publishing in regular intervals, will this in any way interfere with Katello? Is there a better way to do this?

I know about the signing with a custom key, however I explicitly want the upstream metadata for now.

2

You can set a repository’s Mirroring Policy to ‘Complete’ which would preserve the upstream metadata. However I’m not sure if this works with deb repos.

edit: looks like Complete is only available for yum repos.

3

Yes, I think this would cause Katello and Pulp to be out of sync and Katello may not know what’s in your content view. I wouldn’t publish anything via pulp directly.

4

In “lib/katello/repository_types/deb.rb” it looks like the publication class is always “PulpDebClient::DebAptPublication”, while for meta data forwarding it would probably need to use “PulpDebClient::DebVerbatimPublication”. Do you know what the right approach would be to implement this? Probably not an entirely new repository type.

5

I can confirm that “full mirror” for deb type does not currently exist as a Katello feature.

Regarding simply doing verbatim publications on the Pulp side: @jeremylenz has already given the general disclaimer that Katello is designed to fully manage Pulp as its backend and doing things in Pulp without Katello’s knowledge can easily be used to mess up your installation and is in no way a supported scenario.

Regarding implementing the feature for Katello: It is on our wishlist, but not currently planned so we are unlikely to do it any time soon. It is likely not as trivial as one might think. Some possible stumbling blocks and challenges I can think of:

  • How should verbatim publications propagate into CVs? What about CVs with filters? (How does this work for yum content?)
  • There are various known issues with verbatim publications on the pulp_deb side that need to be fixed. (The feature has never been hardened by the trials of usage via Katello after all.)
  • We are still in the midst of the structured APT transition for deb content on Katello, which is likely a prerequisite for “full mirror” for deb content using subscription manager.
  • Even with structured APT enabled, there might still be corner cases (like upstream repos using flat APT repo format), that cannot be made to work with “full mirror” mode and would likely require special handling.

One possibility that might side step some of the issues, might be to initially add “full mirror” for deb content only for the proposed “rolling content views”. Of course that implies implementing rolling content views first, see: https://issues.redhat.com/browse/SAT-28495 (I think we have yet to create a Foreman issue for this, but unlike “full mirror” for deb this is actively being worked on).

=> Overall I don’t think that “full mirror” for deb content can be quickly achieved to a production ready quality level.

3 Likes