This repository uses features which are incompatible with 'mirror' sync

Support
1

I am running Katello 4.1.4 on CentOS 7 with the latest pulpcore updates installed today. I wanted pulpcore 3.14.7 because of Katello 4.1.2.1 - 404 error through content proxy due to incorrect location_href

Anyway, to fix possibly broken urls in the database I run a complete sync on all products, i.e. all repositories. Two repositories, however, showed errors: EPEL 7 and EPEL 8 (the normal, not the EPEL 8 Modular repo):

This repository uses features which are incompatible with ‘mirror’ sync. Please sync without mirroring enabled.

I think this message has been introduced earlier but I guess the error only appears if you do a complete sync and not the optimized one or it only appears now with 3.14.7…

O.K. I have disabled “Mirror on Sync” for those two repository although I don’t really like it. It’s my understanding the mirroring removes old, outdated content which doesn’t exist in the repository anymore. In particular for those huge EPEL repositories it’s something I really want.

It’s a bummer that it doesn’t work for two of the probably most used repositories in the RHEL/EL universe. And I also wonder why they do something so weird in EPEL that it breaks with the pulpcore sync.

I also ran a complete sync on my content proxy and that also showed the same error message (4 times in total). Of course, there I cannot even change the mirror setting (though I haven’t tried to republished everything first…)

But either way, if it’s a problem in the EPEL repositories they should be notified to fix it. If it’s a limitation of pulpcore I think it should be handled properly in pulpcore that it syncs EPEL without errors.

$ rpm -qa *pulp* | sort -d
pulp-client-1.0-1.noarch
pulpcore-selinux-1.2.6-1.el7.x86_64
python3-pulp-2to3-migration-0.12.0-1.el7.noarch
python3-pulp-ansible-0.9.0-1.el7.noarch
python3-pulp-certguard-1.4.0-1.el7.noarch
python3-pulp-container-2.8.1-0.1.el7.noarch
python3-pulpcore-3.14.7-1.el7.noarch
python3-pulp-deb-2.14.1-1.el7.noarch
python3-pulp-file-1.8.2-1.el7.noarch
python3-pulp-rpm-3.14.5-1.el7.noarch
tfm-rubygem-pulp_ansible_client-0.8.0-1.el7.noarch
tfm-rubygem-pulp_certguard_client-1.4.0-1.el7.noarch
tfm-rubygem-pulp_container_client-2.7.0-1.el7.noarch
tfm-rubygem-pulpcore_client-3.14.1-1.el7.noarch
tfm-rubygem-pulp_deb_client-2.13.0-1.el7.noarch
tfm-rubygem-pulp_file_client-1.8.1-1.el7.noarch
tfm-rubygem-pulp_rpm_client-3.13.3-1.el7.noarch
tfm-rubygem-smart_proxy_pulp-3.0.0-1.fm2_5.el7.noarch
2

O.K. The same error now also appears during the standard “Sync Now”. So I guess this is introduced with 3.14.7…

3

O.K. I have disabled “Mirror on Sync” for those two repository although I don’t really like it. It’s my understanding the mirroring removes old, outdated content which doesn’t exist in the repository anymore. In particular for those huge EPEL repositories it’s something I really want.

It’s a bummer that it doesn’t work for two of the probably most used repositories in the RHEL/EL universe. And I also wonder why they do something so weird in EPEL that it breaks with the pulpcore sync.

I also ran a complete sync on my content proxy and that also showed the same error message (4 times in total). Of course, there I cannot even change the mirror setting (though I haven’t tried to republished everything first…)

@gvde This is something we’re currently working on. The current “mirror” sync tells Pulp to copy and publish all of the original metadata files. This repo includes metadata for deltaRPMs, but Pulp doesn’t support deltaRPMs, so clients like yum, DNF, zypper would try to install the deltaRPMs that were not there. Instead of having clients fail when they go looking for deltaRPMs that aren’t there, we added an error to tell users to use a proper sync mode for those repos.

But, as you mention, it really sucks to not have that option available for some repos, so we’re (re)adding a third mode that mirrors the content without mirroring the metadata.

2 Likes
4

Isn’t that what the katello cron job is for that runs foreman-rake katello:delete_orphaned_content RAILS_ENV=production?

5

I don’t think so. Mirror is how the upstream repository and the repository in the library are kept in sync. With mirror, I think, you are supposed to see an exact mirror of the upstream repository in the local katello repository.

However, with content views and published versions, content which may have been removed upstream, may still be used by a content view. Only later, when all content view version which have used the old content have been purged, delete_orphaned_content can eventually clean that up and remove it from the system and database.

1 Like
6

Let me kinda summarize the issue and what we’re planning on doing to help :slight_smile:

  1. Pulp added a metadata mirroring feature as part of pulp-rpm 3.14. When we upgraded, we didn’t realize at the time that this functionality piggy backed on the ‘mirror=true’ option that we call ‘mirror on sync’. So this one option now controls 2 ‘features’, ‘content mirroring’ and ‘metadata mirroring’.
  2. over time it became obvious that piggy backing on this existing option was a mistake, as metadata mirroring was incompatible with a lot of upstream repositories due to various edge cases. The error you see was added to make sure that users didnt’ discover these problems after syncing, and instead you get a nice error message.
  3. in pulp-rpm 3.16, it is now possible to specify a ‘sync_policy’: REST API for Pulp 3 RPM Plugin which allows us to control these somwhat independently.
  4. as part of katello upgrading to pulp-rpm 3.16, we need to support this, and this would give you the ability to ‘mirror content’ without ‘mirroring metadata’, the metadata part which is what imposes many restrictions.
2 Likes
7

What Katello version is this (pulp-rpm 3.16) planned for?

8

we wouldn’t be able to introduce this fix until katello 4.3 at the earliest, due to pulp-rpm 3.16 requiring pulpcore 3.15