"Retain package versions" for repos broken?

On Foreman 3.2/Katello 4.4 and have a google chrome repo with the following settings:
Upstream URL: https://dl.google.com/linux/chrome/rpm/stable/x86_64
Download Policy = Immediate
Mirroring Policy = Additive
Retain package versions = 5
Restrict to architecture = x86_64

With this setting, I assumed I would only see max 5 of the latest versions of each package in the repo. The problem is that I see more then 5 versions of each package.
Did I misunderstood this setting or is it broken?

This is what I got with your settings:

I guess you just created that repo and thus only one version of each package is there.
With “Mirroring Policy = Additive” overtime the amount of versions of packages will grow. I created this repo a few months ago and there are already more then 5 versions of each package:


Just makes me wonder if something is broken…

Have another repo with the same settings taking data from https://packages.gitlab.com/runner/gitlab-runner/el/7/x86_64 for a few months and it has 9 versions of each package. Did I misunderstand the usage of the “Retain package versions” setting on a repo with Mirroring Policy = Additive?
I was hoping to use it to keep the size of the products under control but at the same time have more local versions then 1 that I would get with Mirroring Policy = Current Only.

It’s possible something is broken, I will investigate.

1 Like

I don’t think you’ve misunderstood anything.

This seems to be working fine for me with the currently-supported upstream release of pulp_rpm, I haven’t been able to reproduce. @partha did reproduce using an older build of Katello, but I couldn’t tell you what specifically might have changed between the two.

Hmm yeah it seems to be working correctly even on 4.3 for me. I tested with various repositories and configurations. Can you do a “Complete Sync” and confirm you still see this issue? Basically Go to Advanced Sync -> Complete Sync on the repo details page and see if that fixes it.

Before sync:

# hammer package list --repository-id 51
--------|-----------------------------------------|-------------------------------------
ID      | FILENAME                                | SOURCE RPM
--------|-----------------------------------------|-------------------------------------
143250  | gitlab-runner-14.7.1-1.x86_64.rpm       | gitlab-runner-14.7.1-1.src.rpm
143249  | gitlab-runner-14.8.1-1.x86_64.rpm       | gitlab-runner-14.8.1-1.src.rpm
143248  | gitlab-runner-14.8.2-1.x86_64.rpm       | gitlab-runner-14.8.2-1.src.rpm
143247  | gitlab-runner-14.9.0-1.x86_64.rpm       | gitlab-runner-14.9.0-1.src.rpm
143246  | gitlab-runner-14.9.1-1.x86_64.rpm       | gitlab-runner-14.9.1-1.src.rpm
624143  | gitlab-runner-14.9.2-1.x86_64.rpm       | gitlab-runner-14.9.2-1.src.rpm
310258  | gitlab-runner-14.10.0-1.x86_64.rpm      | gitlab-runner-14.10.0-1.src.rpm
624144  | gitlab-runner-14.10.1-1.x86_64.rpm      | gitlab-runner-14.10.1-1.src.rpm
1062059 | gitlab-runner-15.0.0-1.x86_64.rpm       | gitlab-runner-15.0.0-1.src.rpm
143255  | gitlab-runner-fips-14.7.1-1.x86_64.rpm  | gitlab-runner-fips-14.7.1-1.src.rpm
143254  | gitlab-runner-fips-14.8.1-1.x86_64.rpm  | gitlab-runner-fips-14.8.1-1.src.rpm
143253  | gitlab-runner-fips-14.8.2-1.x86_64.rpm  | gitlab-runner-fips-14.8.2-1.src.rpm
143252  | gitlab-runner-fips-14.9.0-1.x86_64.rpm  | gitlab-runner-fips-14.9.0-1.src.rpm
143251  | gitlab-runner-fips-14.9.1-1.x86_64.rpm  | gitlab-runner-fips-14.9.1-1.src.rpm
624145  | gitlab-runner-fips-14.9.2-1.x86_64.rpm  | gitlab-runner-fips-14.9.2-1.src.rpm
310259  | gitlab-runner-fips-14.10.0-1.x86_64.rpm | gitlab-runner-fips-14.10.0-1.src.rpm
624146  | gitlab-runner-fips-14.10.1-1.x86_64.rpm | gitlab-runner-fips-14.10.1-1.src.rpm
1062060 | gitlab-runner-fips-15.0.0-1.x86_64.rpm  | gitlab-runner-fips-15.0.0-1.src.rpm
--------|-----------------------------------------|-------------------------------------

During “Advanced Sync → Complete Sync”

Total steps: 240/240
--------------------------------
Associating Content: 0/0
Downloading Artifacts: 106/106
Downloading Metadata Files: 4/4
Parsed Packages: 130/130

Production log:

2022-05-27T19:57:50 [I|app|0c488cf5] Started POST "/katello/api/v2/repositories/51/sync?organization_id=1" for 10.107.17.18 at 2022-05-27 19:57:50 +0000
2022-05-27T19:57:50 [I|app|0c488cf5] Processing by Katello::Api::V2::RepositoriesController#sync as HTML
2022-05-27T19:57:50 [I|app|0c488cf5]   Parameters: {"id"=>"51", "skip_metadata_check"=>true, "organization_id"=>"1", "api_version"=>"v2", "repository"=>{"id"=>"51"}}
2022-05-27T19:57:50 [I|bac|0c488cf5] Task {label: , execution_plan_id: a11fe3ca-483f-4d98-8977-9602c83f7b34} state changed: pending
2022-05-27T19:57:50 [I|bac|0c488cf5] Task {label: Actions::Katello::Repository::Sync, id: 9e7fd6d4-cf43-41c2-9794-e6fe655d2c38, execution_plan_id: a11fe3ca-483f-4d98-8977-9602c83f7b34} state changed: planning
2022-05-27T19:57:50 [I|bac|0c488cf5] Task {label: Actions::Katello::Repository::Sync, id: 9e7fd6d4-cf43-41c2-9794-e6fe655d2c38, execution_plan_id: a11fe3ca-483f-4d98-8977-9602c83f7b34} state changed: planned
2022-05-27T19:57:50 [I|app|0c488cf5]   Rendering /usr/share/gems/gems/katello-4.4.0.2/app/views/katello/api/v2/repositories/sync.json.rabl within katello/api/v2/layouts/resource
2022-05-27T19:57:50 [I|app|0c488cf5]   Rendered /usr/share/gems/gems/katello-4.4.0.2/app/views/katello/api/v2/repositories/sync.json.rabl within katello/api/v2/layouts/resource (Duration: 24.8ms | Allocations: 24040)
2022-05-27T19:57:50 [I|app|0c488cf5] Completed 202 Accepted in 353ms (Views: 26.7ms | ActiveRecord: 19.4ms | Allocations: 129944)
2022-05-27T19:57:50 [I|bac|0c488cf5] Task {label: Actions::Katello::Repository::Sync, id: 9e7fd6d4-cf43-41c2-9794-e6fe655d2c38, execution_plan_id: a11fe3ca-483f-4d98-8977-9602c83f7b34} state changed: running
2022-05-27T20:01:02 [I|aud|0c488cf5] Katello::Repository (51) update event on last_contents_changed 2022-05-20 04:09:42 +0200, 2022-05-27 22:01:02 +0200
2022-05-27T20:01:04 [I|aud|0c488cf5] Katello::Repository (51) update event on publication_href /pulp/api/v3/publications/rpm/rpm/df88e9ae-ca32-4133-a4f6-d362c8243f05/, /pulp/api/v3/publications/rpm/rpm/aa962c33-057c-4362-a126-200c54a3c8db/
2022-05-27T20:01:05 [I|app|0c488cf5] CONTENT_INDEX type=Katello::Rpm success=true duration=87.12
2022-05-27T20:01:05 [I|app|0c488cf5] CONTENT_INDEX type=Katello::ModuleStream success=true duration=62.15
2022-05-27T20:01:05 [I|app|0c488cf5] CONTENT_INDEX type=Katello::Erratum success=true duration=60.48
2022-05-27T20:01:05 [I|app|0c488cf5] CONTENT_INDEX type=Katello::PackageGroup success=true duration=59.11
2022-05-27T20:01:06 [I|app|0c488cf5] CONTENT_INDEX type=Katello::Srpm success=true duration=65.06
2022-05-27T20:01:06 [I|aud|0c488cf5] Katello::Repository (51) update event on last_indexed 2022-05-20 04:10:40 +0200, 2022-05-27 22:01:06 +0200
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: , execution_plan_id: 553f6307-da6b-42e1-a3a7-a351d67462d8} state changed: pending
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::Repository::CapsuleSync, id: f0fe223d-c8fe-4422-97d8-66181f2f5664, execution_plan_id: 553f6307-da6b-42e1-a3a7-a351d67462d8} state changed: planning
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::Repository::CapsuleSync, id: f0fe223d-c8fe-4422-97d8-66181f2f5664, execution_plan_id: 553f6307-da6b-42e1-a3a7-a351d67462d8} state changed: planned
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::Repository::CapsuleSync, id: f0fe223d-c8fe-4422-97d8-66181f2f5664, execution_plan_id: 553f6307-da6b-42e1-a3a7-a351d67462d8} state changed: running
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: , execution_plan_id: e1b656a9-9ec0-4d6a-8bb4-a0ea23af0275} state changed: pending
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::CapsuleContent::Sync, id: 7cf326d2-5830-4526-823e-4ba603a45ddd, execution_plan_id: e1b656a9-9ec0-4d6a-8bb4-a0ea23af0275} state changed: planning
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::Repository::Sync, id: 9e7fd6d4-cf43-41c2-9794-e6fe655d2c38, execution_plan_id: a11fe3ca-483f-4d98-8977-9602c83f7b34} state changed: stopped  result: success
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::CapsuleContent::Sync, id: 7cf326d2-5830-4526-823e-4ba603a45ddd, execution_plan_id: e1b656a9-9ec0-4d6a-8bb4-a0ea23af0275} state changed: planned
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: , execution_plan_id: 8d8313f6-4ae5-4c04-b55a-fb3a580975ab} state changed: pending
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::CapsuleContent::Sync, id: 7cf326d2-5830-4526-823e-4ba603a45ddd, execution_plan_id: e1b656a9-9ec0-4d6a-8bb4-a0ea23af0275} state changed: running
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::CapsuleContent::Sync, id: 7e71ef02-488a-4b81-9ae0-4996b27fde57, execution_plan_id: 8d8313f6-4ae5-4c04-b55a-fb3a580975ab} state changed: planning
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::CapsuleContent::Sync, id: 7e71ef02-488a-4b81-9ae0-4996b27fde57, execution_plan_id: 8d8313f6-4ae5-4c04-b55a-fb3a580975ab} state changed: planned
2022-05-27T20:01:06 [I|bac|0c488cf5] Task {label: Actions::Katello::CapsuleContent::Sync, id: 7e71ef02-488a-4b81-9ae0-4996b27fde57, execution_plan_id: 8d8313f6-4ae5-4c04-b55a-fb3a580975ab} state changed: running
2022-05-27T20:01:11 [I|bac|0c488cf5] Task {label: Actions::Katello::CapsuleContent::Sync, id: 7e71ef02-488a-4b81-9ae0-4996b27fde57, execution_plan_id: 8d8313f6-4ae5-4c04-b55a-fb3a580975ab} state changed: stopped  result: success
2022-05-27T20:01:12 [I|bac|0c488cf5] Task {label: Actions::Katello::CapsuleContent::Sync, id: 7cf326d2-5830-4526-823e-4ba603a45ddd, execution_plan_id: e1b656a9-9ec0-4d6a-8bb4-a0ea23af0275} state changed: stopped  result: success
2022-05-27T20:01:12 [I|bac|0c488cf5] Task {label: Actions::Katello::Repository::CapsuleSync, id: f0fe223d-c8fe-4422-97d8-66181f2f5664, execution_plan_id: 553f6307-da6b-42e1-a3a7-a351d67462d8} state changed: stopped  result: success

Packages after:

# hammer package list --repository-id 51
--------|-----------------------------------------|-------------------------------------
ID      | FILENAME                                | SOURCE RPM
--------|-----------------------------------------|-------------------------------------
143250  | gitlab-runner-14.7.1-1.x86_64.rpm       | gitlab-runner-14.7.1-1.src.rpm
143249  | gitlab-runner-14.8.1-1.x86_64.rpm       | gitlab-runner-14.8.1-1.src.rpm
143248  | gitlab-runner-14.8.2-1.x86_64.rpm       | gitlab-runner-14.8.2-1.src.rpm
143247  | gitlab-runner-14.9.0-1.x86_64.rpm       | gitlab-runner-14.9.0-1.src.rpm
143246  | gitlab-runner-14.9.1-1.x86_64.rpm       | gitlab-runner-14.9.1-1.src.rpm
624143  | gitlab-runner-14.9.2-1.x86_64.rpm       | gitlab-runner-14.9.2-1.src.rpm
310258  | gitlab-runner-14.10.0-1.x86_64.rpm      | gitlab-runner-14.10.0-1.src.rpm
624144  | gitlab-runner-14.10.1-1.x86_64.rpm      | gitlab-runner-14.10.1-1.src.rpm
1062059 | gitlab-runner-15.0.0-1.x86_64.rpm       | gitlab-runner-15.0.0-1.src.rpm
143255  | gitlab-runner-fips-14.7.1-1.x86_64.rpm  | gitlab-runner-fips-14.7.1-1.src.rpm
143254  | gitlab-runner-fips-14.8.1-1.x86_64.rpm  | gitlab-runner-fips-14.8.1-1.src.rpm
143253  | gitlab-runner-fips-14.8.2-1.x86_64.rpm  | gitlab-runner-fips-14.8.2-1.src.rpm
143252  | gitlab-runner-fips-14.9.0-1.x86_64.rpm  | gitlab-runner-fips-14.9.0-1.src.rpm
143251  | gitlab-runner-fips-14.9.1-1.x86_64.rpm  | gitlab-runner-fips-14.9.1-1.src.rpm
624145  | gitlab-runner-fips-14.9.2-1.x86_64.rpm  | gitlab-runner-fips-14.9.2-1.src.rpm
310259  | gitlab-runner-fips-14.10.0-1.x86_64.rpm | gitlab-runner-fips-14.10.0-1.src.rpm
624146  | gitlab-runner-fips-14.10.1-1.x86_64.rpm | gitlab-runner-fips-14.10.1-1.src.rpm
1062060 | gitlab-runner-fips-15.0.0-1.x86_64.rpm  | gitlab-runner-fips-15.0.0-1.src.rpm
--------|-----------------------------------------|-------------------------------------

Sync details after:

No content added.
Total steps: 374/374
--------------------------------
Associating Content: 120/120
Downloading Artifacts: 120/120
Downloading Metadata Files: 4/4
Parsed Packages: 130/130

Well, that part isn’t surprising, it’s a known limitation that I’m working on Make retain_package_versions more useful at sync time · Issue #2479 · pulp/pulp_rpm · GitHub

The main concern would be if it isn’t working right at the repository level - all the packages are currently being downloaded and stuff (because of that limitation), but it should still only retain the latest ones in the repo at the end of the sync.

Is this potentially the issue? Bug #34469: Retain packages on Repository does not synchronize the specified number of packages - Katello - Foreman