Different satellite servers pointing to same repository have different amounts of rpms

Problem: We have three different satellite servers serving different regions. All have repositories pointing to upstream EPEL repository. (All are identical in their configuration). I have a content view for EPEL with two filters. One is just all packages without errata. The other is all errata up to a given date. The date is identical on all three servers. All filters are inclusion type. Yet, two of the satellite servers show for the EPEL repository, 14967 packages, 4971 errata in 34 package groups. The third shows 14412 packages, 4852 errata in 34 package groups. I have no idea why this one is different?

Expected outcome: All three satellite servers have the same number of packages for identical repositories.

Foreman and Proxy versions: 2.4.1-1, katello 4.0.3-1

Foreman and Proxy plugin versions:

Distribution and version: CentOS 7.9

Other relevant data:

That is indeed strange. Are all the servers on the same katello version cause there have been some fixes to this in recent months.

1 Like

Katello version is 4.0.3-1.el7.noarch on both. The content view for EPEL has two filters on each server. One if for erratum and the other for rpms. All are inclusion true. Both erratum filter rules have the same end date, are for enhancement, bugfix, and security. I’d like to figure out how to run a hammer command to list the rpms in a content view. Then I could save to a file and compare and see what is missing from one as compared to the other?

You can get into the foreman-rake console and run:

Katello::ContentView.find(:cv_id).versions.find(:version_id).packages to get the exact packages in those versions and compare.

Isn’t this caused by the fact that Epel only ever contains 1 version of a package? So if the servers have a different sync schedule or age, that could be an explanation.

No, they are all on a daily sync schedule. I just re-kicked the one that has different numbers and it still does. Raises the question, “What does the synchronization process do?” If package rpm-1.1.x86_64.rpm is synched, and later that package is removed from the upstream repository, does it automatically get deleted from the downstream satellite server? The other two satellite servers where created quite a bit of time before the one with different numbers.

When I look at the “Upstream URL” from all three servers, it is exactly the same. The number of packages and errata for the two older servers agree at 15079/5007 but the newer server is showing 13676/4707. However, if I go on each server from the command line and issue yum repolist all, all of them agree that the epel repository has 13,669+7. It is baffling.