Content View "Filter" affecting multiple repos

Support
1

Problem:

I have a Content View with 8 repos. In the Content View, I have 1 filter that is configured for 1 “affected repositories”, however, after publishing, it is removing packages from other repositories in my Content View.

Here are screenshots of Include Filter applied to 1 repo:

1st screenshot of Filter:

image
image1311×442 25.4 KB

2nd screeshot of Filter:
image
image1350×982 76 KB

Screenshot of Version before Publishing Filter:

image
image1232×1514 103 KB

Screenshot of the new Version after Publishing Filter (you’ll notice all of the repo counts are different):

image
image1336×1498 134 KB

Expected outcome:
Only the OSSEC repo is affected, however, many repos affected.

Foreman and Proxy versions:
Katello 3.16

Distribution and version:
CentOS 7.8

2

@barn are you using pulp2 or pulp3 to manage yum content?

3

It was installed clean with Katello 3.16. It looks like pulp3 from what i see.

Products > Repos appears to pull all the repository packages without issue. The Repo issue occurs only on Content View versions that have a Filter.

I have confirmed that it happens with both Include and Exclude filters. To elaborate: after the above screenshot, I removed the OSSEC filter, and created a new filter that “Excludes” rhn-client-tools packages from all 8 repositories in Content View. After publishing, I still see many repos in my new Version are empty, just like previous screenshot.

4

@barn thank you for bringing this our attention. I will attempt to reproduce this while investigating possible causes.

5

@barn I attempted to reproduce this with Katello 3.16.1.2 and Katello 3.17.0.rc2.2, using a wildcard “*” in the package rule for an include filter. The problem was just as you described.

I then attempted the same with the nightly katello, and did not have the same issue. The next release is 3.18, and the release candidate which is targeted for a week after the next Foreman release, November 4, so around November 11. That release date is subject to change.

6

Thank you! If you identify a method for me to implement a patch before then, please share.

7

@barn after some discussion with @jjeffers it turns out we haven’t been able to reproduce on 3.16 or 3.17!

After our reproducer attempts, i suspect this doesn’t actually have to do with filters applying incorrectly and has more to do with a copying bug when there is a lot of content. Do you still see the problem if you remove all filters and then:

  • create a new exclude filter that applies to all repos
  • add some package that is only in one repository (some random scl package for example)

and then publish. If you still see ‘empty’ repos, then i think the problem has more to do with publish a large cv with a filter in place.

In the meantime i’m gonna try to reproduce locally with lots of content

8

as an update, i was able to reproduce! I opened an issue here: Bug #31268: publishing a content view w/ a filter with multiple repositories and 10K+ rpms will result in empty repos - Katello - Foreman

we’ll work on fixing it

9

appreciate the update. ill be standing by. if you identify a patch i can apply, please let me know.

10

Looks like bug report is resolved. Is there a possible fix i can manually patch so i dont have to wait for next release?

11

Dropping manual patch steps @Justin_Sherrill shared for reference. I haven’t been able to test it yet though as I ran into a separate roadblock so I have not attempted these steps yet:

  • rpm -ql tfm-rubygem-katello | grep ‘pulp3/repository/yum.rb’
  • should return you the file to edit
  • then find this line: if unit_copy_counter >= UNIT_LIMIT || (i == data_dup.config.size - 1 && leftover_units.empty?)
  • replace it with: if unit_copy_counter != 0
  • then foreman-maintain service restart