Content View publish error when filter applied - Foreman 3.0.1

Support
1

Problem:
Publishing a CV with a filter fails
Both when applying the filter on a published CV but new version and a fresh CV.

Expected outcome:
CV published and packages filtered

Foreman and Proxy versions:
Foreman 3.0.1

Foreman and Proxy plugin versions:
Katello 4.2.1

Distribution and version:
RHEL 8.5

Other relevant data:
Upgraded from foreman 2.5 to 3.0.1 about 2 weeks ago. Repo syncs and CV publish when no filter is present run without issue. Applying a filter to a published CV as well as publishing a new CV with a filter result in errors.

In Dynflow we observe that the error takes place at step : Actions::Pulp3::Repository::CopyContent

Below the message displayed in Foreman UI

Error message: the server returns an error
HTTP status code: 400
Response headers: {"date"=>"Fri, 26 Nov 2021 12:11:48 GMT", "server"=>"gunicorn", "content-type"=>"application/json", "vary"=>"Accept,Cookie", "allow"=>"POST, OPTIONS", "x-frame-options"=>"SAMEORIGIN", "content-length"=>"202", "correlation-id"=>"e512c836-79d5-4c60-947e-205b2dc1033e", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 foremanserver.domain", "connection"=>"close"}
Response body: ["Could not find the following content units: ['/pulp/api/v3/content/rpm/advisories/3d0b3867-5689-4be5-8ed6-8a3a3721e0b0/', '/pulp/api/v3/content/rpm/advisories/c77fd86a-5379-45a1-b790-ac16aed44a94/']"]Error message: the server returns an error
HTTP status code: 400
Response headers: {"date"=>"Fri, 26 Nov 2021 12:11:27 GMT", "server"=>"gunicorn", "content-type"=>"application/json", "vary"=>"Accept,Cookie", "allow"=>"POST, OPTIONS", "x-frame-options"=>"SAMEORIGIN", "content-length"=>"202", "correlation-id"=>"e512c836-79d5-4c60-947e-205b2dc1033e", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 foremanserver.domain", "connection"=>"close"}
Response body: ["Could not find the following content units: ['/pulp/api/v3/content/rpm/advisories/683600e2-c300-4181-babb-b14f687e16c7/', '/pulp/api/v3/content/rpm/advisories/3671b62e-0808-42c8-abf9-9ec4a03b1830/']"]

Filter we are trying to use (displays matching content as expected):
exclude kernel* Greater than version 4.18.0-348.el8.x86_64

Repos in the CV:

  • Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8
  • Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8
2

Hi @smoyson,
Not sure how much this will help you, but I happen to create a VM with Foreman 3.01 and Katello 4.2.1 to test my own RHEL errata issues. So I tried to recreate your issue.

I created a content view with the two RHEL8 repos specified and was able to publish the content view without filters. I then added an exclusion filter for “kernel*” packages like so:

# hammer content-view filter info --content-view-id 3 --id 3
Filter ID:    3
Name:         Exclude Kernel Packages
Type:         rpm
Inclusion:    false
Description:  
Repositories: 

Rules:        
 1) Id:              1
    Name:            kernel*
    Minimum Version: 4.18.0-348
    Created:         2021/11/30 13:21:49
    Updated:         2021/11/30 13:21:49

I was also able to publish this filtered content view without issue. One difference between our setups is that I’m running Foreman on CentOS 7 rather than RHEL 8.5.

2 Likes
3

hi @cbcbcb

I checked the output of the hammer content on my end, as i have mainly been using the GUI, and noticed that my filter rule was slightly different from yours:

Minimum Version: 4.18.0-348.el8.x86_64

I edited the filter to no longer include “.el8.x86_64” so it matches yours which works on your CentOS deployment.

Sadly it still throws the same error.

I additionally tested adding a CentOS 8 CV with the exact same filter which did get published successfully.

Any other tests or troubleshooting steps to help solve this issue would be much appreciated.

Sincerely

4

Hello

I have done some extra tests and the issue seems to only apply to the 2 RHEL 8 repos. In a test CV containing either one of the RHEL8 repos by itself it gave the same error messages as in my original post.

Could anyone advise on any extra steps to take to resolve the issue or further determine the cause?

Thank you in advance

5

@smoyson I’m taking a look. In the mean time can you run foreman-rake katello:delete_orphaned_content on your server and then try the publish again?

6

@Jonathon_Turel I ran the delete_orphaned_content and retried publishing the CV. It still gives the same error message.

Error message: the server returns an error
HTTP status code: 400
Response headers: {"date"=>"Wed, 15 Dec 2021 10:36:11 GMT", "server"=>"gunicorn", "content-type"=>"application/json", "vary"=>"Accept,Cookie", "allow"=>"POST, OPTIONS", "x-frame-options"=>"SAMEORIGIN", "content-length"=>"202", "correlation-id"=>"81ac5257-d030-4846-afaf-486964511be4", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 foremanserver.domain", "connection"=>"close"}
Response body: ["Could not find the following content units: ['/pulp/api/v3/content/rpm/advisories/3d0b3867-5689-4be5-8ed6-8a3a3721e0b0/', '/pulp/api/v3/content/rpm/advisories/c77fd86a-5379-45a1-b790-ac16aed44a94/']"]Error message: the server returns an error
HTTP status code: 400
Response headers: {"date"=>"Wed, 15 Dec 2021 10:35:45 GMT", "server"=>"gunicorn", "content-type"=>"application/json", "vary"=>"Accept,Cookie", "allow"=>"POST, OPTIONS", "x-frame-options"=>"SAMEORIGIN", "content-length"=>"125", "correlation-id"=>"81ac5257-d030-4846-afaf-486964511be4", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 foremanserver.domain", "connection"=>"close"}
Response body: ["Could not find the following content units: ['/pulp/api/v3/content/rpm/advisories/3671b62e-0808-42c8-abf9-9ec4a03b1830/']"]
7

@smoyson maybe have a look at this post: Strange pulp3 error - #35 by iballou
I had a similar error. After deleting the corrupted erratas i was able to republish my content views, although sometimes after a few days when I want to create a new Version I get the same error with different erratas.

1 Like
8

Hello sorry for the late reply

Hope everyone was able to enjoy the holidays.

@Jonathon_Turel
What would the impact be of the actions from the post @greg referred to?
Is there be something else I should try first?

9

Honestly I am not sure! It was an interesting read. @iballou do you feel that your solution in Strange pulp3 error - #35 by iballou is applicable here?

10

Hi @smoyson,

The steps in that post you found should still apply. Here’s what I would do:

  1. Run this command in the Foreman console and check that the repositories returned are in your broken content view:
::Katello::Repository.where(id: ::Katello::RepositoryErratum.where(erratum_pulp3_href: [<comma separated list of the missing content units just like the error>]).pluck(:repository_id)).map{|r| r.root.name}
  1. Run a complete sync on those repositories
  2. Try publishing again
  3. If you get the same error, then delete the RepositoryErratum records.
  4. Publish again

Deleting the RepositoryErratum records is safe. Worst case, if you delete the wrong ones, you will just need to reindex your repositories. That can be done by doing a complete sync, or by using foreman-rake katello:reimport, for example.

1 Like