How to handle Erratas when clients are locked to kernel versions

Support
1

Hi there,

we’ve got a number of Rocky8 clients which are enrolled into Foreman/Katello for content subscriptions. Those clients are locked to a specific kernel version using dnf versionlock. This is unfortunately necessary due to storage client requirements.

When I check these hosts in the foreman gui all of them are reporting some Erratas available. For example there’s a bug fix for package kmod-kvdo which requires a newer kernel-modules version. However due to the dnf versionlock on the kernel-modules package, trying to install/apply the Errata will result in a failure.

Is there a way to tell either the client, subscription manager or foreman/katello to ignore certain packages? I know it’s a long shot but I’d prefer to not have all clients in the foreman gui showing error-statuses.

Thanks,

Foreman and Proxy versions:
foreman: 3.8.0

Foreman and Proxy plugin versions:
foreman-tasks 8.2.0
foreman_puppet 6.0.1
foreman_remote_execution 11.0.0
katello 4.10.0

Distribution and version:
Rocky Linux release 8.8 (Green Obsidian)

2

Well, I guess the short answer is “the feature works as intended” as you do have outstanding errata on those hosts and you really should be yelling at that storage client vendor. But while being technically correct, this answer is not really helpful, so let’s see.

The upload of package lists happens via subscription-manager, and you probably could adjust their code to filter out certain packages:

If a package is not reported as installed, it can’t have errata, right? :wink:
But that would require a patch to sub-man on every host, as there is today no way to locally configure an exclude list.

I don’t see a way to achieve the same on the @katello side, but maybe someone knows batter :wink:

3

Hey @MajorNickle

Maybe Using the KernelCare Plug-in in Managing Hosts helps with your use case?

4

Thanks @evgeni and @maximilian. I know it’s an edge case and more a cosmetic/reporting thing I’m trying to achieve here. I’ll checkout your both suggestions :slight_smile: Thanks!

5

Another possibility is a content view filter, which could filter out packages and/or errata by date. The errata then would show as applicable, but not installable. And you could set the setting “Generate errata status from directly-installable content” to Yes, which would avoid the negative host status.

1 Like
6

That’s a very great idea! Let me try that right now. I might create a separate content view for those hosts and exclude the package group. Thanks @jeremylenz

1 Like
7

@jeremylenz I create a seperate content view and applied the below filters. When publishing a new version which includes the filters the task just sits there and doesn’t progress. Looking at the running steps it seems to attempt to poll some data. It’s not failing but attempting over and over again as I can see the counter going up every couple of seconds. Any idea what would cause that?

image
image1686×1232 98.7 KB

{"pulp_href"=>"/pulp/api/v3/tasks/018bb4ad-68bf-7f0d-8c4b-ff1a0c122cb6/",
    "pulp_created"=>"2023-11-09T15:22:25.344+00:00",
    "state"=>"waiting",
    "name"=>"pulp_rpm.app.tasks.copy.copy_content",
    "logging_cid"=>"95a2bbe0-83b5-44e9-8f6c-83b375b10b82",
    "created_by"=>"/pulp/api/v3/users/1/",
    "child_tasks"=>[],
    "progress_reports"=>[],
    "created_resources"=>[],
    "reserved_resources_record"=>
     ["/pulp/api/v3/repositories/rpm/rpm/018bb4a6-4ba9-7c91-bdc1-fc955ca0eee0/",
      "/pulp/api/v3/repositories/rpm/rpm/018bb4a6-4bb7-7ae7-a4b6-ae464dbb1bd7/",
      "/pulp/api/v3/repositories/rpm/rpm/018bb4a6-4b4e-7abc-bb56-abcad4e71a4f/",
      "/pulp/api/v3/repositories/rpm/rpm/018bb4a6-4bea-7466-be3a-712bd967fbb0/",
      "/pulp/api/v3/repositories/rpm/rpm/018bb4a6-4be7-7e1d-b27e-b94b9d399b20/",
      "/pulp/api/v3/repositories/rpm/rpm/018bb4a6-4c90-7a55-ad58-ef76b6c9dbc4/",
      "shared:/pulp/api/v3/repositories/rpm/rpm/5ce533bc-9cdf-434c-830a-721e560d6777/",
      "shared:/pulp/api/v3/repositories/rpm/rpm/9051a052-cbcf-48b4-a531-d553a095e25c/",
      "shared:/pulp/api/v3/repositories/rpm/rpm/d99c1027-a5b5-4f42-8da4-05dbc366faf4/",
      "shared:/pulp/api/v3/repositories/rpm/rpm/b3e51704-9214-44ce-a364-18d6af2385a7/",
      "shared:/pulp/api/v3/repositories/rpm/rpm/7c046047-7216-4640-8312-b42192a66b35/",
      "shared:/pulp/api/v3/repositories/rpm/rpm/0ec61da4-9ba0-4d9a-b6fe-2c78f5807253/"]}],
 "task_groups"=>[],
 "poll_attempts"=>{"total"=>91, "failed"=>0}}
8

Does hammer ping or foreman-maintain service status show all services running?

You can also see what Pulp is up to with journalctl -u *pulp*

9

hammer ping and foreman-maintain service status show all services are running.

I don’t see any errors being thrown looking at journalctl. But below is an example what’s currently being logged.

Nov 09 07:31:23 foreman pulpcore-api[450624]: pulp [95a2bbe0-83b5-44e9-8f6c-83b375b10b82]:  - - [09/Nov/2023:15:31:23 +0000] "GET /pulp/api/v3/tasks/018bb4ad-635e-729a-82ee-d4aa3b43b2e0/ HTTP/1.1" 200 1394 "-" "OpenAPI-Generator/3.28.11/ruby"
Nov 09 07:31:23 foreman pulpcore-api[443853]: pulp [95a2bbe0-83b5-44e9-8f6c-83b375b10b82]:  - - [09/Nov/2023:15:31:23 +0000] "GET /pulp/api/v3/tasks/018bb4ad-68bf-7f0d-8c4b-ff1a0c122cb6/ HTTP/1.1" 200 1394 "-" "OpenAPI-Generator/3.28.11/ruby
10

Looking at the dynflow console its sitting at step 50. I was able to publish a version without filters just fine but as soon as I add a filter its not getting past this step.

image
image1348×259 20.6 KB

11

Hey @MajorNickle ,

Since I see MultiCopyContent there that means you’re using dependency solving – it dramatically slows down content view publishes. We have some warnings in the UI about it, but I would say to use it only if you really must.

Edit: also make sure that you’re using the latest Pulp packages, there was a bug fixed recently that might’ve caused slower than usual Pulp content copying.

2 Likes
12

@jeremylenz alright I think I got it working. Good old selinux was preventing /usr/bin/rpm from getattr access on /var/lib/rpm/Packages. After running restorecon -v '/var/lib/rpm/Packages' it’s now progressing.

3 Likes
13

Awesome! I’d expect pulp to tell us that and fail if it’s not doing so instead of appearing to hang…Debugging nightmare… :smiley:

1 Like
14

Yeah, I only noticed it when looking at the foreman.service output.

Thanks everyone for the input and suggestions. Much appreciated!

15

I’m still fairly new to Katello, could you tell me how to check? I’m on foreman: 3.8.0 and katello 4.10.0. On the about page I have following pulp packages installed.

pulpcore-obsolete-packages-1.0-2.el8.noarch
pulpcore-selinux-1.3.3-1.el8.x86_64
python39-pulp-ansible-0.18.1-1.el8.noarch
python39-pulp-certguard-1.6.5-1.el8.noarch
python39-pulp-cli-0.21.2-1.el8.noarch
python39-pulp-container-2.15.2-1.el8.noarch
python39-pulp-deb-3.0.0-1.el8.noarch
python39-pulp-file-1.14.3-1.el8.noarch
python39-pulp-glue-0.21.2-1.el8.noarch
python39-pulp-python-3.10.0-1.el8.noarch
python39-pulp-rpm-3.22.3-1.el8.noarch
python39-pulpcore-3.28.16-1.el8.noarch
rubygem-pulp_ansible_client-0.18.0-1.el8.noarch
rubygem-pulp_certguard_client-1.6.5-1.el8.noarch
rubygem-pulp_container_client-2.15.2-1.el8.noarch
rubygem-pulp_deb_client-3.0.0-1.el8.noarch
rubygem-pulp_file_client-1.14.3-1.el8.noarch
rubygem-pulp_ostree_client-2.1.1-1.el8.noarch
rubygem-pulp_python_client-3.10.0-1.el8.noarch
rubygem-pulp_rpm_client-3.22.3-1.el8.noarch
rubygem-pulpcore_client-3.28.11-1.el8.noarch
rubygem-smart_proxy_pulp-3.2.0-3.fm3_3.el8.noarch
16

To check you just need to run a yum update and see if there’s an available Pulpcore update :+1:

17

Ah gotcha. I’m on the latest then. I think as you mentioned before the dependency solving I had enabled contributed to the slow publishing. I’ve disabled it as I noticed it does seem to interfere with the content view filters. I’m all good now :slight_smile:

Thanks again everyone!

1 Like
18

When I need to exclude newer kernels (for out-of-tree kmod reasons), I do something like

pattern arch version
bpftool All architectures Greater than version 4.18.0-425.3.1.el8
kernel* All architectures Greater than version 4.18.0-425.3.1.el8
*perf All architectures Greater than version 4.18.0-425.3.1.el8
kmod-kvdo All architectures Greater than version 6.2.7.17-87.el8

I don’t know if this is relevant to your use case, but if you exclude the package group you may have trouble using the CV to Kickstart.

I would also suggest you make sure you are only applying filters to BaseOS if that’s where all the packages you need to exclude are from, since in my experience that is (a) faster to publish and (b) in the past I’ve lost modulemd_defaults by applying even non-matching filters to AppStream or other repositories.

1 Like
19

Right now I have individuals packages excluded just like you in your example. But I didn’t know or see that you can apply it to only a subset of repositories. That’s a very great tip! Thanks a lot :slight_smile:

RPM name Architecture
kernel All architectures
kernel-core All architectures
kernel-devel All architectures
kernel-headers All architectures
kernel-modules All architectures
kernel-tools All architectures
kernel-tools-libs All architectures
kernel-tools-libs-devel All architectures
kmod All architectures
kmod-devel All architectures
gcc All architectures
elfutils-libelf-devel All architectures
kmod-kvdo All architectures