I have it set to Immediate so I don’t have to click sync 24 times on each smatproxy to get new packages/errata everywhere. On repo itself I had it on Immediate and then switched to on demand, no improvements. But I’ve tried running “Complete sync” on smartproxies afterwards it was first synced under Immediate policy. Didn’t help. I’ll try to switch to “on demand” all smart proxies, promote new version of content view and then manually click sync on each to see if that helps, and get back to you with results.
No luck. I’ve made new content version, set everything “On demand”, both repo and smart proxy. Initiated sync manually and:
# cat cc5266ad051eef71ecf4db75bf9700ab6937fb1938fd624526c86b97ad0d267a-updateinfo.xml | wc -l
5552
Same number of lines. Instead of 220k+.
I saw the issue with On Demand myself, but the fact that you were on Immediate before confirms it’s a separate issue. I got a tip that errata may not be copied if its packages are missing from the repository, I’ll have to check if that applies in this case.
1 Like
I doubt that could be the issue. Because for example for missing errata for sed that we used as example earlier.
I see that new package hosted on smart proxy inside os/base repo: sed-4.2.2-6.el7.x86_64.rpm. Errata is in updates repo. Only if everything needs to be in the same repo, which doesn’t make sense. As it works on main server.
I think the answer is somewhere what steps are made when you sync from main server to smart proxy. Something there what it does with repodata creates this issue. Maybe there is too much errata or there is a limit on file size or number of characters or something like that.
It regenerates repodata in some way that most of the updateinfo file is lost.
After talking with the Pulp team, I think I understand what’s going on. With Pulp 3, the errata in the updateinfo file is not filtered at all. Hence, you still see CEBA-2020:1041 on your Katello server. Pulp 2, on the other hand, actually filters out errata from the updateinfo file if the RPMs are missing from the repository. So it seems that, on your smart proxy with Pulp 2, CEBA-2020:1041 is being filtered out due to the missing sed RPM.
As a test, I manually uploaded the sed RPM to my CentOS 7 updates repository. After republishing the content view version and re-syncing my smart proxy, the CEBA-2020:1041 erratum reappeared.
In general it seems that Pulp 2 does not play well with repositories that don’t serve all of the errata RPMs. Since the errata filtering is a feature on Pulp 2 I do not think that will change. This confusion should be gone once Katello is fully on Pulp 3.
Ahaaa, so it is by design. It still doesn’t make sense to me why would this feature exist in pulp2. CentOS has always had separated os, updates, extras repos. But yes, on the other hand, CentOS has no official errata also. But why would pulp care what is in updateinfo. Server is the one who calculates it for itself, anything extra that does not have packages will simply not be applicable?
Do you perhaps know what is the target version for pulp3 being used on smartproxies as well? Is it the next major release Foreman 2.2/Katello 3.17?
If this is what pulp2 does, a possible workaround would then be cloning official mirror for os and extras locally, modifying that repo to include updateinfo file, and syncing foreman from there. That way when smart proxy syncs each repo, pulp2 will keep errata in os that has os related packages errata, same for extras and same for updates. That way the total of errata should be all there but distributed across 3 repos. Only thing I’m worried is will this cause a conflict on main Foreman/Katello server that is using pulp3, having duplicate errata in 3 repos?
I definitely agree there! I’m not totally sure why it exists. Perhaps Pulp 2 is trying to shield clients from “bad” errata metadata. I did find an old Pulp issue that might be related https://pulp.plan.io/issues/1548
Pulp 3 for Smart Proxies probably won’t come until Katello 4.0, which is when Pulp 3 will fully replace Pulp 2.
I think this is a good idea for a workaround. I’m not sure if Katello or Pulp 3 will mind the duplicate errata, but it’s definitely something to try. I’d be curious to hear what happens if you do.
One thing to look out for is a community solution to importing CentOS errata for Pulp 3. I know people made it work for Pulp 2, and I’m guessing someone is working on it for Pulp 3. Katello 3.16rc5 and pulp-admin wrt CentOS Errata
1 Like
I’ve tried the workaround and it works. It seems pulp3 knows this is duplicate errata and counts it as one. There is no tripling on total number. This is shown on content version page. In latest version each repo: os, updates, extra contain the same errata file, but pulp3 knows its all the same so the number is equal compared to first version.
If I run on host
yum update-minimal --security --bugfix -y
It finds all errata and applies it. When checking back host on Content host page after this, I only see applicable errata for kernel. The bug we first discovered that is caused by multiple kernel versions being installed. Hopefully that one will be resolved in the next version.
But as fix for pulp2, this workaround is working awesome. Errata related to packages is distributed in os, updates, extras and in entirety is available to servers despite pulp2 on smart proxies which is most important.
Thank you very much for troubleshooting this with me. I really appreciate it. I think that’s it then. We’ve found and documented all the bugs that I found in this major version.
2 Likes
Glad the workaround works! I’ll bookmark this for future users who may run into this issue as well.
Happy to help, definitely uncovered some good bugs here. I’ll be keeping an eye on their progress 
1 Like