Was the proxy still involved when you tried syncing the repository locally?
Hi @iballou,
Not entirely sure I understand you what you’re after?
If it does help it seems the smaller the repository is the more likely it is to sync.
Once a repository has synchronised I get the duplicates error.
And no, the proxy wasn’t involved in the local sync, I don’t doubt that the proxy may be part of the issue.
I did manage to download the repo to disc and then import the packages individually to pulp, but then when I pointed the pulp repo through the proxy at the repo I got the file validation error again.
Andy
Since you got the error when you served the repository locally, and the proxy wasn’t involved, I’m thinking something else might be at play. Which repository did you clone locally and try syncing from? It would be helpful if we can reproduce the same issue.
Hi @iballou,
It was multiple repositories, but I guess the primary one was Centos 7 base os form the aarnet mirror(or Optus for that matter).
http://mirror.aarnet.edu.au/pub/centos/7/os/x86_64/
But now I’m getting this from an Ubuntu sync:
Katello::Errors::PulpError: PLP0000: [Errno 2] No such file or directory: u’/var/cache/pulp/reserved_resource_worker-0@ednpas3181/3f8cd76f-0110-4d5f-995a-0bc656751e27/packages/21/40/language-pack-gnome-da_20.04+20200416_all.deb’
Maybe I should just go for a fresh install?
Hi @iballou,
I’m forcing immediate because I’m thinking I need it in order to export a content view to another foreman/katello server on an air gapped network. Is that the case, or can I use on Demand?
Andy
Hi @andyfry,
If you’re looking to use content view import/export on Katello 3.16, I’m going to recommend switching to using Pulp 2. The import/export feature is not fully finished for Katello 3.16 with Pulp 3. I haven’t seen documentation about that, so I’ll see about adding it. Apologies about any confusion there.
To switch to Pulp 2, you must:
- Either do a full Katello reset or delete all of your yum, file, and docker repositories.
- Run the foreman-installer with
foreman-installer --katello-use-pulp-2-for-docker 1 --katello-use-pulp-2-for-file 1 --katello-use-pulp-2-for-yum 1
You’re right about using the Immediate download policy for repositories that you want to export.
Hi @iballou,
I found I had to delete the existing repositories before converting to pulp-2 but that was ok. Sadly I’m still getting these file validation errors so I’m thinking the proxy is “interfering” in some way.
Is it worth taking a step back to katello 3.15?
Hi @andyfry,
You’re still getting:
Katello::Errors::Pulp3Error: A file failed validation due to checksum. ?
If it is still the Pulp 3 error then you’re still on Pulp 3.
Hi @iballou,
No it’s not exactly the same error message and certainly no refernce to Pulp 3.
RuntimeError: An error occurred during the sync
{“expected_size”=>85788, “error_code”=>“size_mismatch”, “name”=>“libfdisk-devel”, “unit_key”=>{“epoch”=>“0”, “version”=>“2.32.1”, “name”=>“libfdisk-devel”, “release”=>“22.el8”, “checksum”=>“18ea773537aa6b6776a33e7cac766c1cc776306ead17b4806c5f810ef3f08205”, “arch”=>“i686”, “checksumtype”=>“sha256”}, “actual_size”=>84010}
As an example
Ah, ok. I’m going to do some digging on that error, maybe something has popped up relating to proxies. I don’t think it would be worth going back to Katello 3.15, unless you’ve tried it before with the same proxy configuration.
Cool, will wait to see what you turn up. In the meantime I’ve kinda got around it… I think.
I created local repos so I could avoid the proxy all together for the initial sync, then switched to the aarnet mirror to sync in future. So far so good, but I’m not convinced that the proxy isn’t going to cause me an issue down the track. Interesting though that small repos and indiviual new packages in a repo don’t seem to be as much of an issue.
Interesting that larger volumes of RPM content seems to cause the issue, I’ll keep that in mind.
After talking with the Pulp team, it sounds likely that the issue has more to do with your proxy configuration than with a Pulp 2/3 issue. Issues arise now and then with proxies and Pulp since there are so many different proxies out there.
I might not be super helpful at helping with the proxy configuration, but perhaps if you give us some of your proxy information something worthy of notice might come up.
Also, it’s a long shot, but maybe something in here is helpful? Foreman :: Plugin Manuals
There is also another user who is posting on the pulp mailing list about proxy issues with syncing. Maybe we’ll be able to learn something from them https://www.redhat.com/archives/pulp-list/2020-November/msg00003.html
Hi @iballou,
I did some testing of local packages and tried to down load them through the proxy using wget. Pretty odd results but it seems the content in the proxy was corrupt. I had to use a --nocache to get the valid file to download. Once I’d done that the repositories were able to sync.
I need to find the proxy owners and get them to fix it up!
Thanks for you help.
Andy
1 Like
Wonder if it would be possible to use no-cache on a reposync? It may defeat the traditional purpose of a proxy but for something like this it makes sense. Or have an option to do that.
Pulp Nectar is the software that downloads files for Pulp 2. I’m not positive if it uses wget behind the scenes, but perhaps you could try turning off the cache in your .wgetrc?
cache = on/off
When set to off, disallow server-caching. See the ‘--no-cache’ option.
https://www.gnu.org/software/wget/manual/html_node/Wgetrc-Commands.html#Wgetrc-Commands
Never mind about that, Nectar uses the Python requests library. There might be another workaround within Nectar itself.
Found some interesting info. It turns out --no-cache with wget sets headers Cache-Control: no-cache and Pragma: no-cache, so you could use the same headers within Pulp. The headers would need to be hacked into the Pulp code somewhere, perhaps in this file: https://github.com/pulp/pulp/blob/f231a283e5d377d56b6c2061df9e2d3d1bc81cd9/server/pulp/plugins/util/nectar_config.py#L98-L99
Hopefully though you can get your proxy fixed!
I think the real problem is the proxy so please don’t worry about this any more. Would be handy to manage this in the proxy configuration but I can’t imagine you coming across this often.
I think the real solution for me is to work towards getting the mirrors whitelisted so I can avoid the proxy all together. It would probably be easier than getting the proxy fixed. Politics 
Thanks for everything.
1 Like