Some packages are blocked by ou company firewall which makes repository sync fail with a warning resulting in no packages in that repository being synced.
When I try to download those files manually (browser, wget, …) that fails too and I get a message that our corporate firewall detected a ‘possible threat’. Makes sense of course. Unfortunately, there seems to be no way of excluding files from a repository sync and the warnings causes no packages being downloaded/updated at all.
I can try to get an exception from corporate security but I’m afraid it will be a (very) tough nut to crack and if it can be cracked, it will surely take a fair amount of time.
Depending on exactly what you are allowed to do, I can think of several possible workarounds, all with different pitfalls and limitations:
If you are forbidden from having these packages in your Foreman/Pulp instance, then you could try to create a mirror of the upstream repo, that does not include the forbidden packages, and then synchronizing that mirror. This would clearly be a lot of work for creating and maintaining the mirror somewhere outside your firewall.
If the only problem is that you won’t get an exception to the firewall rule, but you can somehow get those packages on to the filesystem of your Foreman/Katello host anyway (for example using scp or similar), then you can do the following:
Use Katello’s upload package feature to manually upload the package to any Katello repository (it does not have to be the same repo you are trying to sync to, and as a matter of best practice I recommend using separate repos for uploading and synchronizing). Once you have uploaded the Package to some repo, re-run your sync. It should now work. The reason is that Pulp uses deduplicated artifact storage, so when it tries to sync the package, it will first check if it already has a package with that checksum. Since this will be the case because you uploaded the package from the Foreman/Katello disk, it won’t try to re-download the package in question and won’t fall afoul of whatever the firewall is doing.
Another possible workaround is not to use download policy immediate for the sync in question (though this might be a bad idea for other reasons, for example if the upstream repo you are syncing does not retain its packages). In that case the sync should be successful since it won’t immediately download any packages. Packages will only be downloaded at the time when a client host actually asks for it. So as long as you do not try to install the forbidden packages anywhere you should be able to avoid the error.
Get the firewall exception you need. Self explanatory.
Option 2 is a pretty smart way to solve this - many thanks. Just tried it, now it’s just a matter of finding out which other packages are blocked by the firewall. Sync bombs out on the very first one and I already found one successor, but now at least I know how to get around.
