Is it possible to ignore/skip the file .treeinfo at the top of a remote YUM repository?

Problem:
We have an external repository, which is extremely (hmm, too extremely) strict with their HTTP permissions. You cannot browse it nor can you access the .treeinfo file. You can, however, access the respond.xml file which lists all the packages in it (and you can download them).

Unfortunately, when pulling the repository it stops with this error:
Katello::Errors::Pulp3Error: 403, message='Forbidden', url=URL('http://repo.somewhere.net/yum/abc-1.0/el7-x86_64/.treeinfo')

We used Spacewalk before and it was able to replicate the repo. Is there any setting I can make to allow pulp to mirror such a repository?

Expected outcome:
Pulp should be able to continue and go for the /repodata/repomd.xml file.

Foreman and Proxy versions:
Foreman 2.2.1, Katello 3.17

Foreman and Proxy plugin versions:

Distribution and version:
CentOS 7.9

Other relevant data:

Hey @rbremer,

Sorry to hear about the trouble, what content type is the repository?

-John

I see it’s yum now :slight_smile: @Justin_Sherrill have you seen anything like this with pulp3?

Indeed, yum. I can give you the real URL as a PM, if you need it.

I’m also having the same issue. Here’s the output of Hammer:

$ sudo hammer repository synchronize --id 384
[......................................................................................................................] [100%]
Total steps: 0/0
--------------------------------
Error: 403, message='Forbidden', url=URL('http://download.zfsonlinux.org/epel/7.9/kmod/x86_64/.treeinfo')

Is there any other info I can provide that might be useful?

It seems that it’s a bit too stringent in how it treats this kind of error. If it were a 404, it would probably move-on to the next attempted fetch, and treat that as non-fatal (because not every repo is going to have a .treeinfo file).

But because this is one of those S3-based repos, we get a 403 for trying to fetch a resource that isn’t available (which, technically speaking, goes against the HTTP spec, but that’s neither here nor there).

This gets treated as a fatal error, and all processing stops, which seems a bit extreme to me. It should at least try to fetch /repodata/repomd.xml, which will succeed for this particular repo:

$ curl -skv -X HEAD http://download.zfsonlinux.org/epel/7.9/kmod/x86_64/repodata/repomd.xml