Dealing with vendored content, specially rust

While updating some packages for our Pulpcore RPMs, we discovered that the newest release of python-cryptography now requires rust, and with this comes something new and fun, vendoring crates.

@ekohl raised a good point, that adding vendor tarballs to git will make things complicated in the future.

I’m creating this topic to gather feedback on the best approach for this. Right now Fedora does not provide the vendor/tarball at the moment, and even it did they don’t have the necessary version of cryptography built.

Should we look for one alternative option? Maybe hosting the vendor crates like we do for foreman tarballs, maybe we can automate this with Jenkins and also sign the content to make sure that it’s the one that we want.

I agree that adding such a file to git will become complicated. Simplest option would be to upload the vendorized tarball to our existing file hosting in foreman, and just point to it with the appropriate source option.

If we feel it is worth the time and investment, we could also work towards a workflow of generation + upload + signing of said blob, to ensure it doesn’t get changed.

1 Like

Should we host this inside the yum.theforeman.org? My idea is to create a new folder called vendor in the top level.

For the signing part, we can manually sign and document the procedure in the Pulpcore section in tool-belt and later automate like we have for Foreman tarballs.

I think yum.theforeman.org should be limited to hosting RPMs. This is all naming (because we serve it from the same infra), but downloads.theforeman.org feels like a better place.

One thing to note is that today we don’t have a great way to easily upload content, but it will all depend on how often it will need to change. It’s quite a difference if you do something daily compared to yearly.

Right now the only package that we have that needs vendor is python-cryptografy, for now I can manually upload/sign the tarball with all vendor content for this release. I dind’t remember that we had downloads.theforeman,org

What structure should we create there? One new folder called vendor? or one for Pulp with Vendor inside?

Would you mind creating a HEADER.html file in there describing what the purpose of the directory is? This is something I’d prefer to have for every directory, but let’s start with doing so for new directories.