Zst compression support?

Support
1

Problem:
Unable to sync repository from Index of /rpm/stable (or Index of /rpm/unstable).
Repodata directories contain filenames like “9f21555a2b04c3e0b32eb8ae186cad8678992ddc606e850cc9bf193ff89c21a7-primary.xml.zst” which lead me to believe zstandard compression is being used, but Foreman tells me:
Cannot open /var/lib/pulp/tmp/149199@foreman.tindall/tmp2qyu04_3/tmp6cpnp22_-9f21555a2b04c3e0b32eb8ae186cad8678992ddc606e850cc9bf193ff89c21a7-primary.xml.zst: Cannot detect compression type

Expected outcome:
normal sync

Foreman and Proxy versions:
foreman-3.7.0-1.el8.noarch
katello-4.9.2-1.el8.noarch

Foreman and Proxy plugin versions:
foreman-proxy-3.7.0-1.el8.noarch

Distribution and version:
Rocky 8.8

Other relevant data:
zstd and libs installed:
(19:14:15) root@foreman:pulp# rpm -qa | grep zst
zstd-1.4.4-1.el8.x86_64
libzstd-1.4.4-1.el8.x86_64
libzstd-1.4.4-1.el8.i686

FWIW, having the same issue on my old foreman-3.3.1-1.el7.noarch/katello-4.5.1-1.el7.noarch machine which I am in the process of replacing.

1 Like
2

Does Katello not support Zst? Found this thread after getting the same error with sync’ing one of my own custom repos. Luckily, I can just change the compression type to xz, but that may not be the case for other remote repos.

FWIW, the reason why Zst was used to compress my repodata was that the latest createrepo on el8 appears to be using Zst by default:

# rpm -qa | grep createrepo
createrepo_c-libs-1.0.2-2.el8.x86_64
createrepo_c-1.0.2-2.el8.x86_64
python39-createrepo_c-1.0.2-2.el8.x86_64
3

ZST compression support required a Pulp fix. Per 2241934 – After upstream repo switched to zst compression, Satellite 6.12.5.1 unable to sync, it’s

Fixed In Version: pulp_rpm-3.18.19, pulp_rpm-3.19.11, pulp_rpm-3.22.6 → pulp_rpm-3.18.19, pulp_rpm-3.19.11, pulp_rpm-3.22.6, createrepo_c-1.0.1

As far as I can tell, this didn’t make it into Katello 4.10, since Index of /katello/4.10/katello/el8/source shows pulp_rpm-3.22.3. So it would be resolved in Katello 4.11 once we upgrade Pulp again.

cc @iballou, I’m not sure if we were planning any backports around this?

2 Likes
4

Packaging folks like @pcreech and @Odilhao help keep the Pulp versions up to date. Looks like Katello 4.10 is getting the update: Update python-pulp-rpm to 3.22.6 · theforeman/pulpcore-packaging@805e88a · GitHub

Katello 4.9 hasn’t gotten it yet. All we would need would be a PR to bump the spec and pull in a new tar file for pulp_rpm-3.19.11 here: https://github.com/theforeman/pulpcore-packaging/tree/rpm/3.22/packages/python-pulp-rpm

2 Likes