Client would download wrong packages after publishing new version

Problem:
After publishing a new version in my fedora-ccv client cannot update some packages, because he is searching for a older version, which not exists in my repository.

example:
dnf update -y
[MIRROR] abrt-addon-ccpp-2.13.0-1.fc31_2.13.0-2.fc31.x86_64.drpm: Status code: 404 for https://xxx.de/pulp/repos/bkg/Library/ccv_fedora_base/custom/Fedora/Fedora31_updates/drpms/abrt-addon-ccpp-2.13.0-1.fc31_2.13.0-2.fc31.x86_64.drpm (IP: 141.74.4.1)

my fedora_updates repo includes this version:
abrt-addon-ccpp-2.14.0-1.fc31.x86_64

my fedora_base repo includes this version:
abrt-addon-ccpp-2.12.2-1.fc31.x86_64

Expected outcome:
dnf update works
Foreman and Proxy versions:
Satellite Server 6.6.2
Foreman and Proxy plugin versions:

Distribution and version:
Red Hat Enterprise Linux Server release 7.7 (Maipo)

Other relevant data:

Hi,

just to be sure: Have you cleaned the dnf cache beforehand? (Some dnf variant of yum clean all)
I only had this behavior happen in two scenarios by now:

  • Package manager cache not updated
  • Yum versionlock plugin doing strange things (I don’t know if dnf has a versionlock plugin)
    If that does not help, you could try rebuilding the CV/CCV metadata via Foreman UI. There is an option “regenerate repository metadata” on the CV Versions page, hidden behind the arrow next to the promote button. Try this for both the affected CV and CCV.
    Hope this helps.

Regards

I am also not able to provision new clients in this version. Installation works, but updating in post-scripts fails.

I’ve tested:
regenerate metadata
cleared cache
created new CV&CCV
changed mirror for product sync

nothing works.

working with satellite since 6 years. Never seen this situation.

Yum versionclock is not enabled.

ll /etc/yum/pluginconf.d/
total 32
-rw-r–r--. 1 root root 59 Nov 14 2018 enabled_repos_upload.conf
-rw-r–r--. 1 root root 279 Aug 9 2019 fastestmirror.conf
-rw-r–r--. 1 root root 372 Feb 20 12:10 langpacks.conf
-rw-r–r--. 1 root root 323 Oct 8 2013 langpacks.conf.rpmnew
-rw-r–r--. 1 root root 59 Nov 14 2018 package_upload.conf
-rw-r–r--. 1 root root 17 Sep 13 18:15 product-id.conf
-rw-r–r--. 1 root root 858 Sep 13 18:15 search-disabled-repos.conf
-rw-r–r--. 1 root root 17 Sep 13 18:15 subscription-manager.conf

This is indeed very strange. After looking into this, it seems like that package was not even released in the version your client requests.
I don’t have a lot of experience with dnf, but my best guess would be that another package you are trying to upgrade might be depending on the wrong version of abrt-addon-ccpp? From how far my google-foo got me, you could try something like “dnf repoquery --whatdepends abrt-addon-ccpp”, check the depending packages and see how far that gets you.

dnf repoquery --whatdepends abrt-addon-ccpp
Updating Subscription Management repositories.
Warning: failed loading ‘/etc/yum.repos.d/adobe-linux-x86_64.repo’, skipping.
Fedora31 88 kB/s | 3.6 kB 00:00
pulp_latest 53 kB/s | 2.1 kB 00:00
Fedora31_updates 95 kB/s | 3.9 kB 00:00
Katello3_3_5 49 kB/s | 2.1 kB 00:00
abrt-cli-0:2.12.2-1.fc31.x86_64
abrt-cli-0:2.13.0-2.fc31.x86_64
abrt-desktop-0:2.12.2-1.fc31.x86_64
abrt-desktop-0:2.13.0-2.fc31.x86_64
abrt-tui-0:2.12.2-1.fc31.x86_64
abrt-tui-0:2.13.0-2.fc31.x86_64

Hm, interesting. According to fedora.pkgs.org, those abrt-* packages should also be available in version 2.14.
Could you check if those are available in your CV/CCV in 2.14 or if the newest version you have of those is in fact 2.13? If it is the latter, you might have hit two incomplete mirrors in a row. You might also want to check the mirrors you are using directly via your browser to see if the 2.14 packages are available there.

Only have 2.14.

Requires

/bin/sh

/usr/bin/python3

/usr/bin/sh

abrt = 2.14.0-1.fc31

abrt-libs = 2.14.0-1.fc31

abrt-retrace-client

cpio

elfutils

gdb-headless

libabrt.so.0()(64bit)

libc.so.6(GLIBC_2.4)(64bit)

libgcc_s.so.1()(64bit)

libgcc_s.so.1(GCC_3.0)(64bit)

libgcc_s.so.1(GCC_3.3.1)(64bit)

libglib-2.0.so.0()(64bit)

libreport.so.0()(64bit)

libsatyr.so.4()(64bit)

libsystemd.so.0()(64bit)

libsystemd.so.0(LIBSYSTEMD_209)(64bit)

python3-libreport

rtld(GNU_HASH)

Provides

abrt-addon-ccpp(x86-64)-2.14.0-1.fc31

abrt-addon-ccpp-2.14.0-1.fc31

config(abrt-addon-ccpp)-2.14.0-1.fc31

Mirror also includes only 2.14.
will check other mirrors as well.

checked another 2 mirrors. --> same situation.

Base repo includes 2.12. Update repo includes 2.14.

Have you resolved this by now?
If not, my last guess would be checking what the repos look like on the filesystem. I’m really poking in the dark here, though.
In your Pulp proxy/Foreman Server, the repos should be under /var/lib/pulp/published/yum/https/repos/bkg/Library/ccv_fedora_base/custom/Fedora/Fedora31_updates/
tbh, it somewhat looks like corrupted metadata to me, but that should have resolved itself by now. So I’m really out of ideas :confused:

No. Could not resolve this issue by now.
looking as intended.

[root@xsatellite Packages]# find * -iname abrt-addon*
a/abrt-addon-vmcore-2.14.0-1.fc31.x86_64.rpm
a/abrt-addon-pstoreoops-2.14.0-1.fc31.x86_64.rpm
a/abrt-addon-ccpp-2.14.0-1.fc31.x86_64.rpm
a/abrt-addon-xorg-2.14.0-1.fc31.x86_64.rpm
a/abrt-addon-kerneloops-2.14.0-1.fc31.x86_64.rpm
a/abrt-addon-upload-watch-2.14.0-1.fc31.x86_64.rpm