Foreman 1.24, but same happened on Nightly. I had BaseOS and AppStream repository synchronized, in a product named CentOS 8
A content view was created for the Prod and Test environments, as follows:
Whenever the content view is presented to a CentOS 8 server, the situation is that:
As You can see, I’m only able to list a few packages availables. Some usefull informations:
- If I create a repository with public streams, onto the server, then I see the correct number of packages
- I had a try to create a different content view, destroy and synchronize again the repos and product, regenerate metadatas either on CV or on repos, present to different VM and physical servers, but nothing has changed so far.
Any help appreciated,
I am seeing the same behavior. Foreman 1.24/Katello 3.14. AppStream shows 5532 pkgs in repo but the client only shows 940. I noticed this when I couldn’t install git. The repo sync’d to the foreman box OK (advanced sync completes with no errors). There does appear to be something amiss here.
What is your repo sync type; ondemand or immediate?
I wonder if this is related to this issue: https://bugs.centos.org/view.php?id=16470
I am seeing this too against 8 and 8.1 RHEL repos. I can see the data downloaded from Redhat is correct, but many packages are missing from the content views I have generated as seen by the client ( although all looks OK when viewing the details in Foreman UI)
This is expected behavior with RHEL 8 Appstream, Centos 8 Appstream, and Fedora Appstream repos. These appstream repositories use a new feature of dnf called ‘modules’. Read more about them here:
Think about modules as ‘sub’ repositories that have to be enabled in order to gain access to their content. This allows for multiple versions of the same software package to be available and install-able.
As an example, I am missing packages for sysstat and git ( @WScottB also mentioned this ). It is not clear to me how this is related to install modules from the Appstream. I plan to spend more time later this week. Perhaps enabling debug in Pulp might help.
Ahhh, i was focused more on the package count disparity on appstream (which is expected), i’ll look into those packages not showing up.
I added back the public AppStream repo in addition to my enabled subscription with AppStream and ran dnf list (with grep -i appstream and sed to replace our company name). Here is an example of what shows up from katello vs what shows up from public repo with no module specifications (sticking with the git example):
geronimo-annotation.noarch 1.0-23.module_el8.0.0+39+6a9b6e22 AppStream
geronimo-annotation.noarch 1.0-23.module_el8.0.0+39+6a9b6e22 COMPANYNAME_CentOS_8_CentOS_8_AppStream
gfbgraph.i686 0.2.3-6.el8 AppStream
gfbgraph.x86_64 0.2.3-6.el8 AppStream
ghc-srpm-macros.noarch 1.4.2-7.el8 AppStream
ghostscript.x86_64 9.25-2.el8_0.3 AppStream
giflib.i686 5.1.4-3.el8 AppStream
giflib.x86_64 5.1.4-3.el8 AppStream
gimp.x86_64 2:2.8.22-15.module_el8.0.0+36+bb6a76a2 AppStream
gimp.x86_64 2:2.8.22-15.module_el8.0.0+36+bb6a76a2 COMPANYNAME_CentOS_8_CentOS_8_AppStream
gimp-devel.x86_64 2:2.8.22-15.module_el8.0.0+36+bb6a76a2 AppStream
gimp-devel.x86_64 2:2.8.22-15.module_el8.0.0+36+bb6a76a2 COMPANYNAME_CentOS_8_CentOS_8_AppStream
gimp-devel-tools.x86_64 2:2.8.22-15.module_el8.0.0+36+bb6a76a2 AppStream
gimp-devel-tools.x86_64 2:2.8.22-15.module_el8.0.0+36+bb6a76a2 COMPANYNAME_CentOS_8_CentOS_8_AppStream
gimp-libs.x86_64 2:2.8.22-15.module_el8.0.0+36+bb6a76a2 AppStream
gimp-libs.x86_64 2:2.8.22-15.module_el8.0.0+36+bb6a76a2 COMPANYNAME_CentOS_8_CentOS_8_AppStream
git-all.noarch 2.18.1-3.el8 AppStream
git-clang-format.i686 7.0.1-1.module_el8.0.0+12+30b38a9a AppStream
git-clang-format.i686 7.0.1-1.module_el8.0.0+12+30b38a9a COMPANYNAME_CentOS_8_CentOS_8_AppStream
git-clang-format.x86_64 7.0.1-1.module_el8.0.0+12+30b38a9a AppStream
git-clang-format.x86_64 7.0.1-1.module_el8.0.0+12+30b38a9a COMPANYNAME_CentOS_8_CentOS_8_AppStream
git-daemon.x86_64 2.18.1-3.el8 AppStream
git-email.noarch 2.18.1-3.el8 AppStream
git-gui.noarch 2.18.1-3.el8 AppStream
git-instaweb.x86_64 2.18.1-3.el8 AppStream
git-subtree.x86_64 2.18.1-3.el8 AppStream
git-svn.x86_64 2.18.1-3.el8 AppStream
gitk.noarch 2.18.1-3.el8 AppStream
gitweb.noarch 2.18.1-3.el8 AppStream
gjs.i686 1.52.5-2.el8 AppStream
gjs.x86_64 1.52.5-2.el8 AppStream
gl-manpages.noarch 1.1-15.20161227.el8 AppStream
glade-libs.i686 3.22.1-1.el8 AppStream
glade-libs.x86_64 3.22.1-1.el8 AppStream
glassfish-el-api.noarch 3.0.1-0.7.b08.module_el8.0.0+39+6a9b6e22 AppStream
glassfish-el-api.noarch 3.0.1-0.7.b08.module_el8.0.0+39+6a9b6e22 COMPANYNAME_CentOS_8_CentOS_8_AppStream
glassfish-fastinfoset-javadoc.noarch 1.2.13-9.module_el8.0.0+42+51564204 AppStream
glassfish-jax-rs-api.noarch 2.0.1-6.module_el8.0.0+42+51564204 AppStream
Note: git didn’t show up here because I had already installed it from public repo.
yep, i have reproduced, digging into it.
It took a while to dig into this, and i got some help from the pulp team (thanks a ton to them!)
I found a few things:
- this only happens within a content view. If you just subscribe to “Library”, the problem isn’t seen.
- The problem is happening due to copying of contents incorrectly at publish time. I’ve tracked it down to a pulp bug: https://pulp.plan.io/issues/5942
- The problem would happen currently regardless of whether or not you are using Content view filters
I imagine it will be a while before the pulp team is able to deliver a full fix for this issue, in the mean time we could fairly easy code fix this for content view publishes that do NOT involve filters (on the appstream repo).
Would that be helpful?
A full fix that also fixes it for filtered-publishes would require a pulp fix.
Actually after discussing it with a fellow katello dev, i went ahead and opened a pr for this here: https://github.com/Katello/katello/pull/8494
Note that this will only help if there are no fitlers assigned to this repository in the content view.
If you are adventurous and want to test the patch:
- apply the patch (https://github.com/Katello/katello/pull/8494.patch) to your tfm-rubygem-katello install
- restart foreman-tasks: systemctl restart dynflowd
- republish the content, promote the new version to your desired lifecycle environment
- The first time you do this, you will need to forcibly regenerate the content view version repository metadata, from the content view versions list:
Let me know how it goes!
Thank you! I’ll try your process and patches tomorrow and report back.
Success! I first used the Default Organization View, with Library which worked. I then tried your patch, published a new view and promoted it. This too worked successfully and I am able to install the required packages.
Thank you very much for taking the time to look at this!
The patch worked for us as well. Thanks!