Hello!
Problem:
Not able to add/sync Gitlab-ee repo. I use correct baseurl and gpgkey as pointed out here: https://packages.gitlab.com/gitlab/gitlab-ee/install#manual-rpm
But if i try to sync i get warning and im not sure, what im missing.
Has anyone managed to add this repo to the foreman ?
Foreman Katello and versions:
Foreman - 3.15
Distribution and version:
CentOS 7
Can you show the warning?
First guess would be: Have you replaced $basearch with x86_64?
Thanks for the quick replay.
No, i didnt change that. I specified under “Restrict to Architecture” to use x86_64.
Error/Warning:
Action:
Actions::BulkAction
Input:
{“action_class”=>“Actions::Katello::Repository::Sync”,
“target_ids”=>[8],
“target_class”=>“Katello::Repository”,
“args”=>,
“current_request_id”=>nil,
“current_timezone”=>“Europe/Helsinki”,
“current_user_id”=>4,
“current_organization_id”=>1,
“current_location_id”=>2}
Output:
{“planned_count”=>1,
“cancelled_count”=>0,
“total_count”=>1,
“failed_count”=>1,
“pending_count”=>0,
“success_count”=>0}
Exception:
RuntimeError: A sub task failed
…
Did you mean that info/log?
You have to change this, restrict would only limit which packages are downloaded, but the URL has to be complete and the variable is only added by yum and not in Foreman.
I tried it myself and get the error:
PulpCodedException: Malformed repository: metadata is specified for different set of packages in filelists.xml and in other.xml
I downloaded the two files and filelists.xml is “empty” which is not so easy when the repository is not browseable (something I really hate 
) .
As the repository is “broken” you have to contact Gitlab to fix the metadata.
I have tested it with GitLab CE and it gets the same error.
So pulp is checking more than yum. If you set up a local repo file you can access the rpm without problems.
It depends on which operations you do, filelists is only required by yum/dnf when you search for a specific file with provides or if a dependency is written as a file.
So yes, in theory pulp needs more metadata than yum/dnf for its most common operation. But it has to provide a valid repository and if filelist tells there are no packages while other does, I totally understand this is a problem.
To work around this, you could add the repository locally and use some tool like yum-downloader or even reposync and create a local repo with createrepo_c which will create valid metadata.
To expedite this, I have created a topic at gitlab: https://forum.gitlab.com/t/ce-el7-repository-filelists-xml/43102 Let’s see what they say.
It’s going to be an issue for me soon, too…
Same problem with the gitlab-ce repo at https://packages.gitlab.com/gitlab/gitlab-ce/el/7/x86_64
Works fine to mirror it from Spacewalk.
You could “me too” on my topic in the gitlab forum… Maybe someone sees it…
Looks like they’re not gonna fix it.
I guess I have to write a small script to pull packages from that repo and update that repo in the background. That repo is sorta a beast to mirror anyway since they keep all old versions around! Spacewalk have that nice feature of filtering out what you do not want to download and do not see that in Foreman/Katello so even if it would work it would download way to much old packages.
I workaround a different problem with reposync a while ago for Amazon Linux. Perhaps it is also useful here: https://access.redhat.com/solutions/23016
-n is for newest-only which could provide enough filtering for you.
This topic is a bit stale, but maybe this will help someone searching in the future.
I switched from “immediate” to “on demand” in the download policy and the docs say that it only grabs the metadata and downloads the packages as they are requested by clients. My sync worked after. The only caveat being I’m assuming I’ll need to clean up older packages since gitlab-ce rpms are over 1GB each. Actually on demand seems like the overall better option for any repo as we have a consistent production environment, so why mirror 100s or 1000s of packages that are never used?
Thanks for this thread, it got me on the right track. We’re migrating from cobbler and cobbler includes the “-n” option in its reposyncs, so this was never an issue for me before. Maybe one day foreman will have more flexibility to add custom options to a repository reposync.