Katello::Errors::Pulp3Error: 'arm64' by a Debian Bullseye Repo for HashiCorp

jeschero · February 8, 2022, 3:27pm

Problem:
I create a Debian Bullseye Repo for HashiCorp and get an Error by the sync:

Output:

{"response"=>
  {"task"=>"/pulp/api/v3/tasks/cd36573c-dea0-4e2b-a1b8-e7221a87abe4/"},
 "pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/cd36573c-dea0-4e2b-a1b8-e7221a87abe4/",
    "pulp_created"=>"2022-02-05T03:07:32.139+00:00",
    "state"=>"failed",
    "name"=>"pulp_deb.app.tasks.publishing.publish",
    "logging_cid"=>"61c2c794-3049-4173-83ec-228add4d29c5",
    "started_at"=>"2022-02-05T03:08:14.458+00:00",
    "finished_at"=>"2022-02-05T03:08:59.253+00:00",
    "error"=>
     {"traceback"=>
       #<Sequel::SQL::Blob:0x39c74 bytes=139 start="  File \"/o" end="form_task\n"> +
       #<Sequel::SQL::Blob:0x39c88 bytes=35 start="    result" end="**kwargs)\n"> +
       #<Sequel::SQL::Blob:0x39c9c bytes=130 start="  File \"/o" end="n publish\n"> +
       #<Sequel::SQL::Blob:0x39cb0 bytes=76 start="    releas" end="_package(\n"> +
       #<Sequel::SQL::Blob:0x39cc4 bytes=134 start="  File \"/o" end="d_package\n"> +
       #<Sequel::SQL::Blob:0x39cd8 bytes=54 start="    self.p" end="cture][0]\n">,
      "description"=>"'arm64'"},
    "worker"=>"/pulp/api/v3/workers/b3e49c04-3acc-4002-8590-987427475e12/",
    "child_tasks"=>[],
    "progress_reports"=>[],
    "created_resources"=>[],
    "reserved_resources_record"=>
     ["shared:/pulp/api/v3/repositories/deb/apt/e57d7614-7eaf-4f36-aca7-e38687185bb6/"]}],
 "task_groups"=>[],
 "poll_attempts"=>{"total"=>9, "failed"=>1}}

Exception:

Katello::Errors::Pulp3Error: 'arm64'

Expected outcome:

Sync is succesfull

Foreman and Proxy versions:

Foreman/Proxy 3.1.1

Foreman and Proxy plugin versions:

foreman-tasks 5.2.0
foreman_ansible 7.0.2
foreman_puppet 	2.0.0
foreman_remote_execution 5.0.1
katello 4.3.0

Distribution and version:

NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.9"
PRETTY_NAME="Red Hat Enterprise Linux"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.9
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.9"
Red Hat Enterprise Linux Server release 7.9 (Maipo)
Red Hat Enterprise Linux Server release 7.9 (Maipo)

Other relevant data:

Repo settings:

Sync Settings

Upstream URL    https://apt.releases.hashicorp.com/
Releases        bullseye
Components      main
Architectures   amd64
Verify SSL      Yes
Upstream Authorization
HTTP Proxy      Global Default (None)
Publish via HTTPS   Yes
Unprotected     Yes
Published At    https://<server_dns>/pulp/content/tecin_net/Library/custom/HashiCorp/DebianBullseye_HashiCorp/ 
GPG Key         RPM-GPG-KEY-HASHICORP (https://apt.releases.hashicorp.com/gpg)
SSL CA Cert
SSL Client Cert
SSL Client Key
Download Policy   Immediate
Mirroring Policy  Content Only

quba42 · February 8, 2022, 5:04pm

This is a known issue, which has been fixed on the pulp_deb main branch, but has not yet been released:

github.com/pulp/pulp_deb

Handle upstream package indices containing packages with the wrong architecture

opened 12:52PM - 05 Jan 22 UTC

closed 01:28PM - 26 Jan 22 UTC

quba42

.bugfix

Minimal reproducer: ``` pip install pulp-cli-deb # Constants: ENTITIES_NAM…E='test' REMOTE_OPTIONS='--url=https://apt.releases.hashicorp.com --distribution=bionic --component=main --architecture=amd64 --policy=on_demand' # Steps using CLI: pulp deb remote create --name=${ENTITIES_NAME} ${REMOTE_OPTIONS} pulp deb repository create --name=${ENTITIES_NAME} --remote=${ENTITIES_NAME} pulp deb repository sync --name=${ENTITIES_NAME} APT_PUBLICATION_HREF=$(pulp deb publication create --repository=${ENTITIES_NAME} --simple=True --structured=True | jq -r '.pulp_href') # Creating the publication fails! pulp deb distribution create --name=${ENTITIES_NAME} --base-path=${ENTITIES_NAME} --publication=${APT_PUBLICATION_HREF} ``` Error message: ``` "error"=> {"traceback"=> " File \"/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py\", line 317, in _perform_task\n" + " result = func(*args, **kwargs)\n" + " File \"/usr/lib/python3.6/site-packages/pulp_deb/app/tasks/publishing.py\", line 168, in publish\n" + " prc.package\n" + " File \"/usr/lib/python3.6/site-packages/pulp_deb/app/tasks/publishing.py\", line 207, in add_package\n" + " self.package_index_files[package.architecture][0]\n", "description"=>"'arm64'"}, ```

It is mostly an error with the Hashicorp repo’s structure I might add, but I have not yet managed to find a channel to talk to any Hashicorp engineers: Buggy/weirdly structured HashiCorp APT repo - What's Your Story? - HashiCorp Discuss (I tried emailing their support but they pointed me at the community forum).

Anyway, the above pulp_deb fix will work around the errors in the Hashicorp repo.

I can have a look into backporting the fix to pulp_deb 2.16 to make it available to Katello 4.3.

jeschero · February 8, 2022, 9:14pm

This is a known issue, which has been fixed on the pulp_deb main branch, but has not yet been released:

Thanks for this Information, i haven’t found this bug report.

I can have a look into backporting the fix to pulp_deb 2.16 to make it available to Katello 4.3.

If it is not too much effort, this would be great.

Thank you.