Can't sync certain repos after Katello 4.0.3 / Foreman 2.4.1 upgrade

Support
1

Problem:
I can’t sync certain repos after upgrading to Katello 4.0 / Foreman 2.4. Previous version was Foreman 2.3.5 / Katello 3.18.5. They hang at syncing (3%) in the console. I’m trying to get us to a supported version of Katello and Foreman

I believe this originally had to do with ALLOWED_CONTENT_CHECKSUMS. I added this to /etc/pulp/settings.py (ALLOWED_CONTENT_CHECKSUMS = [‘md5’, ‘sha1’, ‘sha224’, ‘sha256’, ‘sha384’, ‘sha512’]) and ran:
PULP_SETTINGS=’/etc/pulp/settings.py’ pulpcore-manager handle-artifact-checksums

It seems to be related to this but I’m not sure how best to resolve:
https://pulp.plan.io/issues/8430

Getting the following error:
Error: Package matching query does not exist.

Foreman and Proxy versions:

  • foreman-2.4.1-1.el7.noarch
  • foreman-cli-2.4.1-1.el7.noarch
  • foreman-debug-2.4.1-1.el7.noarch
  • foreman-dynflow-sidekiq-2.4.1-1.el7.noarch
  • foreman-installer-2.4.1-1.el7.noarch
  • foreman-installer-katello-2.4.1-1.el7.noarch
  • foreman-postgresql-2.4.1-1.el7.noarch
  • foreman-proxy-2.4.1-1.el7.noarch
  • foreman-release-2.4.1-1.el7.noarch
  • foreman-service-2.4.1-1.el7.noarch
  • katello-4.0.3-1.el7.noarch
  • katello-certs-tools-2.7.3-1.el7.noarch
  • katello-client-bootstrap-1.7.6-1.el7.noarch
  • katello-common-4.0.3-1.el7.noarch
  • katello-debug-4.0.3-1.el7.noarch
  • katello-default-ca-1.0-1.noarch
  • katello-repos-4.0.3-1.el7.noarch
  • katello-server-ca-1.0-1.noarch
  • pulp-client-1.0-1.noarch
  • python-gofer-qpid-2.12.5-3.el7.noarch
  • python2-qpid-1.37.0-5.el7.noarch
  • python2-qpid-qmf-1.39.0-1.el7.x86_64
  • python3-pulp-2to3-migration-0.11.5-1.el7.noarch
  • python3-pulp-certguard-1.1.0-1.el7.noarch
  • python3-pulp-container-2.2.2-1.el7.noarch
  • python3-pulp-deb-2.9.2-1.el7.noarch
  • python3-pulp-file-1.5.0-1.el7.noarch
  • python3-pulp-rpm-3.11.2-1.el7.noarch
  • python3-pulpcore-3.9.1-3.el7.noarch

Distribution and version:
RHEL 7.9

Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: pulp [4602d68cd8ef4c249f049024f78cd26c]: rq.worker:ERROR: Traceback (most recent call last):
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self.cursor.execute(sql, params)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: DETAIL: Key (“pkgId”)=(436055e0c773cebff9d5dcfdbab7e771fffc64e5) already exists.
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: The above exception was the direct cause of the following exception:
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: Traceback (most recent call last):
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 95, in run
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: d_content.content.save()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py”, line 115, in save
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return super().save(*args, **kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py”, line 129, in save
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: save(*args, **kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 744, in save
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: force_update=force_update, update_fields=update_fields)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 782, in save_base
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: force_update, using, update_fields,
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 873, in _save_table
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 911, in _do_insert
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: using=using, raw=raw)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return getattr(self.get_queryset(), name)(*args, **kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 1186, in _insert
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return query.get_compiler(using=using).execute_sql(return_id)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py”, line 1377, in execute_sql
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: cursor.execute(sql, params)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 67, in execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 76, in _execute_with_wrappers
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return executor(sql, params, many, context)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self.cursor.execute(sql, params)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/utils.py”, line 89, in exit
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: raise dj_exc_value.with_traceback(traceback) from exc_value
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self.cursor.execute(sql, params)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: django.db.utils.IntegrityError: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: DETAIL: Key (“pkgId”)=(436055e0c773cebff9d5dcfdbab7e771fffc64e5) already exists.
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: During handling of the above exception, another exception occurred:
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: Traceback (most recent call last):
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 975, in perform_job
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: rv = job.perform()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 696, in perform
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: self._result = self._execute()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 719, in _execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self.func(*self.args, **self.kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: dv.create()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 148, in create
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: loop.run_until_complete(pipeline)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return future.result()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: await asyncio.gather(*futures)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in call
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: await self.run()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 98, in run
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: d_content.content.q()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return getattr(self.get_queryset(), name)(*args, **kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 408, in get
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: self.model._meta.object_name
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: pulp_rpm.app.models.package.Package.DoesNotExist: Package matching query does not exist.
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: Traceback (most recent call last):
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self.cursor.execute(sql, params)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: DETAIL: Key (“pkgId”)=(436055e0c773cebff9d5dcfdbab7e771fffc64e5) already exists.
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: The above exception was the direct cause of the following exception:
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: Traceback (most recent call last):
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 95, in run
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: d_content.content.save()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py”, line 115, in save
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return super().save(*args, **kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py”, line 129, in save
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: save(*args, **kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 744, in save
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: force_update=force_update, update_fields=update_fields)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 782, in save_base
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: force_update, using, update_fields,
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 873, in _save_table
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 911, in _do_insert
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: using=using, raw=raw)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return getattr(self.get_queryset(), name)(*args, **kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 1186, in _insert
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return query.get_compiler(using=using).execute_sql(return_id)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py”, line 1377, in execute_sql
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: cursor.execute(sql, params)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 67, in execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 76, in _execute_with_wrappers
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return executor(sql, params, many, context)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self.cursor.execute(sql, params)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/utils.py”, line 89, in exit
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: raise dj_exc_value.with_traceback(traceback) from exc_value
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self.cursor.execute(sql, params)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: django.db.utils.IntegrityError: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: DETAIL: Key (“pkgId”)=(436055e0c773cebff9d5dcfdbab7e771fffc64e5) already exists.
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: During handling of the above exception, another exception occurred:
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: Traceback (most recent call last):
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 975, in perform_job
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: rv = job.perform()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 696, in perform
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: self._result = self._execute()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 719, in _execute
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return self.func(*self.args, **self.kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: dv.create()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 148, in create
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: loop.run_until_complete(pipeline)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return future.result()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: await asyncio.gather(*futures)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in call
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: await self.run()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 98, in run
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: d_content.content.q()
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: return getattr(self.get_queryset(), name)(*args, **kwargs)
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 408, in get
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: self.model._meta.object_name
Jun 7 20:57:16 puppetmaster-prod-01 pulpcore-worker-1: pulp_rpm.app.models.package.Package.DoesNotExist: Package matching query does not exist.

2

Are there any workarounds to this? My build process is unfortunately completely broken until I figure out why I cant’ sync this repo.

3

Which repo is failing with these errors? Also, could you run the delete_orphaned_content rake task and check if that helps with the duplicate packages in the system that the log seems to be pointing to…

4

Is there a reason you cannot upgrade past Katello 4.0? These versions are, relatively speaking, somewhat ancient.

I believe this bug may have been fixed in pulpcore 3.12 released last April. Issue #8430: Pulp raises misleading message "Package matching query does not exist" - Pulp

5

I just upgraded to Katello 4.1.4 (I realize this is still outdated) and the problem still persists.

We are on the following now:

  • python3-pulp-rpm-3.14.14-1.el7.noarch
  • python3-pulpcore-3.14.15-1.el7.noarch
  • foreman-2.5.4-1.el7.noarch
  • katello-4.1.4-1.el7.noarch

The problem is at least more descriptive n the error logs

duplicate key value violates unique constraint “rpm_package_pkgId_key”
DETAIL: Key (“pkgId”)=(7f3cfc35444f65d485002f21d060c494efa23660) already exists.
Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Wed, 08 Jun 2022 14:12:09 GMT”, “server”=>“gunicorn”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“GET, PUT, PATCH, DELETE, HEAD, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“62”, “correlation-id”=>“195e707e-a522-4177-b8e1-1c4097b52044”, “access-control-expose-headers”=>“Correlation-ID”, “via”=>“1.1 puppetmaster-prod-01”, “connection”=>“close”}
Response body: {“publication”:[“Invalid hyperlink - Object does not exist.”]}

6

I saw this here where you commented: Issue #8967: "duplicate key value violates unique constraint" when syncing two repositories with identical content in parallel - RPM Support - Pulp

Any idea if pulp packages in Katello 4.2.x fix this or if there is another workaround?

I’ve just went from Katello 3.18.5 → Katello 4.0 → Katello 4.1.4 so I am trying to be a little cautious and test everything before another upgrade to Katello 4.2

7

Well I’m continuing on with the upgrade paths. We are now at Katello 4.2 and the problem still persists.

Looks like same version of pulpcore from Katello 4.1

  • python3-pulp-rpm-3.14.14-1.el7.noarch
  • python3-pulpcore-3.14.15-1.el7.noarch
  • foreman-3.0.2-1.el7.noarch
  • katello-4.2.2-1.el7.noarch
8

Here we go. I’m now at the latest Katello 4.2. Problem still presists with Puppet6 RHEL8 repo (and MySQL repos)

duplicate key value violates unique constraint “rpm_package_pkgId_key”
DETAIL: Key (“pkgId”)=(436055e0c773cebff9d5dcfdbab7e771fffc64e5) already exists.

9

@macado Could you provide a list of the repos you were syncing, and we can try to reproduce it locally

10

Sure the main 3 I am having issues with are the followig.

Puppet6-RHEL8
https://yum.puppetlabs.com/puppet6/el/8/x86_64/index.html

MySQL 5.7 Community-RHEL7
http://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/

MySQL 5.6 Community-RHEL7
http://repo.mysql.com/yum/mysql-5.6-community/el/7/x86_64/

I believe the original issue that I encountered here was because some of these packages may have had SHA1 checksums. I added ALLOWED_CONTENT_CHECKSUMS = [“sha1”, “sha224”, “sha256”, “sha384”, “sha512”] to /etc/pulp/settings.py
and then ran PULP_SETTINGS=’/etc/pulp/settings.py’ pulpcore-manager handle-artifact-checksums

I am happy to provide any more troubleshooting or information. These particular repositories had errors throughout the entire upgrade from Katello 3.18.5 to Katello 4.3.1. Everything else in this environment seems to be working.

Installed Packages

  • foreman-3.1.3-1.el7.noarch
  • foreman-cli-3.1.3-1.el7.noarch
  • katello-4.3.1-1.el7.noarch
  • pulp-puppet-plugins-2.21.5-1.el7.noarch
  • pulp-puppet-tools-2.21.5-1.el7.noarch
  • pulp-rpm-plugins-2.21.5-1.el7.noarch
  • pulp-selinux-2.21.5-1.el7.noarch
  • pulp-server-2.21.5-1.el7.noarch
  • python-pulp-common-2.21.5-1.el7.noarch
  • tfm-pulpcore-python3-pulp-certguard-1.5.1-1.el7.noarch
  • tfm-pulpcore-python3-pulp-container-2.9.2-1.el7.noarch
  • tfm-pulpcore-python3-pulp-deb-2.16.2-1.el7.noarch
  • tfm-pulpcore-python3-pulp-file-1.10.1-1.el7.noarch
  • tfm-pulpcore-python3-pulp-python-3.5.2-1.el7.noarch
  • tfm-pulpcore-python3-pulp-rpm-3.17.5-2.el7.noarch
  • tfm-pulpcore-python3-pulpcore-3.16.8-2.el7.noarch

Example:
Sync task failure:
duplicate key value violates unique constraint “rpm_package_pkgId_key”
DETAIL: Key (“pkgId”)=(4f8a089cf788210c4f0659f69d513b44886d0e6f) already exists.

Pulpcore database
pulpcore=# select name from rpm_package where “pkgId”=‘4f8a089cf788210c4f0659f69d513b44886d0e6f’;
name

mysql-community-client
(1 row)

11

Sorry for the late response.

I haven’t been able to reproduce this so far - just verifying that it remains a problem for you?

12

Also, per chance are you syncing only one repo at a time or many at once?