Katello 3.18.4 / Pulp 2.21.5 can't sync third party repos with SHA checksums

Problem:
I’m running Katello 3.18.4 / Pulp 2.21.5 and I can’t sync certain third party repos that are using SHA checksums

Expected outcome:
Looking for a workaround to be able to sync certain third party repos.

Distribution and version:
RHEL7

Right now I can’t sync Puppet6, MySQL Community or SaltStack repos because they all seem to have RPMS that have SHA checksums

Package matching query does not exist.Package puppet-bolt-0:2.16.0-1.el6.x86_64 as content unit 0688f757-1031-4c69-bdd5-de325284e477 contains forbidden checksum type ‘sha’, thus can’t be published. Checksum must be one of the allowed checksum types.
You can adjust these with the ‘ALLOWED_CONTENT_CHECKSUMS’ setting.

I tried adding SHA1 to /etc/pulp/settings.py and restart Foreman services but this does not help.
ALLOWED_CONTENT_CHECKSUMS = [‘md5’, ‘sha1’, ‘sha224’, ‘sha256’, ‘sha384’, ‘sha512’]

Adding “SHA” to array above breaks Pulp completely.

Is there any workaround here? I will eventually upgrade to Katello 4.0 but we’re in the middle of patching and do not want to break foreman/katello right before our patch cycle.

The problem you are reporting here must be in pulp3 (also referred to as pulpcore).
you need to run pulpcore-manager handle-artifact-checksum after changing ALLOWED_CONTENT_CHECKSUMS.

Thanks. I will give this a try

How do I run this properly on katello / foreman? I know CONTENT_ORIGIN is set in /etc/pulp/setting.py

[root@puppetmaster-prod-01 ~]# pulpcore-manager handle-artifact-checksums --report
Traceback (most recent call last):
File “/usr/lib/python3.6/site-packages/pulpcore/app/settings.py”, line 254, in
CONTENT_ORIGIN
NameError: name ‘CONTENT_ORIGIN’ is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/bin/pulpcore-manager”, line 11, in
load_entry_point(‘pulpcore==3.7.6’, ‘console_scripts’, ‘pulpcore-manager’)()
File “/usr/lib/python3.6/site-packages/pulpcore/app/manage.py”, line 11, in manage
execute_from_command_line(sys.argv)
File “/usr/lib/python3.6/site-packages/django/core/management/init.py”, line 381, in execute_from_command_line
utility.execute()
File “/usr/lib/python3.6/site-packages/django/core/management/init.py”, line 357, in execute
django.setup()
File “/usr/lib/python3.6/site-packages/django/init.py”, line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File “/usr/lib/python3.6/site-packages/django/apps/registry.py”, line 122, in populate
app_config.ready()
File “/usr/lib/python3.6/site-packages/pulpcore/app/apps.py”, line 81, in ready
self.import_viewsets()
File “/usr/lib/python3.6/site-packages/pulpcore/app/apps.py”, line 110, in import_viewsets
from pulpcore.app.viewsets import NamedModelViewSet
File “/usr/lib/python3.6/site-packages/pulpcore/app/viewsets/init.py”, line 1, in
from .base import ( # noqa
File “/usr/lib/python3.6/site-packages/pulpcore/app/viewsets/base.py”, line 17, in
from pulpcore.app import tasks
File “/usr/lib/python3.6/site-packages/pulpcore/app/tasks/init.py”, line 1, in
from pulpcore.app.tasks import base, repository, upload # noqa
File “/usr/lib/python3.6/site-packages/pulpcore/app/tasks/upload.py”, line 6, in
from pulpcore.app.serializers import ArtifactSerializer
File “/usr/lib/python3.6/site-packages/pulpcore/app/serializers/init.py”, line 42, in
from .exporter import ( # noqa
File “/usr/lib/python3.6/site-packages/pulpcore/app/serializers/exporter.py”, line 8, in
from pulpcore.app import models, settings
File “/usr/lib/python3.6/site-packages/pulpcore/app/settings.py”, line 258, in
"CONTENT_ORIGIN is a required setting but it was not configured. This may be caused "
django.core.exceptions.ImproperlyConfigured: CONTENT_ORIGIN is a required setting but it was not configured. This may be caused by invalid read permissions of the settings file. Note that CONTENT_ORIGIN is set by the installer automatically.

[root@puppetmaster-prod-01 pulp]# pulpcore-manager handle-artifact-checksums –report
Unknown command: ‘handle-artifact-checksums’
Type ‘pulpcore-manager help’ for usage.
[root@puppetmaster-prod-01 pulp]# pulpcore-manager help

Type 'pulpcore-manager help ’ for help on a specific subcommand.

Available subcommands:

[app]
reset-admin-password
stage-profile-summary

[auth]
changepassword
createsuperuser

[contenttypes]
remove_stale_contenttypes

[django]
check
compilemessages
createcachetable
dbshell
diffsettings
dumpdata
flush
inspectdb
loaddata
makemessages
makemigrations
migrate
sendtestemail
shell
showmigrations
sqlflush
sqlmigrate
sqlsequencereset
squashmigrations
startapp
startproject
test
testserver

[drf_spectacular]
spectacular

[guardian]
clean_orphan_obj_perms

[rest_framework]
generateschema

[sessions]
clearsessions

[staticfiles]
collectstatic
findstatic
runserver
[root@puppetmaster-prod-01 pulp]# pulpcore-manager handle-artifact-checksums –report
Unknown command: ‘handle-artifact-checksums’
Type ‘pulpcore-manager help’ for usage.
[root@puppetmaster-prod-01 pulp]# foreman-installer --foreman-proxy-content-pulpcore-allowed-content-checksums md5 \

--foreman-proxy-content-pulpcore-allowed-content-checksums sha1 \
--foreman-proxy-content-pulpcore-allowed-content-checksums sha224 \
--foreman-proxy-content-pulpcore-allowed-content-checksums sha256 \
--foreman-proxy-content-pulpcore-allowed-content-checksums sha384 \
--foreman-proxy-content-pulpcore-allowed-content-checksums sha512

2021-12-06 12:06:16 [NOTICE] [pre_migrations] Executing hooks in group pre_migrations
2021-12-06 12:06:16 [NOTICE] [pre_migrations] All hooks in group pre_migrations finished
2021-12-06 12:06:16 [NOTICE] [boot] Executing hooks in group boot
2021-12-06 12:06:16 [NOTICE] [boot] All hooks in group boot finished
2021-12-06 12:06:16 [NOTICE] [init] Executing hooks in group init
2021-12-06 12:06:16 [NOTICE] [init] All hooks in group init finished
2021-12-06 12:06:16 [NOTICE] [root] Loading default values from puppet modules…
2021-12-06 12:06:28 [NOTICE] [root] … finished
2021-12-06 12:06:28 [NOTICE] [pre_values] Executing hooks in group pre_values
2021-12-06 12:06:28 [NOTICE] [pre_values] All hooks in group pre_values finished
ERROR: Unrecognised option ‘–foreman-proxy-content-pulpcore-allowed-content-checksums’

See: ‘foreman-installer --help’

Looks like that version of pulp didn’t have this command.
@iballou do you have a better idea?

Hi @macado,

I just tried syncing Puppet 6 on Katello 3.18.5 and it worked fine for me. Try updating to pulpcore 3.7.8 and pulp-rpm 3.11.2 and sync again. The traceback suggests you’re on 3.7.6, but you can double check with pip3 list | grep pulp.

If that’s not it, do you have FIPS enabled on your server? That could potentially be related.

Thanks! I didnt realize katello 3.18.5 was out so I will try upgrading pulpcore and pulp-rpm. I do not have FIPS enabled.

[root@puppetmaster-prod-01 ~]# pip3 list | grep pulp
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
pulp-2to3-migration (0.11.1)
pulp-certguard (1.0.3)
pulp-container (2.1.2)
pulp-deb (2.9.2)
pulp-file (1.3.0)
pulp-rpm (3.10.0)
pulpcore (3.7.6)

I successfully upgraded to Katello 3.18.5 which has:
python3-pulpcore-3.7.9-1.el7.noarch
python3-pulp-rpm-3.11.2-1.el7.noarch

[root@puppetmaster-prod-01 ~]# pip3 list | grep pulp
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
pulp-2to3-migration (0.11.5)
pulp-certguard (1.0.3)
pulp-container (2.1.2)
pulp-deb (2.9.2)
pulp-file (1.3.0)
pulp-rpm (3.11.2)
pulpcore (3.7.9)

Unfortunately I still cannot sync Puppet6 and other repos with SHA1 checksums and those commands do not work for me.

[root@puppetmaster-prod-01 ~]# PULP_SETTINGS=’/etc/pulp/settings.py’ pulpcore-manager handle-artifact-checksums –report
Unknown command: ‘handle-artifact-checksums’
Type ‘pulpcore-manager help’ for usage.

@macado can you give us an example upstream URL for a repository that you are syncing?

You could also try setting “Yum Metadata Checksum” to “sha1” and sync again.

Sure. I will give that a try. Puppet6 (RHEL8) is one particular one giving me issues. Anther example is MySQL 5.7 Community Server-RHEL7

https://yum.puppetlabs.com/puppet6/el/8/x86_64/
http://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/

When I try running a “Complete Sync” I get the following error
Package matching query does not exist.Package pdk-0:1.18.0.0-1.el8.x86_64 as content unit 093820dd-7874-4056-8292-621bc592ef5f contains forbidden checksum type ‘sha’, thus can’t be published. Checksum must be one of the allowed checksum types.
You can adjust these with the ‘ALLOWED_CONTENT_CHECKSUMS’ setting.

Package matching query does not exist.Package mysql-ref-manual-5.7-en-pdf-0:1-20210326.noarch as content unit 0059623b-df7d-4220-b150-b805e1741a57 contains forbidden checksum type ‘sha’, thus can’t be published. Checksum must be one of the allowed checksum types.
You can adjust these with the ‘ALLOWED_CONTENT_CHECKSUMS’ setting.

What is interesting is if I run “Validate Content Sync” it completes with no warnings

Interesting. I am seeing this in my /var/log/foreman/production.log

Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: pulp: rq.worker:ERROR: Traceback (most recent call last):
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self.cursor.execute(sql, params)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: DETAIL: Key (“pkgId”)=(2f4224dee658e772cd0b7fbf81a6d2a62871440b) already exists.
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: The above exception was the direct cause of the following exception:
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: Traceback (most recent call last):
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 102, in run
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: d_content.content.save()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py”, line 115, in save
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return super().save(*args, **kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py”, line 129, in save
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: save(*args, **kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 744, in save
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: force_update=force_update, update_fields=update_fields)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 782, in save_base
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: force_update, using, update_fields,
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 873, in _save_table
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 911, in _do_insert
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: using=using, raw=raw)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 1186, in _insert
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return query.get_compiler(using=using).execute_sql(return_id)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py”, line 1377, in execute_sql
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: cursor.execute(sql, params)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 67, in execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 76, in _execute_with_wrappers
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return executor(sql, params, many, context)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self.cursor.execute(sql, params)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/utils.py”, line 89, in exit
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: raise dj_exc_value.with_traceback(traceback) from exc_value
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self.cursor.execute(sql, params)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: django.db.utils.IntegrityError: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: DETAIL: Key (“pkgId”)=(2f4224dee658e772cd0b7fbf81a6d2a62871440b) already exists.
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: During handling of the above exception, another exception occurred:
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: Traceback (most recent call last):
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 936, in perform_job
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: rv = job.perform()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 684, in perform
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: self._result = self._execute()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 690, in _execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self.func(*self.args, **self.kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: dv.create()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 148, in create
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: loop.run_until_complete(pipeline)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return future.result()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: await asyncio.gather(*futures)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in call
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: await self.run()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 105, in run
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: d_content.content.q()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 408, in get
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: self.model._meta.object_name
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: pulp_rpm.app.models.package.Package.DoesNotExist: Package matching query does not exist.
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: Traceback (most recent call last):
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self.cursor.execute(sql, params)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: DETAIL: Key (“pkgId”)=(2f4224dee658e772cd0b7fbf81a6d2a62871440b) already exists.
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: The above exception was the direct cause of the following exception:
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: Traceback (most recent call last):
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 102, in run
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: d_content.content.save()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py”, line 115, in save
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return super().save(*args, **kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py”, line 129, in save
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: save(*args, **kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 744, in save
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: force_update=force_update, update_fields=update_fields)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 782, in save_base
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: force_update, using, update_fields,
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 873, in _save_table
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 911, in _do_insert
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: using=using, raw=raw)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 1186, in _insert
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return query.get_compiler(using=using).execute_sql(return_id)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py”, line 1377, in execute_sql
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: cursor.execute(sql, params)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 67, in execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 76, in _execute_with_wrappers
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return executor(sql, params, many, context)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self.cursor.execute(sql, params)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/utils.py”, line 89, in exit
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: raise dj_exc_value.with_traceback(traceback) from exc_value
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self.cursor.execute(sql, params)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: django.db.utils.IntegrityError: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: DETAIL: Key (“pkgId”)=(2f4224dee658e772cd0b7fbf81a6d2a62871440b) already exists.
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: During handling of the above exception, another exception occurred:
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: Traceback (most recent call last):
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 936, in perform_job
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: rv = job.perform()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 684, in perform
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: self._result = self._execute()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 690, in _execute
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return self.func(*self.args, **self.kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: dv.create()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 148, in create
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: loop.run_until_complete(pipeline)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return future.result()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: await asyncio.gather(*futures)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in call
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: await self.run()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 105, in run
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: d_content.content.q()
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 408, in get
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: self.model._meta.object_name
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: pulp_rpm.app.models.package.Package.DoesNotExist: Package matching query does not exist.
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: pulp: rq.worker:INFO: 26133@puppetmaster-prod-01.uit.tufts.edu: f06c511a-d530-4d7e-8c22-c63a5abff851
Dec 7 15:11:49 puppetmaster-prod-01 pulpcore-worker-3: pulp: rq.worker:INFO: 26133@puppetmaster-prod-01.uit.tufts.edu: Job OK (f06c511a-d530-4d7e-8c22-c63a5abff851)

This is after I try syncing Puppet6-RHEL7 repo.

Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-content: [07/Dec/2021:20:15:20 +0000] “GET /pulp/content/Tufts_University/Library/RHEL7-ESAI-Default/custom/Puppet6/Puppet6-RHEL7/repodata/repomd.xml HTTP/1.1” 200 3763 “-” “urlgrabber/3.10 yum/3.4.3”
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: pulp: rq.worker:ERROR: Traceback (most recent call last):
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self.cursor.execute(sql, params)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: DETAIL: Key (“pkgId”)=(2f4224dee658e772cd0b7fbf81a6d2a62871440b) already exists.
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: The above exception was the direct cause of the following exception:
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: Traceback (most recent call last):
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 102, in run
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: d_content.content.save()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py”, line 115, in save
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return super().save(*args, **kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py”, line 129, in save
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: save(*args, **kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 744, in save
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: force_update=force_update, update_fields=update_fields)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 782, in save_base
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: force_update, using, update_fields,
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 873, in _save_table
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 911, in _do_insert
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: using=using, raw=raw)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 1186, in _insert
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return query.get_compiler(using=using).execute_sql(return_id)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py”, line 1377, in execute_sql
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: cursor.execute(sql, params)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 67, in execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-content: [07/Dec/2021:20:15:20 +0000] “GET /pulp/content/Tufts_University/Library/RHEL6-ESAI-Default/custom/Extra_Packages_for_Enterprise_Linux/EPEL-RHEL6/repodata/repomd.xml HTTP/1.1” 200 4089 “-” “urlgrabber/3.9.1 yum/3.2.29”
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 76, in _execute_with_wrappers
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return executor(sql, params, many, context)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self.cursor.execute(sql, params)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/utils.py”, line 89, in exit
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: raise dj_exc_value.with_traceback(traceback) from exc_value
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self.cursor.execute(sql, params)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: django.db.utils.IntegrityError: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: DETAIL: Key (“pkgId”)=(2f4224dee658e772cd0b7fbf81a6d2a62871440b) already exists.
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: During handling of the above exception, another exception occurred:
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: Traceback (most recent call last):
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 936, in perform_job
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: rv = job.perform()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 684, in perform
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: self._result = self._execute()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 690, in _execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self.func(*self.args, **self.kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: dv.create()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 148, in create
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: loop.run_until_complete(pipeline)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return future.result()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: await asyncio.gather(*futures)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in call
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: await self.run()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 105, in run
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: d_content.content.q()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 408, in get
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: self.model._meta.object_name
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: pulp_rpm.app.models.package.Package.DoesNotExist: Package matching query does not exist.
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: Traceback (most recent call last):
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self.cursor.execute(sql, params)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: DETAIL: Key (“pkgId”)=(2f4224dee658e772cd0b7fbf81a6d2a62871440b) already exists.
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: The above exception was the direct cause of the following exception:
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: Traceback (most recent call last):
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 102, in run
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: d_content.content.save()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py”, line 115, in save
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return super().save(*args, **kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py”, line 129, in save
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: save(*args, **kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 744, in save
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: force_update=force_update, update_fields=update_fields)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 782, in save_base
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: force_update, using, update_fields,
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 873, in _save_table
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 911, in _do_insert
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: using=using, raw=raw)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 1186, in _insert
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return query.get_compiler(using=using).execute_sql(return_id)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py”, line 1377, in execute_sql
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: cursor.execute(sql, params)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 67, in execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 76, in _execute_with_wrappers
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return executor(sql, params, many, context)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self.cursor.execute(sql, params)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/utils.py”, line 89, in exit
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: raise dj_exc_value.with_traceback(traceback) from exc_value
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self.cursor.execute(sql, params)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: django.db.utils.IntegrityError: duplicate key value violates unique constraint “rpm_package_pkgId_key”
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: DETAIL: Key (“pkgId”)=(2f4224dee658e772cd0b7fbf81a6d2a62871440b) already exists.
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: During handling of the above exception, another exception occurred:
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: Traceback (most recent call last):
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 936, in perform_job
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: rv = job.perform()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 684, in perform
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: self._result = self._execute()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 690, in _execute
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return self.func(*self.args, **self.kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: dv.create()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 148, in create
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: loop.run_until_complete(pipeline)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return future.result()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: await asyncio.gather(*futures)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in call
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: await self.run()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/content_stages.py”, line 105, in run
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: d_content.content.q()
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: return getattr(self.get_queryset(), name)(*args, **kwargs)
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 408, in get
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: self.model._meta.object_name
Dec 7 15:15:20 puppetmaster-prod-01 pulpcore-worker-2: pulp_rpm.app.models.package.Package.DoesNotExist: Package matching query does not exist.

@Justin_Sherrill said he thinks I am hitting this bug
https://www.mail-archive.com/pulp-list@redhat.com/msg06171.html
https://pulp.plan.io/issues/8411

Unfortunately I don’t see a workaround. I am eventually planning on upgrading to Katello 4.0 but we’re in the process of patching all our RHEL systems and I really dont want to completely break things at the moment.

I’m checking if I can get some ideas from people more familiar with Pulp’s inner workings.

‘sha’ as an alias for ‘sha1’ was fixed in Issue #8052: Some repositories cannot be synchronized with error: Artifact() got an unexpected keyword argument 'sha' - RPM Support - Pulp . A refactoring shortly thereafter left A Hole,m which I literally just fixed under Issue #9580: Artifact() got an unexpected keyword argument 'sha' - RPM Support - Pulp .

I don’t believe 8052 was backported to 3.7