There are some additional prerequisites should meets before content import
pulp:pulp permissions so that Pulp can read and write the .json files.Sorry, I had to reproduce the problem first.
Me too! Since those 500GB was my total available storage and we need double the space for the import, so in the end I needed to go with a different repo set, one with 123 GB and 65542 packages in total.
With this I was able to reproduced the issue, here is the traceback:
May 26 10:16:16 SAT pulpcore-worker-5[213434]: pulp [75a0642a-6ddb-41b7-97e5-c0e410e0044a]: pulpcore.tasking.tasks:INFO: Task[pulp_rpm.app.tasks.copy.copy_content] 019e6467-ed55-7997-9a4f-34b930138a19 failed (OperationalError: sending query and params failed: number of parameters must be between 0 and 65535) in domain: default
May 26 10:16:16 SAT pulpcore-worker-5[213434]: pulp [75a0642a-6ddb-41b7-97e5-c0e410e0044a]: pulpcore.tasking.tasks:INFO: File “/usr/lib/python3.12/site-packages/pulpcore/tasking/tasks.py”, line 109, in _execute_task
May 26 10:16:16 SAT pulpcore-worker-5[213434]: func(*args, **kwargs)
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib64/python3.12/contextlib.py”, line 81, in inner
May 26 10:16:16 SAT pulpcore-worker-5[213434]: return func(*args, **kwds)
May 26 10:16:16 SAT pulpcore-worker-5[213434]: ^^^^^^^^^^^^^^^^^^^
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/pulp_rpm/app/tasks/copy.py”, line 225, in copy_content
May 26 10:16:16 SAT pulpcore-worker-5[213434]: children = find_children_of_content(content, source_repo_version)
May 26 10:16:16 SAT pulpcore-worker-5[213434]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/pulp_rpm/app/tasks/copy.py”, line 100, in find_children_of_content
May 26 10:16:16 SAT pulpcore-worker-5[213434]: for packagegroup in packagegroups.iterator():
May 26 10:16:16 SAT pulpcore-worker-5[213434]: ^^^^^^^^^^^^^^^^^^^^^^^^
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/django/db/models/query.py”, line 518, in _iterator
May 26 10:16:16 SAT pulpcore-worker-5[213434]: yield from iterable
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/django/db/models/query.py”, line 93, in iter
May 26 10:16:16 SAT pulpcore-worker-5[213434]: results = compiler.execute_sql(
May 26 10:16:16 SAT pulpcore-worker-5[213434]: ^^^^^^^^^^^^^^^^^^^^^
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/django/db/models/sql/compiler.py”, line 1562, in execute_sql
May 26 10:16:16 SAT pulpcore-worker-5[213434]: cursor.execute(sql, params)
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/django/db/backends/utils.py”, line 67, in execute
May 26 10:16:16 SAT pulpcore-worker-5[213434]: return self._execute_with_wrappers(
May 26 10:16:16 SAT pulpcore-worker-5[213434]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/django/db/backends/utils.py”, line 80, in _execute_with_wrappers
May 26 10:16:16 SAT pulpcore-worker-5[213434]: return executor(sql, params, many, context)
May 26 10:16:16 SAT pulpcore-worker-5[213434]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/django/db/backends/utils.py”, line 84, in _execute
May 26 10:16:16 SAT pulpcore-worker-5[213434]: with self.db.wrap_database_errors:
May 26 10:16:16 SAT pulpcore-worker-5[213434]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/django/db/utils.py”, line 91, in exit
May 26 10:16:16 SAT pulpcore-worker-5[213434]: raise dj_exc_value.with_traceback(traceback) from exc_value
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/django/db/backends/utils.py”, line 89, in _execute
May 26 10:16:16 SAT pulpcore-worker-5[213434]: return self.cursor.execute(sql, params)
May 26 10:16:16 SAT pulpcore-worker-5[213434]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 26 10:16:16 SAT pulpcore-worker-5[213434]: File “/usr/lib/python3.12/site-packages/psycopg/_server_cursor.py”, line 98, in execute
May 26 10:16:16 SAT pulpcore-worker-5[213434]: raise ex.with_traceback(None)
May 26 10:16:22 SAT pulpcore-api[194464]: pulp [75a0642a-6ddb-41b7-97e5-c0e410e0044a]: - - [26/May/2026:14:16:22 +0000] “GET /pulp/api/v3/tasks/019e6467-ed55-7997-9a4f-34b930138a19/ HTTP/1.1” 200 3191 “-” “OpenAPI-Generator/3.85.11/ruby”
I think we have all we need to file a bug!
I’m not sure, but I think it’s a combination of packages and errata. Red Hat Content View doesn’t have that many packages, but it does have a lot of errata. Ubuntu doesn’t have any errata, but it has a lot of packages.
Great! But is this a pulpcore, pulp_rpm or/and pulp_deb issue?
IMHO in my case it’s just matter of the packages count - I went only 7 packages over the 65535 threshold (2^16), while errata was at comfortable 7793 (not anywhere near 2^N). Although, similar issue may arise with errata count over 2^16.
Great! But is this a pulpcore, pulp_rpm or/and pulp_deb issue?
It can be both and/or even some missed codepath in pulpcore. Or all three. I’m gonna check it with pulp folks later today.
Based on my experience and the symptoms, I expect the fix will be in pulpcore, and that will fix it for all plugins. When we had the same error on the export (as opposed to the import), it was fixed in pulpcore.
Filed a Jira issue: Jira
Will check what upstream issue(s) should this become (pulpcore, pulp_rpm, pulp_deb).
I’m having the same problem here when trying to import a content view containing bionic, bionic-security and bionic-updates for Ubuntu 18, as well as another one containing noble, noble-security, noble-updates and noble-backports for Ubuntu 24.
Trying to split the distributions into smaller content views doesn’t seem to work either: a content view containing only bionic-security does indeed trigger the same error (“sending query and params failed: number of parameters must be between 0 and 65535”).
Is there any workaround?
I believe that until this is fixed in pulpcore, there is no workaround other than building CVs with less than the magic number of 65535 content units in it. Since pulp_deb (Ubuntu) repos contain a PRC for each package, the limit may be roughly half of 65535 in packages.
I still find it hard to believe that bionic-security alone has so many packages. Can you check your CV how many packages are in it?
Is syncing only individual architectures e.g.: amd64 an option?
@vsedmik Is there a pulpcore issue to go with the Satellite Jira issue you posted?
I didn’t expect “bionic-security” to fail either, since it only has 28,851 packages, but I’m getting an error when importing it. Ubuntu Noble would, in my opinion, be a better example for reproducing the issue.
FYI, I’m already only syncing the amd64 architecture.
In fact, it also seems strange that the base “bionic” repository, that contains 60,867 packages, doesn’t trigger this error…
That does sound strange. Not sure what to make of it.
There is yet this Pulp Jira issue
Looks like I don’t have permissions to see that one.
A few weeks ago, I still had access to the ticket, but today I don’t.
Foreman 3.14 – Katello 4.16 works perfectly and without issues for me.
Are you no longer getting any “65,535 items” errors with these versions?
Have you tried other versions? (3.16, for example)
No errors with this version (foreman-3.14.0, pulpcore-3.63.11, katello-4.16.3).
I haven’t tried other versions, maybe this or next week.
Okay, I’m interested in hearing about the results you get here!
By any chance, have you noticed any major changes resulting from the downgrade to Foreman version 3.14?
We hit this problem today too, a fix rather sooner than later would be nice.