Problem: Pulp image lacking libraries
Expected outcome: Successful upgrade
Foreman and Proxy versions: 5.0
Foreman and Proxy plugin versions: 5.0
Distribution and version: CentOS Stream 10 (foremanctl/pulp images)
Other relevant data:
quay.io/foreman/pulp:foreman-5.0 boots Gunicorn, then every worker dies because psycopg 3 cannot load libpq and Django’s fallback to psycopg2 is not installed either. The same failure hits pulp-api, pulp-content, and the pulp-worker units.
Image
| Tag | quay.io/foreman/pulp:foreman-5.0 |
| Digest | sha256:a371cf2ad1645eaca2b331fbbc46c0763e4375b43c08d2f969f83c2647edc163 |
| Image id | 6d3c2c0d62473ea602e40dbf448a8b2edf03d6793b357b19016de55ba0b0af6b |
| Created | 2026-09-30T12:41:11Z |
| Source | GitHub - theforeman/pulp-oci-images · GitHub |
| Revision | e2f905e7ad07f0f9b55ecd194be80cf8806b5a87 |
| Base | CentOS Stream 10 (org.label-schema.build-date=20260928) |
| User | pulp:pulp |
| Host | CentOS Stream 10, rootful Podman, Network=host |
| Installed database packages in that image: |
python3.12-pulpcore-3.118.1-2.el10python3.12-django-5.2.17-1.el10python3.12-gunicorn-26.2.0-1.el10python3.12-psycopg-3.3.6-1.el10(noarch only)
Not installed:libpq,python3.12-psycopg2,psycopg_c,psycopg_binary.find /usr -name 'libpq*'is empty.
A freshpython3in the image reports:
psycopg present at /usr/lib/python3.12/site-packages/psycopg
psycopg_c None
psycopg_binary None
psycopg2 None
Installing libpq-16.11-3.el10 into that image makes import psycopg succeed (psycopg 3.3.6, libpq 160011). No other package change was required.
Log
From pulp-api on 2026-10-01 18:16:20 UTC, after Gunicorn 26.2.0 listened on unix:/run/httpd.pulp-api.sock and spawned workers 2–6:
[2026-10-01 18:16:20 +0000] [2] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/lib/python3.12/site-packages/django/db/backends/postgresql/psycopg_any.py", line 5, in <module>
from psycopg import ClientCursor, IsolationLevel, adapt, adapters, errors, sql
File "/usr/lib/python3.12/site-packages/psycopg/__init__.py", line 9, in <module>
from . import pq # noqa: F401 import early to stabilize side effects
File "/usr/lib/python3.12/site-packages/psycopg/pq/__init__.py", line 114, in <module>
import_from_libpq()
File "/usr/lib/python3.12/site-packages/psycopg/pq/__init__.py", line 108, in import_from_libpq
raise ImportError(f"""\
ImportError: no pq wrapper available.
Attempts made:
- couldn't import psycopg 'c' implementation: No module named 'psycopg_c'
- couldn't import psycopg 'binary' implementation: No module named 'psycopg_binary'
- couldn't import psycopg 'python' implementation: libpq library not found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.12/site-packages/gunicorn/arbiter.py", line 753, in spawn_worker
worker.init_process()
File "/usr/lib/python3.12/site-packages/pulpcore/app/entrypoint.py", line 69, in init_process
django.setup()
File "/usr/lib/python3.12/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/lib/python3.12/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/lib/python3.12/site-packages/django/apps/config.py", line 123, in create
mod = import_module(mod_path)
File "/usr/lib/python3.12/site-packages/django/contrib/postgres/apps.py", line 4, in <module>
from django.db.backends.postgresql.psycopg_any import RANGE_TYPES
File "/usr/lib/python3.12/site-packages/django/db/backends/postgresql/psycopg_any.py", line 77, in <module>
from psycopg2 import errors, extensions, sql # NOQA
ModuleNotFoundError: No module named 'psycopg2'
Gunicorn then exits with gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>. Systemd restarts the unit; the counter was at 587 before the image was replaced.