Problem:
Greetings All,
After updating to Foreman 1.24.3 / Katello 3.14 we have an issue syncing some repos.
Logs indicate the following:
Nov 16 09:43:30 foreman daemon.info pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) Unhandled Exception in Worker Thread [140155432154880]
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) Traceback (most recent call last):
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) File “/usr/lib/python2.7/site-packages/nectar/downloaders/threaded.py”, line 187, in worker
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) self._fetch(request, session)
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) File “/usr/lib/python2.7/site-packages/nectar/downloaders/threaded.py”, line 348, in _fetch
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) if nretry < DEFAULT_GENERIC_TRIES - 1 and e.args[1] == errno.ECONNRESET:
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) IndexError: tuple index out of range
Nov 16 09:43:30 foreman daemon.info pulp: nectar.downloaders.threaded:INFO: Download of https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/extras/os/Packages/d/docker-1.13.1-203.git0be3e21.el7_9.x86_64.rpm was cancelled
Nov 16 09:43:30 foreman daemon.info pulp: nectar.downloaders.threaded:INFO: Download of https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/extras/os/Packages/c/containers-common-0.1.40-12.el7_9.x86_64.rpm was cancelled
Nov 16 09:43:30 foreman daemon.info pulp: nectar.downloaders.threaded:INFO: Download of https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/extras/os/Packages/d/docker-common-1.13.1-203.git0be3e21.el7_9.x86_64.rpm was cancelled
Seems to fail on random packages every time, however it does only seem to occur on SOME repos. The exception it’s hitting seems to indicate timeout/connection retries but I’m not seeing any evidence of that in the logs.
I haven’t been able to find any other evidence on the Forums of anyone else having this issue. Happy to provide any further information required.
Any help would be most appreciated.
Expected outcome:
Repos should sync.
Foreman and Proxy versions:
- foreman-1.24.3-1.el7.noarch
- foreman-proxy-1.24.3-1.el7.noarch
- katello-3.14.1-1.el7.noarch
Distribution and version:
CentOS Linux release 7.8.2003 (Core)
Other relevant data: