Issues syncing (some) repos

Support
1

Problem:

Greetings All,

After updating to Foreman 1.24.3 / Katello 3.14 we have an issue syncing some repos.

Logs indicate the following:

Nov 16 09:43:30 foreman daemon.info pulp: urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) Unhandled Exception in Worker Thread [140155432154880]
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) Traceback (most recent call last):
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) File “/usr/lib/python2.7/site-packages/nectar/downloaders/threaded.py”, line 187, in worker
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) self._fetch(request, session)
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) File “/usr/lib/python2.7/site-packages/nectar/downloaders/threaded.py”, line 348, in _fetch
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) if nretry < DEFAULT_GENERIC_TRIES - 1 and e.args[1] == errno.ECONNRESET:
Nov 16 09:43:30 foreman daemon.err pulp: nectar.downloaders.threaded:ERROR: (4550-54880) IndexError: tuple index out of range
Nov 16 09:43:30 foreman daemon.info pulp: nectar.downloaders.threaded:INFO: Download of https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/extras/os/Packages/d/docker-1.13.1-203.git0be3e21.el7_9.x86_64.rpm was cancelled
Nov 16 09:43:30 foreman daemon.info pulp: nectar.downloaders.threaded:INFO: Download of https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/extras/os/Packages/c/containers-common-0.1.40-12.el7_9.x86_64.rpm was cancelled
Nov 16 09:43:30 foreman daemon.info pulp: nectar.downloaders.threaded:INFO: Download of https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/extras/os/Packages/d/docker-common-1.13.1-203.git0be3e21.el7_9.x86_64.rpm was cancelled

Seems to fail on random packages every time, however it does only seem to occur on SOME repos. The exception it’s hitting seems to indicate timeout/connection retries but I’m not seeing any evidence of that in the logs.

I haven’t been able to find any other evidence on the Forums of anyone else having this issue. Happy to provide any further information required.

Any help would be most appreciated.

Expected outcome:

Repos should sync.

Foreman and Proxy versions:

  • foreman-1.24.3-1.el7.noarch
  • foreman-proxy-1.24.3-1.el7.noarch
  • katello-3.14.1-1.el7.noarch

Distribution and version:

CentOS Linux release 7.8.2003 (Core)

Other relevant data:

2

Hi @tmuz,

Sorry to hear about the trouble! Are you using a http proxy?

3

Hi @John_Mitsch

Thanks for your reply. Yes we are behind a proxy.

After some further investigation yesterday I’ve noticed that it seems to be only the Red hat repos that are failing, it fails at different points through each run by does manage to grab a few packages each time.

I’m going to focus some attention on the proxies and see if I can see any issues around connections to cdn.redhat.com