CentOS repository sync frustrations

The issue still persist in the latest Katello Foreman releases. As mentioned by some folks earlier in this thread, the extraneous parameters are not removed when loading the RPM from the mirror

Sep 23 11:04:45 foreman pulpcore-worker-2: pulp [50980a98-d6ef-490b-b2e9-bfc29bd651d2]: backoff:ERROR: Giving up download_wrapper(...) after 1 tries (aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found', url=URL('http://linuxsoft.cern.ch/epel//7/x86_64/repodata/repomd.xml.asc?repo=epel-7&arch=x86_64'))

and you sill see the timeout error

Sep 23 11:11:19 foreman pulpcore-worker-2: pulp [50980a98-d6ef-490b-b2e9-bfc29bd651d2]: pulpcore.tasking.pulpcore_worker:INFO: Task eed1ba33-5b55-446a-a31f-9ae9a8fd4fc6 failed ()
Sep 23 11:11:19 foreman pulpcore-worker-2: pulp [50980a98-d6ef-490b-b2e9-bfc29bd651d2]: pulpcore.tasking.pulpcore_worker:INFO:   File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 272, in _perform_task
Sep 23 11:11:19 foreman pulpcore-worker-2: result = func(*args, **kwargs)
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 427, in synchronize
Sep 23 11:11:19 foreman pulpcore-worker-2: if optimize and is_optimized_sync(repository, remote, remote_url):
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 151, in create
Sep 23 11:11:19 foreman pulpcore-worker-2: loop.run_until_complete(pipeline)
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Sep 23 11:11:19 foreman pulpcore-worker-2: return future.result()
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Sep 23 11:11:19 foreman pulpcore-worker-2: await asyncio.gather(*futures)
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Sep 23 11:11:19 foreman pulpcore-worker-2: await self.run()
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 181, in run
Sep 23 11:11:19 foreman pulpcore-worker-2: # asyncio.wait does not cancel its tasks when cancelled, we need to do this
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 207, in _handle_content_unit
Sep 23 11:11:19 foreman pulpcore-worker-2: A Stages API stage that saves any unsaved :attr:`DeclarativeArtifact.artifact` objects.
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/models.py", line 89, in download
Sep 23 11:11:19 foreman pulpcore-worker-2: download_result = await downloader.run(extra_data=self.extra_data)
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 258, in run
Sep 23 11:11:19 foreman pulpcore-worker-2: return await download_wrapper()
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/backoff/_async.py", line 133, in retry
Sep 23 11:11:19 foreman pulpcore-worker-2: ret = await target(*args, **kwargs)
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 256, in download_wrapper
Sep 23 11:11:19 foreman pulpcore-worker-2: return await self._run(extra_data=extra_data)
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/downloaders.py", line 95, in _run
Sep 23 11:11:19 foreman pulpcore-worker-2: url, proxy=self.proxy, proxy_auth=self.proxy_auth, auth=self.auth
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib/python3.6/site-packages/pulpcore/download/http.py", line 210, in _handle_response
Sep 23 11:11:19 foreman pulpcore-worker-2: chunk = await response.content.read(1048576)  # 1 megabyte
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib64/python3.6/site-packages/aiohttp/streams.py", line 380, in read
Sep 23 11:11:19 foreman pulpcore-worker-2: await self._wait("read")
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib64/python3.6/site-packages/aiohttp/streams.py", line 306, in _wait
Sep 23 11:11:19 foreman pulpcore-worker-2: await waiter
Sep 23 11:11:19 foreman pulpcore-worker-2: File "/usr/lib64/python3.6/site-packages/aiohttp/helpers.py", line 656, in __exit__
Sep 23 11:11:19 foreman pulpcore-worker-2: raise asyncio.TimeoutError from None

I will now try to create a new repository with a direct URL (no mirror list) and see if that finishes the download.

An additional observation, there is still no info in the logs which file was about to be downloaded. Also not in the Dynflow console.

Created a new repository with a different URL from the mirror list, same problem. It just gets stuck reading 1MB of data, not telling my where or why. Nothing in the logs, no traces anywhere. Simple EPEL 7 repo. I am again at a dead end.

Mine all seem to be sync’ing gracefully, including mirrorlist URLs, after upgrading to:

foreman-3.0.0-0.1.rc2.el7.noarch
katello-4.2.0.rc1-1.el7.noarch

Since 4.2.0 RC1 was released, any sync’ing issues I’ve had were somewhere else.