Can't Sync CentOS 7 Repo

eroji · October 22, 2019, 8:31pm

I’m unable to sync down CentOS 7 base repo with a fresh install of Katello 3.13. However, if I set up updates repo using the same exact method, it works.

Log output

Oct 22 13:19:52 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Downloading metadata from http://mirror.centos.org/centos/7/os/x86_64/.
Oct 22 13:19:52 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 13:19:52 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/repomd.xml.
Oct 22 13:19:53 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Parsing metadata.
Oct 22 13:19:53 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Downloading metadata from http://mirror.centos.org/centos/7/os/x86_64/.
Oct 22 13:19:53 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 13:19:53 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/repomd.xml.
Oct 22 13:19:54 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Parsing metadata.
Oct 22 13:19:54 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Downloading metadata from http://mirror.centos.org/centos/7/os/x86_64/.
Oct 22 13:19:54 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 13:19:54 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/repomd.xml.
Oct 22 13:19:55 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Parsing metadata.
Oct 22 13:19:55 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Downloading metadata from http://mirror.centos.org/centos/7/os/x86_64/.
Oct 22 13:19:55 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 13:19:55 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/repomd.xml.
Oct 22 13:19:56 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Parsing metadata.
Oct 22 13:19:56 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Downloading metadata files.
Oct 22 13:19:56 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (2): mirror.centos.org
Oct 22 13:19:56 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (3): mirror.centos.org
Oct 22 13:19:56 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (4): mirror.centos.org
Oct 22 13:19:56 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (5): mirror.centos.org
Oct 22 13:19:56 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/4af1fba0c1d6175b7e3c862b4bddfef93fffb84c37f7d5f18cfbff08abc47f8a-c7-x86_64-comps.xml.gz.
Oct 22 13:19:56 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/521f322f05f9802f2438d8bb7d97558c64ff3ff74c03322d77787ade9152d8bb-c7-x86_64-comps.xml.
Oct 22 13:19:57 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/dc871e4d16c013169e2e42f7a7a0659116f5b63f7e07d9a30dccbef2f71a3f0c-other.xml.gz.
Oct 22 13:19:57 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/afe9c5f88d9c365382c0553424b18192b79f3da7c1e3d120818203e9a5d1a70a-primary.xml.gz.
Oct 22 13:19:57 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/repodata/c1977f4def8aa2f0addb2695b9cc6a391ece8bb5d5b53d601db15df6240261f1-filelists.xml.gz.
Oct 22 13:19:58 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Generating metadata databases.
Oct 22 13:20:38 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Determining which units need to be downloaded.
Oct 22 13:22:30 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Downloading 0 RPMs.
Oct 22 13:22:59 katello pulp: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: [2e725669] The content container reported: {'downloads': {}, 'total_sources': 0} for base URL: http://mirror.centos.org/centos/7/os/x86_64/
Oct 22 13:22:59 katello pulp: pulp_rpm.plugins.importers.yum.purge:INFO: [2e725669] Removing missing units.
Oct 22 13:23:38 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [2e725669] Downloading additional units.
Oct 22 13:23:38 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 13:23:38 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/.treeinfo.
Oct 22 13:23:39 katello pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:INFO: [2e725669] downloading distribution units
Oct 22 13:23:39 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 13:23:40 katello pulp: nectar.downloaders.threaded:INFO: Download failed: Download of http://mirror.centos.org/centos/7/os/x86_64/PULP_DISTRIBUTION.xml failed with code 404: Not Found
Oct 22 13:23:40 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 13:23:40 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (2): mirror.centos.org
Oct 22 13:23:40 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (3): mirror.centos.org
Oct 22 13:23:40 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (4): mirror.centos.org
Oct 22 13:24:08 katello pulp: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.centos.org due to repeated connection failures: HTTPConnectionPool(host='mirror.centos.org', port=80): Read timed out.
Oct 22 13:24:08 katello pulp: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.centos.org due to repeated connection failures: HTTPConnectionPool(host='mirror.centos.org', port=80): Read timed out.
Oct 22 13:24:08 katello pulp: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.centos.org due to repeated connection failures: HTTPConnectionPool(host='mirror.centos.org', port=80): Read timed out.
Oct 22 13:24:08 katello pulp: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.centos.org due to repeated connection failures: HTTPConnectionPool(host='mirror.centos.org', port=80): Read timed out.
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: [2e725669] some distro file downloads failed
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888) Traceback (most recent call last):
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 302, in run
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)     dist_sync.run()
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 128, in run
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)     self._run(tmp_dir)
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 206, in _run
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)     for destination, location in downloaded:
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 295, in download_files
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888)     raise DownloadFailed()
Oct 22 13:24:08 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [2e725669] (8247-05888) DownloadFailed
Oct 22 13:24:09 katello pulp: pulp.server.async.tasks:INFO: [2e725669] Task failed : [2e725669-d08a-41c0-8bf6-bc8865946ae8]
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888) Task pulp.server.managers.repo.sync.sync[2e725669-d08a-41c0-8bf6-bc8865946ae8] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888) Traceback (most recent call last):
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)     R = retval = fun(*args, **kwargs)
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 529, in __call__
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)     return super(Task, self).__call__(*args, **kwargs)
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)     return super(PulpTask, self).__call__(*args, **kwargs)
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)     return self.run(*args, **kwargs)
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 828, in sync
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Oct 22 13:24:09 katello pulp: celery.app.trace:ERROR: [2e725669] (8247-05888) PulpExecutionException: Importer indicated a failed response

eroji · October 23, 2019, 12:07am

The above is the log output from re-running sync. First time running, it clearly shows packages were downloaded. However, it will end in error in exact same way.

Oct 22 15:55:57 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/Packages/zziplib-utils-0.13.62-11.el7.x86_64.rpm.
Oct 22 15:55:58 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/Packages/zenity-3.28.1-1.el7.x86_64.rpm.
Oct 22 15:56:00 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/Packages/xulrunner-31.6.0-2.el7.centos.x86_64.rpm.
Oct 22 15:56:04 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/Packages/xulrunner-31.6.0-2.el7.centos.i686.rpm.
Oct 22 15:56:05 katello pulp: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: [55385c55] The content container reported: {'downloads': {'___/primary/___': {'total_failed': 0, 'total_succeeded': 10097}}, 'total_sources': 0} for base URL: http://mirror.centos.org/centos/7/os/x86_64/
Oct 22 15:56:05 katello pulp: pulp_rpm.plugins.importers.yum.purge:INFO: [55385c55] Removing missing units.
Oct 22 15:56:27 katello mongod.27017[2618]: [conn49] build index on: pulp_database.repo_content_units properties: { v: 2, unique: true, key: { repo_id: -1, unit_type_id: -1, unit_id: -1 }, name: "repo_id_-1_unit_type_id_-1_unit_id_-1", ns: "pulp_database.repo_content_units", background: true }
Oct 22 15:56:27 katello mongod.27017[2618]: [conn49] build index done.  scanned 10097 total records. 0 secs
Oct 22 15:56:27 katello mongod.27017[2618]: [conn81] build index on: pulp_database.repo_content_units properties: { v: 2, key: { repo_id: -1, unit_type_id: -1 }, name: "repo_id_-1_unit_type_id_-1", ns: "pulp_database.repo_content_units", background: true }
Oct 22 15:56:27 katello mongod.27017[2618]: [conn81] build index done.  scanned 10097 total records. 0 secs
Oct 22 15:56:27 katello mongod.27017[2618]: [conn49] build index on: pulp_database.repo_content_units properties: { v: 2, key: { unit_type_id: -1, created: -1 }, name: "unit_type_id_-1_created_-1", ns: "pulp_database.repo_content_units", background: true }
Oct 22 15:56:27 katello mongod.27017[2618]: [conn49] build index done.  scanned 10097 total records. 0 secs
Oct 22 15:56:27 katello mongod.27017[2618]: [conn81] build index on: pulp_database.repo_content_units properties: { v: 2, key: { unit_id: -1 }, name: "unit_id_-1", ns: "pulp_database.repo_content_units", background: true }
Oct 22 15:56:27 katello mongod.27017[2618]: [conn81] build index done.  scanned 10097 total records. 0 secs
Oct 22 15:56:34 katello pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [55385c55] Downloading additional units.
Oct 22 15:56:34 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 15:56:34 katello pulp: nectar.downloaders.threaded:INFO: Download succeeded: http://mirror.centos.org/centos/7/os/x86_64/.treeinfo.
Oct 22 15:56:35 katello pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:INFO: [55385c55] downloading distribution units
Oct 22 15:56:35 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 15:56:36 katello pulp: nectar.downloaders.threaded:INFO: Download failed: Download of http://mirror.centos.org/centos/7/os/x86_64/PULP_DISTRIBUTION.xml failed with code 404: Not Found
Oct 22 15:56:36 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): mirror.centos.org
Oct 22 15:56:36 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (2): mirror.centos.org
Oct 22 15:56:36 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (3): mirror.centos.org
Oct 22 15:56:36 katello pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (4): mirror.centos.org
Oct 22 15:57:04 katello pulp: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.centos.org due to repeated connection failures: HTTPConnectionPool(host='mirror.centos.org', port=80): Read timed out.
Oct 22 15:57:04 katello pulp: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.centos.org due to repeated connection failures: HTTPConnectionPool(host='mirror.centos.org', port=80): Read timed out.
Oct 22 15:57:04 katello pulp: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.centos.org due to repeated connection failures: HTTPConnectionPool(host='mirror.centos.org', port=80): Read timed out.
Oct 22 15:57:04 katello pulp: nectar.downloaders.threaded:ERROR: Skipping requests to mirror.centos.org due to repeated connection failures: HTTPConnectionPool(host='mirror.centos.org', port=80): Read timed out.
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.parse.treeinfo:ERROR: [55385c55] some distro file downloads failed
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016) Traceback (most recent call last):
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 302, in run
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)     dist_sync.run()
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 128, in run
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)     self._run(tmp_dir)
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 206, in _run
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)     for destination, location in downloaded:
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 295, in download_files
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016)     raise DownloadFailed()
Oct 22 15:57:05 katello pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [55385c55] (8053-10016) DownloadFailed

eroji · October 24, 2019, 6:22pm

Anyone?