Katello::Errors::Pulp3Error: 'NoneType' object has no attribute 'url'

caseybea · May 26, 2021, 7:43pm

Still trying to migrate from pulp2 to pulp3… I’ve worked through some other issues (thank you all for the help), but now I’m stuck here:

Any ideas?

Running Prepare content for Pulp 3
================================================================================
Prepare content for Pulp 3:
Starting task.
2021-05-26 14:12:18 -0500: Migrating distribution content to Pulp 3 0/5Migration failed, You will want to investigate: https://katello.ctsi.mcw.edu/foreman_tasks/tasks/9b4cb2f0-0c03-4e79-b36e-6c729c6785d7
rake aborted!
ForemanTasks::TaskError: Task 9b4cb2f0-0c03-4e79-b36e-6c729c6785d7: Katello::Errors::Pulp3Error: 'NoneType' object has no attribute 'url'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.2.1/lib/katello/tasks/pulp3_migration.rake:33:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:pulp3_migration
(See full trace by running task with --trace)
                                                                      [FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 1
--------------------------------------------------------------------------------
Scenario [Prepare content for Pulp 3] failed.

The following steps ended up in failing state:

  [content-prepare]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="content-prepare"

Justin_Sherrill · May 27, 2021, 7:38pm

Check: journalctl -u pulpcore-worker@* and see if there are any tracebacks during that time.

caseybea · May 27, 2021, 10:54pm

Just ran it again to get fresh data. Here you go! Hope this help, thank you.

May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: c5f35d84-3b21-41de-a63f-c2112b7010ab
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: bf7ab3ac-f0c6-40f8-8799-8cacf5d19e87
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: d0ba0836-aecd-4769-9f05-f74cea0b4b12
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: d2eec823-5f46-4618-83fd-eff5dbd322c2
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: ceccf3f9-d4ab-4ae4-9146-629b71e650b8
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: c91595ad-5da2-475d-8398-00c9ca74e085
May 27 17:28:48 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without a remote pulp2 unit_id: ccba0a6f-cb68-44a8-9cf0-4cd8210efcfe
May 27 17:33:47 katello.ctsi.mcw.edu pulpcore-worker-8[50551]: pulp: rq.worker:INFO: Cleaning registries for queue: 50551@katello.ctsi.mcw.edu
May 27 17:33:48 katello.ctsi.mcw.edu pulpcore-worker-4[51201]: pulp: rq.worker:INFO: Cleaning registries for queue: 51201@katello.ctsi.mcw.edu
May 27 17:33:48 katello.ctsi.mcw.edu pulpcore-worker-6[51202]: pulp: rq.worker:INFO: Cleaning registries for queue: 51202@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-7[50814]: pulp: rq.worker:INFO: Cleaning registries for queue: 50814@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-3[51204]: pulp: rq.worker:INFO: Cleaning registries for queue: 51204@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-5[51200]: pulp: rq.worker:INFO: Cleaning registries for queue: 51200@katello.ctsi.mcw.edu
May 27 17:33:49 katello.ctsi.mcw.edu pulpcore-worker-1[51203]: pulp: rq.worker:INFO: Cleaning registries for queue: 51203@katello.ctsi.mcw.edu
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:ERROR: Traceback (most recent call last):
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: rv = job.perform()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self._result = self._execute()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return self.func(*self.args, **self.kwargs)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: migrate_content(plan, skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 55, in migrate_content
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 150, in migrate_content_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: loop.run_until_complete(dm.create())
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return future.result()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await asyncio.gather(*futures)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await self.run()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 181, in run
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self.migrate_to_pulp3(cmodel, ctype)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 375, in migrate_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: url=os.path.join(remote.url, relative_path),
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: AttributeError: 'NoneType' object has no attribute 'url'
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: Traceback (most recent call last):
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: rv = job.perform()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self._result = self._execute()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return self.func(*self.args, **self.kwargs)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: migrate_content(plan, skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 55, in migrate_content
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 150, in migrate_content_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: loop.run_until_complete(dm.create())
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: return future.result()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await asyncio.gather(*futures)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: await self.run()
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 181, in run
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: self.migrate_to_pulp3(cmodel, ctype)
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 375, in migrate_to_pulp3
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: url=os.path.join(remote.url, relative_path),
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: AttributeError: 'NoneType' object has no attribute 'url'
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: Cleaning registries for queue: 51205@katello.ctsi.mcw.edu
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: 51205@katello.ctsi.mcw.edu: 9f786f0e-c6be-4d40-bd02-30625d04bc0d
May 27 17:39:51 katello.ctsi.mcw.edu pulpcore-worker-2[51205]: pulp: rq.worker:INFO: 51205@katello.ctsi.mcw.edu: Job OK (9f786f0e-c6be-4d40-bd02-30625d04bc0d)
[root@katello ~]#

betriebsrat · June 1, 2021, 7:49am

running into the same problem as well

caseybea · June 3, 2021, 3:37pm

Any suggestions? I really want to migrate to pulp3

Justin_Sherrill · June 3, 2021, 10:03pm

the pulp team filed this issue: Issue #8862: ‘NoneType’ object has no attribute ‘url’ when migrating distribution tree content - Migration Plugin - Pulp

i think it may have to wait until they get a chance to look at it

dralley · June 4, 2021, 12:19am

@caseybea, Do you happen to know which repositories are present? (if there are few enough of them to list - if not, don’t worry too much about it)

caseybea · June 4, 2021, 1:58pm

I have repos for:
RedHat7
RedHat8
AlmaLinux (8)
Oracle Linux (8)
Rocky Linux (8)
EPEL7
EPEL8

And thank you for letting me know the pulp team logged it. Yay! Progress.

betriebsrat · June 7, 2021, 6:48am

Since i am running into the same issue… here are the ones we have in common:
Epel7
Epel8
OL8
furthermore CentOS6/7/8.

maniac777 · June 8, 2021, 5:07am

The same issue here. The same katello version. Repositories CentOS 6/7/8 epel 6/7/8 and some other repositories. I’ve tried to debug issue a bit. I’ve discovered that function get_remote_by_importer_id(lce.pulp2_importer_id) some times returns “None” and this is then added to list “remotes” and since it is none it does not have in fact “url”.