Pulp3 migration failed(foreman 2.3.3/katello 3.18.1)

Problem:
After upgrade foreman and katello, (based on Foreman :: Plugin Manuals) the Pulp3 Migration failed.
The first time with: HTTP status code: 403 . That was corrected with: UPDATE auth_user SET is_superuser = true WHERE id =1; on pulpcore BD
After that, the next error: column “pulp2_subid” of relation “pulp_2to3_migration_pulp2content” does not exist. That was corrected relaunching : foreman-installer
Now, the error is:
Distribution creation 0/8Migration failed, You will want to investigate: https://hostname/foreman_tasks/tasks/3568639a-17d0-4198-a12a-8b0a7d29cfb0
rake aborted!
ForemanTasks::TaskError: Task 3568639a-17d0-4198-a12a-8b0a7d29cfb0: Katello::Errors::Pulp3Error: 1 subtask(s) failed for task group /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/.
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/pulp3_migration.rake:31:in block (2 levels) in <top (required)>' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in block in execute’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in each' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in execute’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:195:in block in invoke_with_call_chain' /opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in mon_synchronize’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:188:in invoke_with_call_chain' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:181:in invoke’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:160:in invoke_task' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in block (2 levels) in top_level’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in each' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in block in top_level’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:125:in run_with_threads' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:110:in top_level’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:83:in block in run' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:186:in standard_exception_handling’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:80:in run' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in <top (required)>’
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in load' /opt/rh/rh-ruby25/root/usr/bin/rake:23:in
Tasks: TOP => katello:pulp3_migration

Expected outcome:

Foreman and Proxy versions:
foreman-2.3.3-1
foreman-proxy-2.3.3-1

Foreman and Proxy plugin versions:
foreman-tasks 3.0.3
foreman_ansible 6.1.1
foreman_chef 0.10.0
foreman_docker 5.0.0
foreman_remote_execution 4.2.2
katello 3.18.1

Distribution and version:
CentOS 7

Other relevant data:

pip3 list | grep pulp

pulp-2to3-migration (0.8.0)
pulp-certguard (1.0.3)
pulp-container (2.1.0)
pulp-deb (2.7.0)
pulp-file (1.3.0)
pulp-rpm (3.7.0)
pulpcore (3.7.3)

Content Migration Task, Tab Errors:

Action:

Actions::Pulp3::ContentMigration

Input:

{“services_checked”=>[“pulp3”],
“smart_proxy_id”=>1,
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>nil,
“current_timezone”=>“UTC”,
“current_user_id”=>1,
“current_organization_id”=>nil,
“current_location_id”=>nil}

Output:

{“pulp_tasks”=>
[{“pulp_href”=>"/pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/",
“pulp_created”=>“2021-03-11T20:06:53.779+00:00”,
“state”=>“completed”,
“name”=>“pulp_2to3_migration.app.tasks.migrate.migrate_from_pulp2”,
“started_at”=>“2021-03-11T20:06:53.904+00:00”,
“finished_at”=>“2021-03-11T20:07:20.925+00:00”,
“worker”=>"/pulp/api/v3/workers/b749cbfc-e5b3-4394-b7c8-9929ad67093b/",
“child_tasks”=>
["/pulp/api/v3/tasks/061eb774-bee1-44a4-b1f1-9a3617333ab8/",
“/pulp/api/v3/tasks/887c4f3f-3ea1-458e-8a2f-1be3bfd39388/”,
“/pulp/api/v3/tasks/77000d03-4533-4696-896e-7c8d69e796c1/”,
“/pulp/api/v3/tasks/5e8e263c-daaa-4d45-857e-de4a1698aa51/”,
“/pulp/api/v3/tasks/f63d7573-50bc-4857-88e2-e0484897d3ab/”,
“/pulp/api/v3/tasks/c535ff6d-1000-4447-bc53-05a252bba5d5/”,
“/pulp/api/v3/tasks/9cb31026-39ac-4ac4-b4ff-bdeeb6c0e6b1/”,
“/pulp/api/v3/tasks/23ab0a51-38f4-4f1c-bb89-f0d3ac107657/”,
“/pulp/api/v3/tasks/fc86fe61-72e5-44d8-aa5a-5b68d5cbabf8/”,
“/pulp/api/v3/tasks/3d0f29fb-94d1-49ff-8dd9-260b980952cc/”,
“/pulp/api/v3/tasks/e417d028-e8ca-4dc6-9618-37ddd52df0d3/”,
“/pulp/api/v3/tasks/17c5ea2a-aaed-43e4-a43d-302a6414d460/”,
“/pulp/api/v3/tasks/5caba0c0-1c2a-48e7-b4c2-cb7830c81363/”,
“/pulp/api/v3/tasks/631e9aa4-c1f0-47bb-97a1-ee1fc543b0bc/”,
“/pulp/api/v3/tasks/e3323633-2e91-4f69-97ce-37a7398d802b/”,
“/pulp/api/v3/tasks/d938eaa5-4e67-4e3e-b3f4-bec29112cd8e/”,
“/pulp/api/v3/tasks/4f8fb4e9-5fbe-4948-a044-243dcb15c70b/”,
“/pulp/api/v3/tasks/3249428f-7247-47de-a81d-59c746ebec6d/”,
“/pulp/api/v3/tasks/5214d155-a7e2-4035-91e0-df9efc1c8c95/”,
“/pulp/api/v3/tasks/9eb85d28-23de-4a33-aed9-c0125b1437c3/”,
“/pulp/api/v3/tasks/05149e1c-e43d-4137-a5ae-15349118d442/”,
“/pulp/api/v3/tasks/2144a007-af91-4ac1-a68b-597c24a8fa8a/”,
“/pulp/api/v3/tasks/38dc3b62-9853-4faa-980a-7fab5009ca87/”,
“/pulp/api/v3/tasks/6f82bcd7-a7f9-4aa7-8d58-2cdc4839b1a9/”,
“/pulp/api/v3/tasks/db7458ae-5d06-405a-a4a6-0d461e725938/”,
“/pulp/api/v3/tasks/bdfcba99-1a61-4ce7-afa2-4e9e86fdff7a/”,
“/pulp/api/v3/tasks/881b7927-66a1-4769-bbb8-e11fe8e7e3a8/”,
“/pulp/api/v3/tasks/4819e552-8634-4e66-b7b8-7ba85404b35d/”,
“/pulp/api/v3/tasks/989809cc-3d66-4e71-8a82-fab50b343f96/”,
“/pulp/api/v3/tasks/bfcd2e98-7235-4ad5-8594-1a8cd65f18d7/”,
“/pulp/api/v3/tasks/5b3214ab-5467-4aaf-b3d4-25a3708cc01d/”,
“/pulp/api/v3/tasks/d325b5d6-48a7-46f5-96d9-4f6dfc17926d/”,
“/pulp/api/v3/tasks/e1117369-fa93-4279-83b6-e9ce7fe2f445/”,
“/pulp/api/v3/tasks/be8e15c1-d52a-48cd-9ad8-cc4af230005f/”,
“/pulp/api/v3/tasks/1cfe39d0-87dd-46f0-a594-bd2449392ca0/”,
“/pulp/api/v3/tasks/d9bcc091-53d2-49a6-a026-6e53460bcc0e/”,
“/pulp/api/v3/tasks/33a42287-93bd-45a4-a8b9-d7a623291d7b/”,
“/pulp/api/v3/tasks/18281fe0-1f69-4880-9aa0-867bde8bfdd0/”,
“/pulp/api/v3/tasks/ea6375bc-bde4-4583-bb14-ac78c1cc65c5/”,
“/pulp/api/v3/tasks/742e36c2-dbe7-4fd5-992c-7d00b530fba3/”,
“/pulp/api/v3/tasks/9b576fdb-f72d-49dc-aecc-02ca63b8d429/”,
“/pulp/api/v3/tasks/32e228cb-5dec-4dd5-9691-6010a5a2fd06/”,
“/pulp/api/v3/tasks/8afb813f-5f2e-4adf-a47e-beb604b79c1a/”,
“/pulp/api/v3/tasks/fc1a855e-c47c-4ab8-810e-f3a58e718b5a/”,
“/pulp/api/v3/tasks/aaf14284-b219-4afa-b43f-d6dc46dbda68/”,
“/pulp/api/v3/tasks/b1effe13-6f53-473a-b1d1-fa4152d9007b/”,
“/pulp/api/v3/tasks/3e6af084-b8ec-4675-b20b-668033808919/”,
“/pulp/api/v3/tasks/3e6add08-aec6-40b4-b9c0-80d6d20a2fb6/”,
“/pulp/api/v3/tasks/4bd74494-e73d-4472-8c00-eba65929dd2b/”,
“/pulp/api/v3/tasks/f80d7735-bd6f-42e8-bdb9-f3872070b8db/”,
“/pulp/api/v3/tasks/989f8a14-3230-4da0-979e-016359125f35/”,
“/pulp/api/v3/tasks/63aed7e6-1bcf-49ab-9aec-891a50a8bb15/”,
“/pulp/api/v3/tasks/bf73d403-e174-40f4-a702-f46b81cb706c/”,
“/pulp/api/v3/tasks/70ca1b61-6e23-4492-9d7a-599c090f1839/”,
“/pulp/api/v3/tasks/999b2cfe-8d5f-449b-a21c-7a17119502cd/”,
“/pulp/api/v3/tasks/82a3e0eb-1c45-4f9a-af98-844b6d02dad7/”,
“/pulp/api/v3/tasks/a0ae0006-4f6f-40b3-bb69-c5908a804316/”,
“/pulp/api/v3/tasks/9ff0120b-b660-41f1-b026-ba03088252ab/”,
“/pulp/api/v3/tasks/b90d04a5-28ff-4d22-8836-fa43afe4ff6b/”,
“/pulp/api/v3/tasks/92eb0315-82a7-4b93-a944-8e1f28807a17/”,
“/pulp/api/v3/tasks/b5ff3b4e-be4f-4509-a337-370016126cda/”],
“task_group”=>
“/pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/”,
“progress_reports”=>
[{“message”=>“Pre-migrating Pulp 2 RPM content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 RPM content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 SRPM content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 SRPM content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 DISTRIBUTION content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 DISTRIBUTION content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 ERRATUM content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>514,
“done”=>473},
{“message”=>“Pre-migrating Pulp 2 ERRATUM content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>514,
“done”=>473},
{“message”=>“Processing Pulp 2 repositories, importers, distributors”,
“code”=>“processing.repositories”,
“state”=>“completed”,
“total”=>4,
“done”=>4},
{“message”=>“Pre-migrating Pulp 2 DEB content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 DEB content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 DEB_RELEASE content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 DEB_RELEASE content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 DEB_COMPONENT content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 DEB_COMPONENT content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 MODULEMD content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 MODULEMD content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 MODULEMD_DEFAULTS content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 MODULEMD_DEFAULTS content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 YUM_REPO_METADATA_FILE content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 YUM_REPO_METADATA_FILE content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 PACKAGE_LANGPACKS content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 PACKAGE_LANGPACKS content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 PACKAGE_GROUP content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Pre-migrating Pulp 2 PACKAGE_GROUP content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 PACKAGE_CATEGORY content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 PACKAGE_CATEGORY content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 PACKAGE_ENVIRONMENT content (general info)”,
“code”=>“premigrating.content.general”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>
“Pre-migrating Pulp 2 PACKAGE_ENVIRONMENT content (detail info)”,
“code”=>“premigrating.content.detail”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Creating repositories in Pulp 3”,
“code”=>“creating.repositories”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating importers to Pulp 3”,
“code”=>“migrating.importers”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating deb content to Pulp 3 deb_release”,
“code”=>“migrating.deb.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating deb content to Pulp 3 deb”,
“code”=>“migrating.deb.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating deb content to Pulp 3 deb_component”,
“code”=>“migrating.deb.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating deb content to Pulp 3 deb_component2”,
“code”=>“migrating.deb.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating deb content to Pulp 3 deb_component3”,
“code”=>“migrating.deb.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 rpm”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>4196,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 srpm”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 distribution”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 erratum”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 modulemd”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 modulemd_defaults”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 yum_repo_metadata_file”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 package_langpacks”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 package_group”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 package_category”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating rpm content to Pulp 3 package_environment”,
“code”=>“migrating.rpm.content”,
“state”=>“completed”,
“total”=>0,
“done”=>0},
{“message”=>“Migrating content to Pulp 3”,
“code”=>“migrating.content”,
“state”=>“completed”,
“total”=>4196,
“done”=>4196}],
“created_resources”=>
["/pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/"],
“reserved_resources_record”=>[“pulp_2to3_migration”]}],
“task_groups”=>
[{“pulp_href”=>
“/pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/”,
“description”=>“Migration Sub-tasks”,
“all_tasks_dispatched”=>true,
“waiting”=>0,
“skipped”=>0,
“running”=>2,
“completed”=>59,
“canceled”=>0,
“failed”=>1,
“group_progress_reports”=>
[{“message”=>“Distribution creation”,
“code”=>“create.distribution”,
“total”=>8,
“done”=>0},
{“message”=>“Repo version creation”,
“code”=>“create.repo_version”,
“total”=>0,
“done”=>0}]}],
“poll_attempts”=>{“total”=>27, “failed”=>1}}

Exception:

Katello::Errors::Pulp3Error: 1 subtask(s) failed for task group /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/.

Backtrace:

/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:102:in block in check_for_errors' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:100:in each’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:100:in check_for_errors' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:133:in poll_external_task’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:100:in poll_external_task_with_rescue' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:22:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/cancellable.rb:14:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:10:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:571:in block (3 levels) in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:16:in block in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:40:in block in as_remote_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/models/katello/concerns/user_extensions.rb:21:in cp_config’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:27:in as_cp_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:39:in as_remote_user’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:16:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/rails_executor_wrap.rb:14:in block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/execution_wrapper.rb:88:in wrap’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/rails_executor_wrap.rb:13:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:31:in with_progress_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:17:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_request_id.rb:15:in block in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_request_id.rb:49:in restore_current_request_id' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_request_id.rb:15:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_timezone.rb:15:in block in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_timezone.rb:44:in restore_curent_timezone' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_timezone.rb:15:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_user.rb:15:in block in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_user.rb:44:in restore_curent_user' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_user.rb:15:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in block in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in restore_current_taxonomies' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/world.rb:31:in execute’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:570:in block (2 levels) in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in catch’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in block in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in block in with_error_handling’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in with_error_handling’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:564:in execute_run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:285:in execute’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in block (2 levels) in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in with_meta_calculation’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in block in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in open_action’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/director.rb:93:in execute’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in block (2 levels) in perform' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors.rb:18:in run_user_code’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in block in perform' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in with_telemetry’
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in perform' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in perform’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in execute_job' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in block (2 levels) in process’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in block in invoke' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in invoke’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in block in process' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in block (6 levels) in dispatch’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in local' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in block (5 levels) in dispatch’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq.rb:37:in block in <module:Sidekiq>' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in block (4 levels) in dispatch’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in stats' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in block (3 levels) in dispatch’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in block (2 levels) in dispatch’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in global' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in block in dispatch’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in with_context' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in with_job_hash_context’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in dispatch' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in process’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in process_one' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in watchdog' /opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in block in safe_thread’
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context’

Hi @amesa-medina ,

The following error:

likely points to an error in Pulp 3’s backend. Do you see any error backtraces in /var/log/messages that popped up around the same time as that error? Sometimes the backtrace from the task in Foreman shows us the actual Pulp 3 error, but in the case of task groups it seems it isn’t reported there.

1 Like

Hi and thank you in advance :slight_smile:
I show you here fragments of this log(messages) around the time of the last error:

Mar 11 15:07:10 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:10 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:12 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:12 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 3960 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:12 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:12 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:13 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without an entry in the lazy catalog, pulp2 unit_id: 6b7f5719-ff27-4246-b86b-b53fd996881b
Mar 11 15:07:13 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without an entry in the lazy catalog, pulp2 unit_id: 28d28583-a22b-43c2-b6d7-25263a1d36bf
Mar 11 15:07:13 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without an entry in the lazy catalog, pulp2 unit_id: 51f5d2fd-e1cc-4e19-9389-f4d7da27ee7b
… many similar lines …
Mar 11 15:07:18 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: pulp_2to3_migration.app.plugin.content:WARNING: On_demand content cannot be migrated without an entry in the lazy catalog, pulp2 unit_id: edb8dd22-8893-48a2-bf83-7e5f4cc206f9
Mar 11 15:07:19 cpt-infra22l-d pulpcore-resource-manager[25633]: pulp: rq.worker:INFO: resource-manager: 66f04d27-c44d-4eab-b78e-079370434c02
Mar 11 15:07:19 cpt-infra22l-d pulpcore-worker-3[25160]: pulp: rq.worker:INFO: 25160@cpt-infra22l-d.acceo.com: 5e8e263c-daaa-4d45-857e-de4a1698aa51
Mar 11 15:07:19 cpt-infra22l-d pulpcore-resource-manager[25633]: pulp: rq.worker:INFO: resource-manager: Job OK (66f04d27-c44d-4eab-b78e-079370434c02)
Mar 11 15:07:19 cpt-infra22l-d pulpcore-resource-manager[25633]: pulp: rq.worker:INFO: resource-manager: 97a5e73c-794c-486d-8774-948bcd28559f
… many similar lines …
Mar 11 15:07:26 cpt-infra22l-d pulpcore-worker-2[25156]: pulp: rq.worker:INFO: 25156@cpt-infra22l-d.acceo.com: 584131d3-1f44-4927-af32-73bd6f432fe4
Mar 11 15:07:26 cpt-infra22l-d pulpcore-worker-2[25156]: pulp: rq.worker:INFO: 25156@cpt-infra22l-d.acceo.com: Job OK (584131d3-1f44-4927-af32-73bd6f432fe4)
Mar 11 15:07:27 cpt-infra22l-d pulpcore-worker-3[25160]: pulp: pulp_rpm.app.tasks.publishing:INFO: Publishing: repository=f5b6756b-bd2e-402f-8269-8847b8bea678, version=3
Mar 11 15:07:29 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:29 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 11587 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:29 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:29 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:07:33 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:07:33 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 11587 “-” “OpenAPI-Generator/3.7.1/ruby”
… many similar lines …
Mar 11 15:08:30 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:08:30 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 11587 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:08:30 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:08:30 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py”, line 78, in migrate_to_pulp3
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: TypeError: publish() got an unexpected keyword argument ‘sqlite_metadata’
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: During handling of the above exception, another exception occurred:
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: Traceback (most recent call last):
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 936, in perform_job
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: rv = job.perform()
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 684, in perform
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: self._result = self._execute()
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 690, in _execute
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: return self.func(*self.args, **self.kwargs)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py”, line 240, in complex_repo_migration
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: migrated_repo.pulp3_repository_version
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py”, line 360, in migrate_repo_distributor
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: pulp2dist, repo_version)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py”, line 81, in migrate_to_pulp3
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: publish(repo_version.pk, checksum_types=checksum_types)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py”, line 290, in publish
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: metadata_signing_service=metadata_signing_service
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py”, line 483, in create_repomd_xml
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: record.fill(checksum_type)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: OSError: Error while computing stat of compressed content of updateinfo.xml.zck:Cannot open a file updateinfo.xml.zck: createrepo_c wasn’t compiled with zchunk support
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: Traceback (most recent call last):
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py”, line 78, in migrate_to_pulp3
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: TypeError: publish() got an unexpected keyword argument ‘sqlite_metadata’
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: During handling of the above exception, another exception occurred:
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: Traceback (most recent call last):

Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: record.fill(checksum_type)
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: OSError: Error while computing stat of compressed content of updateinfo.xml.zck:Cannot open a file updateinfo.xml.zck: createrepo_c wasn’t compiled with zchunk support
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: pulp: rq.worker:INFO: 25159@cpt-infra22l-d.acceo.com: 88d7a555-20bb-4ce3-9408-dd05005f9fcf
Mar 11 15:08:44 cpt-infra22l-d pulpcore-worker-4[25159]: pulp: rq.worker:INFO: 25159@cpt-infra22l-d.acceo.com: Job OK (88d7a555-20bb-4ce3-9408-dd05005f9fcf)
Mar 11 15:08:46 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:08:46 +0000] “GET /pulp/api/v3/tasks/ea052d37-c235-4b4a-9cef-4ce62718c1c6/ HTTP/1.1” 200 11587 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:08:46 cpt-infra22l-d pulpcore-api[25051]: - - [11/Mar/2021:20:08:46 +0000] “GET /pulp/api/v3/task-groups/2e11a580-8850-4f2f-ade4-d27d5f9e9481/ HTTP/1.1” 200 440 “-” “OpenAPI-Generator/3.7.1/ruby”
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: pulp: rq.worker:ERROR: Traceback (most recent call last):
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py”, line 78, in migrate_to_pulp3
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: TypeError: publish() got an unexpected keyword argument ‘sqlite_metadata’
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: During handling of the above exception, another exception occurred:
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: Traceback (most recent call last):
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 936, in perform_job
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: rv = job.perform()
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 684, in perform
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: self._result = self._execute()
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 690, in _execute
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: return self.func(*self.args, **self.kwargs)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py”, l
ine 240, in complex_repo_migration
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: migrated_repo.pulp3_repository_version
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py”, l
ine 360, in migrate_repo_distributor
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: pulp2dist, repo_version)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repos
itory.py”, line 81, in migrate_to_pulp3
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: publish(repo_version.pk, checksum_types=checksum_types)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py”, line
290, in publish
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: metadata_signing_service=metadata_signing_service
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py”, line
483, in create_repomd_xml
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: record.fill(checksum_type)
Mar 11 15:08:50 cpt-infra22l-d pulpcore-worker-1[25158]: OSError: Error while computing stat of compressed content of updateinfo.xml.zck
:Cannot open a file updateinfo.xml.zck: createrepo_c wasn’t compiled with zchunk support

Thanks @amesa-medina. The Pulp 3 backtrace in there is helpful and I’m pretty certain it’s a Pulp 3 bug.

Do you mind filing a bug here with the backtrace you sent me and your Pulp versions here? https://pulp.plan.io/projects/migration/issues/new

I opened Issue #8400 on https://pulp.plan.io

Thank you

1 Like

Hi,
According to the answer given in this ticket Issue #8400: Migration fails if there is .zck metadata in pulp2 - Migration Plugin - Pulp, will there be a new katello 3.18.z package, to apply the fix? Thanks.