[ContentMigration] foreman-maintain content switchover - null value in column

Problem:

*# foreman-maintain content switchover*
*Running Switch support for certain content from Pulp 2 to Pulp 3*
*================================================================================*
*Switch support for certain content from Pulp 2 to Pulp 3: *
*Performing final content migration before switching content           [FAIL]*
*Failed executing foreman-rake katello:pulp3_migration, exit status 1:*
* enabled*
*Migration failed, You will want to investigate: https://katello/foreman_tasks/tasks/469c87ff-1433-468c-acf9-8565d2223f17*
*rake aborted!*
*ForemanTasks::TaskError: Task 469c87ff-1433-468c-acf9-8565d2223f17: Katello::Errors::Pulp3Error: null value in column "stream" violates not-null constraint*
*DETAIL:  Failing row contains (9c908b9d-d383-47af-af5a-60a995144405, 2021-01-07 13:15:21.860409+00, 2021-01-07 13:15:21.860415+00, libuv, null, {"epel8-buildroot": ["devel"]}, 35915a072c14b42259f6a283cb3848d0a8d251c0f9b9e8777fea0c7e12fdd2ef, 3-CV_CentOS_8-v16_0-9c3c3f91-0eaa-4fdc-8250-3ecddff4689c, a8cb0ecc-8480-4cc2-b9bc-bff1cac57891).*
*/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/lib/katello/tasks/pulp3_migration.rake:31:in `block (2 levels) in <top (required)>'*
*/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'*
*Tasks: TOP => katello:pulp3_migration*
*(See full trace by running task with --trace)*
*Starting task.*
*2021-01-07 08:15:33 -0500: Pre-migrating Pulp 2 ERRATUM content (detail info) 34615/35159*  
--------------------------------------------------------------------------------
Scenario [Switch support for certain content from Pulp 2 to Pulp 3] failed.

The following steps ended up in failing state:

  [content-switchover]
**NOTE**
This execution took less than 10 minutes to come up with this error.
I am in the process of re-running the migration
* gathered information for this post then re-ran the below.  No changes to the system or content. I did not restart the services nor reboot the server.
* foreman-maintain content migration-stats
* foreman-maintain content prepare
* foreman-maintain content switchover

It has been running for ~30 minutes at this time. I will update this with the results.

I am currently re-running
Expected outcome:
Successful switch to pulp3

Foreman and Proxy versions:

  • foreman-2.3.1-1.el7.noarch
  • foreman-cli-2.3.1-1.el7.noarch
  • foreman-debug-2.3.1-1.el7.noarch
  • foreman-dynflow-sidekiq-2.3.1-1.el7.noarch
  • foreman-installer-2.3.1-1.el7.noarch
  • foreman-installer-katello-2.3.1-1.el7.noarch
  • foreman-postgresql-2.3.1-1.el7.noarch
  • foreman-proxy-2.3.1-1.el7.noarch
  • foreman-release-2.3.1-1.el7.noarch
  • foreman-release-scl-7-3.el7.noarch
  • foreman-selinux-2.3.1-1.el7.noarch
  • foreman-service-2.3.1-1.el7.noarch
  • katello-3.18.0-1.el7.noarch
  • katello-3.18.0-1.el7.noarch

Distribution and version:
CentOS Linux release 7.9.2009 (Core)

Other relevant data:

From TASK:
null value in column "stream" violates not-null constraint
DETAIL:  Failing row contains (9c908b9d-d383-47af-af5a-60a995144405, 2021-01-07 13:15:21.860409+00, 2021-01-07 13:15:21.860415+00, libuv, null, {"epel8-buildroot": ["devel"]}, 35915a072c14b42259f6a283cb3848d0a8d251c0f9b9e8777fea0c7e12fdd2ef, 3-CV_CentOS_8-v16_0-9c3c3f91-0eaa-4fdc-8250-3ecddff4689c, a8cb0ecc-8480-4cc2-b9bc-bff1cac57891).

From DYNFLOW:

2: Actions::Pulp3::ContentMigration (skipped) [ 244.19s / 5.54s ]
Queue: default

Started at: 2021-01-07 13:11:32 UTC

Ended at: 2021-01-07 13:15:36 UTC

Real time: 244.19s

Execution time (excluding suspended state): 5.54s

Input:

---
services_checked:
- pulp3
smart_proxy_id: 1
remote_user: admin
remote_cp_user: admin
current_request_id: 
current_timezone: UTC
current_user_id: 1
current_organization_id: 
current_location_id: 
Output:

---
pulp_tasks:
- pulp_href: "/pulp/api/v3/tasks/46506b1c-9019-4e62-913d-edcf0ae71ff5/"
  pulp_created: '2021-01-07T13:11:34.626+00:00'
  state: failed
  name: pulp_2to3_migration.app.tasks.migrate.migrate_from_pulp2
  started_at: '2021-01-07T13:11:34.947+00:00'
  finished_at: '2021-01-07T13:15:21.997+00:00'
  error:
    traceback: |2
        File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
          rv = job.perform()
        File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
          self._result = self._execute()
        File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
          return self.func(*self.args, **self.kwargs)
        File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 137, in migrate_from_pulp2
          pre_migrate_all_content(plan)
        File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 68, in pre_migrate_all_content
          pre_migrate_content_type(content_model, mutable_type, lazy_type, premigrate_hook)
        File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 212, in pre_migrate_content_type
          content_model.pulp_2to3_detail.pre_migrate_content_detail(pulp2content_batch)
        File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/pulp_2to3_models.py", line 672, in pre_migrate_content_detail
          batch_size=DEFAULT_BATCH_SIZE)
        File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
          return getattr(self.get_queryset(), name)(*args, **kwargs)
        File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 468, in bulk_create
          self._batched_insert(objs_with_pk, fields, batch_size, ignore_conflicts=ignore_conflicts)
        File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1211, in _batched_insert
          self._insert(item, fields=fields, using=self.db, ignore_conflicts=ignore_conflicts)
        File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 1186, in _insert
          return query.get_compiler(using=using).execute_sql(return_id)
        File "/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1377, in execute_sql
          cursor.execute(sql, params)
        File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
          return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
        File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
          return executor(sql, params, many, context)
        File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
          return self.cursor.execute(sql, params)
        File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
          raise dj_exc_value.with_traceback(traceback) from exc_value
        File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
          return self.cursor.execute(sql, params)
    description: |
      null value in column "stream" violates not-null constraint
      DETAIL:  Failing row contains (9c908b9d-d383-47af-af5a-60a995144405, 2021-01-07 13:15:21.860409+00, 2021-01-07 13:15:21.860415+00, libuv, null, {"epel8-buildroot": ["devel"]}, 35915a072c14b42259f6a283cb3848d0a8d251c0f9b9e8777fea0c7e12fdd2ef, 3-CV_CentOS_8-v16_0-9c3c3f91-0eaa-4fdc-8250-3ecddff4689c, a8cb0ecc-8480-4cc2-b9bc-bff1cac57891).
  worker: "/pulp/api/v3/workers/c63718df-291f-44fd-bc0f-b28a93c3acb8/"
  child_tasks: []
  task_group: "/pulp/api/v3/task-groups/52421fdd-2e82-4908-8235-9cebc50a4ed4/"
  progress_reports:
  - message: Pre-migrating Pulp 2 RPM content (general info)
    code: premigrating.content.general
    state: completed
    total: 71624
    done: 71624
  - message: Pre-migrating Pulp 2 RPM content (detail info)
    code: premigrating.content.detail
    state: completed
    total: 71624
    done: 71624
  - message: Pre-migrating Pulp 2 ERRATUM content (general info)
    code: premigrating.content.general
    state: completed
    total: 35303
    done: 35303
  - message: Pre-migrating Pulp 2 SRPM content (general info)
    code: premigrating.content.general
    state: completed
    total: 0
    done: 0
  - message: Pre-migrating Pulp 2 SRPM content (detail info)
    code: premigrating.content.detail
    state: completed
    total: 0
    done: 0
  - message: Pre-migrating Pulp 2 ERRATUM content (detail info)
    code: premigrating.content.detail
    state: completed
    total: 35303
    done: 35303
  - message: Processing Pulp 2 repositories, importers, distributors
    code: processing.repositories
    state: completed
    total: 533
    done: 533
  - message: Pre-migrating Pulp 2 DISTRIBUTION content (general info)
    code: premigrating.content.general
    state: completed
    total: 5
    done: 5
  - message: Pre-migrating Pulp 2 DISTRIBUTION content (detail info)
    code: premigrating.content.detail
    state: completed
    total: 5
    done: 5
  - message: Pre-migrating Pulp 2 MODULEMD content (general info)
    code: premigrating.content.general
    state: completed
    total: 189
    done: 189
  - message: Pre-migrating Pulp 2 MODULEMD content (detail info)
    code: premigrating.content.detail
    state: completed
    total: 189
    done: 189
  - message: Pre-migrating Pulp 2 MODULEMD_DEFAULTS content (detail info)
    code: premigrating.content.detail
    state: running
    total: 360
    done: 0
  - message: Pre-migrating Pulp 2 MODULEMD_DEFAULTS content (general info)
    code: premigrating.content.general
    state: running
    total: 360
    done: 360
  created_resources:
  - "/pulp/api/v3/task-groups/52421fdd-2e82-4908-8235-9cebc50a4ed4/"
  reserved_resources_record:
  - pulp_2to3_migration
task_groups:
- pulp_href: "/pulp/api/v3/task-groups/52421fdd-2e82-4908-8235-9cebc50a4ed4/"
  description: Migration Sub-tasks
  all_tasks_dispatched: false
  waiting: 0
  skipped: 0
  running: 0
  completed: 0
  canceled: 0
  failed: 1
  group_progress_reports:
  - message: Repo version creation
    code: create.repo_version
    total: 0
    done: 0
  - message: Distribution creation
    code: create.distribution
    total: 0
    done: 0
poll_attempts:
  total: 35
  failed: 1
Error:

Katello::Errors::Pulp3Error

null value in column "stream" violates not-null constraint DETAIL: Failing row contains (9c908b9d-d383-47af-af5a-60a995144405, 2021-01-07 13:15:21.860409+00, 2021-01-07 13:15:21.860415+00, libuv, null, {"epel8-buildroot": ["devel"]}, 35915a072c14b42259f6a283cb3848d0a8d251c0f9b9e8777fea0c7e12fdd2ef, 3-CV_CentOS_8-v16_0-9c3c3f91-0eaa-4fdc-8250-3ecddff4689c, a8cb0ecc-8480-4cc2-b9bc-bff1cac57891).

---
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/pulp3/abstract_async_task.rb:102:in
  `block in check_for_errors'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/pulp3/abstract_async_task.rb:100:in
  `each'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/pulp3/abstract_async_task.rb:100:in
  `check_for_errors'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/pulp3/abstract_async_task.rb:133:in
  `poll_external_task'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:100:in
  `poll_external_task_with_rescue'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:22:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/cancellable.rb:14:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/pulp3/abstract_async_task.rb:10:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:571:in
  `block (3 levels) in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/middleware/remote_action.rb:16:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/middleware/remote_action.rb:40:in
  `block in as_remote_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/models/katello/concerns/user_extensions.rb:21:in
  `cp_config'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/middleware/remote_action.rb:27:in
  `as_cp_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/middleware/remote_action.rb:39:in
  `as_remote_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.0/app/lib/actions/middleware/remote_action.rb:16:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/rails_executor_wrap.rb:14:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/execution_wrapper.rb:88:in
  `wrap'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/rails_executor_wrap.rb:13:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:31:in
  `with_progress_calculation'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:17:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_request_id.rb:49:in
  `restore_current_request_id'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_timezone.rb:44:in
  `restore_curent_timezone'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_user.rb:15:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_user.rb:44:in
  `restore_curent_user'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_user.rb:15:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in
  `block in run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in
  `restore_current_taxonomies'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in
  `pass'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/world.rb:31:in
  `execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:570:in
  `block (2 levels) in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in
  `catch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in
  `block in execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in
  `block in with_error_handling'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in
  `catch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in
  `with_error_handling'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:564:in
  `execute_run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:285:in
  `execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in
  `block (2 levels) in execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in
  `with_meta_calculation'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in
  `block in execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in
  `open_action'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in
  `execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/director.rb:93:in
  `execute'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in
  `block (2 levels) in perform'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors.rb:18:in
  `run_user_code'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in
  `block in perform'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in
  `with_telemetry'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in
  `perform'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in
  `perform'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in
  `execute_job'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in
  `block (2 levels) in process'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in
  `block in invoke'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in
  `invoke'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in
  `block in process'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in
  `block (6 levels) in dispatch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in
  `local'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in
  `block (5 levels) in dispatch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq.rb:37:in
  `block in <module:Sidekiq>'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in
  `block (4 levels) in dispatch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in
  `stats'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in
  `block (3 levels) in dispatch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in
  `call'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in
  `block (2 levels) in dispatch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in
  `global'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in
  `block in dispatch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in
  `with_context'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in
  `with_job_hash_context'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in
  `dispatch'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in
  `process'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in
  `process_one'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in
  `run'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in
  `watchdog'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in
  `block in safe_thread'"
- "/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in
  `block in create_with_logging_context'"

#cmd line# date ; echo "======================" ;foreman-maintain content switchover ; echo "===============================" ; date

Thu Jan 7 08:54:27 EST 2021

======================

Running Switch support for certain content from Pulp 2 to Pulp 3

================================================================================

Switch support for certain content from Pulp 2 to Pulp 3:

Performing final content migration before switching content

enabled

Starting task.

2021-01-07 10:45:08 -0500: Importing migrated content type erratum: 36000/35303Content Migration completed successfully

Performing a check to verify everything that is needed has been migrated

Switching specified content over to pulp 3 [FAIL]

Failed executing foreman-rake katello:pulp3_content_switchover, exit status 1:

ERROR: at least one katello_rpms record has migrated_pulp3_href NULL value


Scenario [Switch support for certain content from Pulp 2 to Pulp 3] failed.

The following steps ended up in failing state:

[content-switchover]

Resolve the failed steps and rerun

the command. In case the failures are false positives,

use --whitelist="content-switchover"

===============================

Thu Jan 7 10:45:45 EST 2021

Hi @kc_cohen,

Did you try anything in particular to get the Pulp 3 migration that was failing to work? In your latest comment it looks like the migration worked but now the switchover is failing.

Also, can you please confirm your Pulpcore versions?

pip3 list | grep pulp

Here is the output from the pip3 list

  • pulp-2to3-migration (0.6.0)
  • pulp-certguard (1.0.3)
  • pulp-container (2.1.0)
  • pulp-file (1.3.0)
  • pulp-rpm (3.7.0)
  • pulpcore (3.7.3)

I did nothing special … All I did is re run the "foreman-maintain content switchover " command a 2nd time.

The server is a VM. So I was unsure of the state of the server … I rolled back to the snapshot that I took before I started this process. Part of the reason was to free up the additional 200 GB that I had to add for the migration (which it did not look like it used any of that space)

Thanks for the info.

Which repositories are you using that have modular content? I see epel8 mentioned. It would be good if we can reproduce the issue you had.

This issue might be related https://pulp.plan.io/issues/7846

It seems to be an ongoing issue that others have experienced.

Here is the output from “hammer repository list” (had to get rid of the http in for the URLs in order to post the below)

-------------------------------|---------------------|--------------|--------------------------------------------------------------------------------
 NAME                          | PRODUCT             | CONTENT TYPE | URL                                                                                 
-------------------------------|---------------------|--------------|--------------------------------------------------------------------------------
 Alienvault-Agent-CentOS-7     | Binary Defense      | yum          | s://prod-usm-saas-agent-config.s3.amazonaws.com/repo/rpm/x86_64/                    
 CandlePin                     | Foreman-Katello     | yum          | s://fedorapeople.org/groups/katello/releases/yum/3.14/candlepin/el7/x86_64/
 CentOS-7-base                 | CentOS              | yum          | ://mirror.centos.org/centos/7/os/x86_64/                                            
 CentOS-7-extras               | CentOS              | yum          | ://mirror.centos.org/centos/7/extras/x86_64/                                        
 CentOS-7-SCLo-rh              | CentOS              | yum          | ://mirror.centos.org/centos/7/sclo/x86_64/rh/                                       
 CentOS-7-updates              | CentOS              | yum          | ://mirror.centos.org/centos/7/updates/x86_64/                                       
 CentOS-8-Appstream            | CentOS              | yum          | ://mirror.centos.org/centos-8/8/AppStream/x86_64/os/                                
 CentOS-8-base                 | CentOS              | yum          | ://mirror.centos.org/centos-8/8/BaseOS/x86_64/os/                                   
 CentOS-8-extras               | CentOS              | yum          | ://mirror.centos.org/centos-8/8/extras/x86_64/os/                                   
 CentOS-8-PowerTools           | CentOS              | yum          | ://mirror.centos.org/centos-8/8/PowerTools/x86_64/os/                               
 Cisco_AMP_7                   | Cisco_amp           | yum          | ://katello.cohen.cpa/pub/custom_repo/cisco_amp                                      
 Cisco Amp CentOS 7            | File Based Products | file         |                                                                                     
 Docker repo                   | Docker              | yum          | s://download.docker.com/linux/centos/7/x86_64/stable                                
 DUO_Security-CentOS-7         | DUO                 | yum          | s://pkg.duosecurity.com/CentOS/7/x86_64                                             
 DUO_Security-CentOS-8         | DUO                 | yum          | s://pkg.duosecurity.com/CentOS/8/x86_64                                             
 EPEL-7                        | EPEL                | yum          | ://dl.fedoraproject.org/pub/epel/7//x86_64/                                         
 EPEL-8                        | EPEL                | yum          | s://dl.fedoraproject.org/pub/epel/8/Everything/x86_64                               
 EPEL-8-modular                | EPEL                | yum          | s://dl.fedoraproject.org/pub/epel/8/Modular/x86_64                                  
 Foreman                       | Foreman-Katello     | yum          | s://yum.theforeman.org/releases/1.24/el7/x86_64                                     
 Foreman Client EL7            | Foreman Client      | yum          | s://yum.theforeman.org/client/1.24/el7/x86_64                                       
 Foreman-Plugins               | Foreman-Katello     | yum          | s://yum.theforeman.org/plugins/1.24/el7/x86_64                                      
 grafana                       | Grafana             | yum          | s://packages.grafana.com/oss/rpm                                                
 Jenkins-RH-Stable             | Jenkins             | yum          | ://pkg.jenkins.io/redhat-stable                                                     
 Katello                       | Foreman-Katello     | yum          | s://fedorapeople.org/groups/katello/releases/yum/3.14/katello/el7/x86_64/  
 MySQL_8_CommunityServer_el7   | MySQL               | yum          | ://repo.mysql.com/yum/mysql-8.0-community/el/7/x86_64/                              
 MySQL_CommunityConnectors_el7 | MySQL               | yum          | ://repo.mysql.com/yum/mysql-connectors-community/el/7/x86_64                        
 MySQL_CommunityTools_el7      | MySQL               | yum          | ://repo.mysql.com/yum/mysql-tools-community/el/7/x86_64/                            
 NGINX_7                       | NGINX               | yum          | ://nginx.org/packages/centos/7/x86_64/                                              
 NGINX_8                       | NGINX               | yum          | ://nginx.org/packages/centos/8/x86_64/                                              
 NodeSource_7_v10              | NodeSource          | yum          | s://rpm.nodesource.com/pub_10.x/el/7/x86_64/                                        
 NodeSource_7_v12              | NodeSource          | yum          | s://rpm.nodesource.com/pub_12.x/el/7/x86_64/                                        
 NodeSource_7_v14              | NodeSource          | yum          | s://rpm.nodesource.com/pub_14.x/el/7/x86_64/                                        
 NodeSource_8_v10              | NodeSource          | yum          | s://rpm.nodesource.com/pub_10.x/el/8/x86_64/                                        
 NodeSource_8_v12              | NodeSource          | yum          | s://rpm.nodesource.com/pub_12.x/el/8/x86_64/                                        
 NodeSource_8_v14              | NodeSource          | yum          | s://rpm.nodesource.com/pub_14.x/el/8/x86_64/                                        
 Packages-7                    | Microsoft           | yum          | s://packages.microsoft.com/rhel/7/prod/                                             
 pulp-2                        | pulp-2              | yum          | s://repos.fedorapeople.org/repos/pulp/pulp/stable/2/7/x86_64                        
 Puppet-6-CentOS-7             | Puppet              | yum          | ://yum.puppetlabs.com/puppet6/el/7/x86_64                                           
 Puppet-6-CentOS-8             | Puppet              | yum          | ://yum.puppetlabs.com/puppet/el/8/x86_64                                            
 zabbix_official_el7           | Zabbix              | yum          | ://repo.zabbix.com/zabbix/5.0/rhel/7/x86_64/

@kc_cohen thanks for that list. Granted that this is an ongoing Pulp 3 bug that some others have experienced, I might suggest waiting to do the content switchover and sticking around with Pulp 2 a little bit longer. Maybe at least until that bug is no longer an issue. Let me know if for some reason you can’t roll back your VM again and need to find a work around.

I’ll add that a Pulp 3 developer recently found an issue that might be the cause of the bug, so there just might be a fix soon!

@iballou -

The server is a VM and I rolled back to the snapshot I took prior to the Pulp 3 migration attempt. So, I am good on Pulp 2.

Once a fix is rolled out, I can apply it and try the migration again.

Thank you for the updates.

1 Like