More Pulp Woes

Support
,
1

Problem:
Tried to disable a RHEL 7 repo and received the following:

Katello::Errors::Pulp3Error: column pulp_2to3_migration_pulp2content.pulp2_subid does not exist LINE 1: …ulp_2to3_migration_pulp2content".“pulp2_repo_id”, "pulp_2to3… ^ HINT: Perhaps you meant to reference the column “pulp_2to3_migration_pulp2content.pulp2_id”.

Expected outcome:
Able to disable repository

Foreman and Proxy versions:
Foreman 2.3.2

Foreman and Proxy plugin versions:
Katello 3.18.1

Distribution and version:
CentOS 7.9

Other relevant data:
I expect it’s something I’ve done… again.
I did spot some pulp related selinux failures prior to this when trying to sync repositories and rectified with the following policy:

module pulp-fix 1.0;

require {
        type pulpcore_server_t;
        type pulpcore_t;
        type httpd_sys_rw_content_t;
        type httpd_config_t;
        type http_cache_port_t;
        class tcp_socket name_connect;
        class file { create getattr ioctl open read write };
        class dir { add_name getattr open read search write };
}

#============= pulpcore_server_t ==============
allow pulpcore_server_t httpd_config_t:dir search;
allow pulpcore_server_t httpd_sys_rw_content_t:dir { getattr open read search };

#============= pulpcore_t ==============
allow pulpcore_t http_cache_port_t:tcp_socket name_connect;
allow pulpcore_t httpd_sys_rw_content_t:dir { add_name getattr open read search write };
allow pulpcore_t httpd_sys_rw_content_t:file { create getattr ioctl open read write };
indent preformatted text by 4 spaces

Stack Trace

2021-03-04T09:55:32 [I|app|a622bf6a] Completed 200 OK in 2151ms (Views: 731.0ms | ActiveRecord: 1402.0ms | Allocations: 789674)
2021-03-04T09:55:40 [I|app|3c6942a4] Started GET “/notification_recipients” for 131.185.41.82 at 2021-03-04 09:55:40 +1030
2021-03-04T09:55:40 [I|app|3c6942a4] Processing by NotificationRecipientsController#index as JSON
2021-03-04T09:55:40 [I|app|3c6942a4] Completed 200 OK in 6ms (Views: 0.1ms | ActiveRecord: 0.9ms | Allocations: 1779)
2021-03-04T09:55:45 [I|app|d8b86bb0] Started PUT “/katello/api/v2/products/56/repository_sets/2456/disable” for 131.185.41.82 at 2021-03-04 09:55:45 +1030
2021-03-04T09:55:45 [I|app|d8b86bb0] Processing by Katello::Api::V2::RepositorySetsController#disable as JSON
2021-03-04T09:55:45 [I|app|d8b86bb0] Parameters: {“id”=>“2456”, “product_id”=>“56”, “basearch”=>“x86_64”, “releasever”=>“7.9”, “api_version”=>“v2”, “repository_set”=>{“id”=>“2456”, “product_id”=>“56”, “basearch”=>“x86_64”, “releasever”=>“7.9”}}
2021-03-04T09:55:45 [I|bac|d8b86bb0] Task {label: Actions::Katello::RepositorySet::DisableRepository, id: 59ec8280-bb2c-4cdb-b233-c7f69ac40222, execution_plan_id: 54f5eceb-a5f2-4268-8e2f-96240138b1e5} state changed: planning
2021-03-04T09:55:45 [I|bac|] Task {label: Actions::Katello::RepositorySet::DisableRepository, id: 59ec8280-bb2c-4cdb-b233-c7f69ac40222, execution_plan_id: 54f5eceb-a5f2-4268-8e2f-96240138b1e5} state changed: planned
2021-03-04T09:55:45 [I|bac|] Task {label: Actions::Katello::RepositorySet::DisableRepository, id: 59ec8280-bb2c-4cdb-b233-c7f69ac40222, execution_plan_id: 54f5eceb-a5f2-4268-8e2f-96240138b1e5} state changed: running
2021-03-04T09:55:47 [E|bac|] column pulp_2to3_migration_pulp2content.pulp2_subid does not exist
| LINE 1: …ulp_2to3_migration_pulp2content".“pulp2_repo_id”, "pulp_2to3…
| ^
| HINT: Perhaps you meant to reference the column “pulp_2to3_migration_pulp2content.pulp2_id”.
| (Katello::Errors::Pulp3Error)
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:102:in block in check_for_errors' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:100:in each’
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:100:in check_for_errors' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:133:in poll_external_task’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:100:in poll_external_task_with_rescue' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:22:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/cancellable.rb:14:in run' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/pulp3/abstract_async_task.rb:10:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:571:in block (3 levels) in execute_run' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:16:in block in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:40:in block in as_remote_user' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/models/katello/concerns/user_extensions.rb:21:in cp_config’
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:27:in as_cp_user' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:39:in as_remote_user’
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/lib/actions/middleware/remote_action.rb:16:in run' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/rails_executor_wrap.rb:14:in block in run' | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/execution_wrapper.rb:88:in wrap’
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/rails_executor_wrap.rb:13:in run' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:31:in with_progress_calculation' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:17:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_request_id.rb:15:in block in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_request_id.rb:49:in restore_current_request_id' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_request_id.rb:15:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_timezone.rb:15:in block in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_timezone.rb:44:in restore_curent_timezone' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_timezone.rb:15:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_user.rb:15:in block in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_user.rb:44:in restore_curent_user' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_user.rb:15:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in block in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in restore_current_taxonomies' | /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-3.0.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in pass’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in pass' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in run’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in call' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/world.rb:31:in execute’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:570:in block (2 levels) in execute_run' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in catch’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in block in execute_run' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in block in with_error_handling’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in catch' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in with_error_handling’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:564:in execute_run' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:285:in execute’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in block (2 levels) in execute' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in with_meta_calculation’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in block in execute' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in open_action’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in execute' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/director.rb:93:in execute’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in block (2 levels) in perform' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors.rb:18:in run_user_code’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in block in perform' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in with_telemetry’
| /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in perform' | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in perform’
| [ sidekiq ]
| [ concurrent-ruby ]

2

Are you sure all Pulp migrations are fully applied? “column… does not exist” usually means that your code includes migrations that have not been applied to your DB yet.

Rerunning foreman-installer should ensure all migrations are fully applied.

2 Likes
3

Hi @quba42 ,

Something odd happening. I’m sure I haven’t run any updates so I’m not sure why migrations would be required. You were dead right though. Running the installer fixed it up.

Thanks

Andy

4

It looks like a new version of the pulp_2to3_migration plugin, was made available to the existing version of Foreman/Katello (the existing Foreman/Katello repo). Therefore it is possible to get a new migration just from running yum update on your Foreman host.