I have the same problem in Katello 3.16.2, I have uploaded a foreman-debug in this post -> Katello smart proxy content sync error
Still working on this, will get an update soon. Sorry for the delay.
2 Likes
Hi all,
Not trying to put pressure on the issue or anything like that, but I believe Iām experiencing this same challenge and would like to offer any information I can provide to help.
The only difference I can see (if Iām properly understanding this thread thus far) however is CentOS8 or its AppStream repositories are not part of the equation in my situation.
Katello Server Packages:
# rpm -qa --qf '%{VERSION}\t%{RELEASE}\t%{NAME}\n' centos-release {foreman,katello}\* | sort | grep -v ca-consumer
1.0 1 katello-default-ca
1.0 1 katello-server-ca
1.7.5 1.el7 katello-client-bootstrap
2.1.4 1.el7 foreman
2.1.4 1.el7 foreman-cli
2.1.4 1.el7 foreman-debug
2.1.4 1.el7 foreman-dynflow-sidekiq
2.1.4 1.el7 foreman-installer
2.1.4 1.el7 foreman-installer-katello
2.1.4 1.el7 foreman-postgresql
2.1.4 1.el7 foreman-proxy
2.1.4 1.el7 foreman-release
2.1.4 1.el7 foreman-selinux
2.1.4 1.el7 foreman-service
2.7.0 1.el7 katello-certs-tools
3.16.2 1.el7 katello
3.16.2 1.el7 katello-common
3.16.2 1.el7 katello-debug
3.16.2 2.el7 katello-repos
3.4.0 1.el7 katello-selinux
3.5.4 1.el7 katello-agent
3.5.4 1.el7 katello-host-tools
3.5.4 1.el7 katello-host-tools-fact-plugin
7 9.2009.1.el7.centos centos-release
SmartProxy Packages:
# rpm -qa --qf '%{VERSION}\t%{RELEASE}\t%{NAME}\n' centos-release {foreman,katello}\* | sort | grep -v ca-consumer
1.0 1 katello-default-ca
1.0 1 katello-server-ca
1.7.5 1.el7 katello-client-bootstrap
2.1.4 1.el7 foreman-debug
2.1.4 1.el7 foreman-installer
2.1.4 1.el7 foreman-installer-katello
2.1.4 1.el7 foreman-proxy
2.1.4 1.el7 foreman-release
2.7.0 1.el7 katello-certs-tools
3.16.2 1.el7 foreman-proxy-content
3.16.2 1.el7 katello-common
3.16.2 1.el7 katello-debug
3.16.2 2.el7 katello-repos
3.5.4 1.el7 katello-agent
3.5.4 1.el7 katello-host-tools
3.5.4 1.el7 katello-host-tools-fact-plugin
7 9.2009.1.el7.centos centos-release
I actually just stood this instance up in my lab. I created a product with the CentOS7 āOSā, āUpdatesā, and āExtrasā repositories:
# hammer product info --id 1 --organization "MyOrg"
ID: 1
Name: CentOS 7 Linux for x86_64
Label: CentOS_7_Linux_for_x86_64
Description:
Sync State (all): Syncing Complete.
Sync State (last): Syncing Complete.
Sync Plan ID:
GPG:
GPG Key ID:
GPG Key:
Organization: MyOrg
Content:
1) Repo Name: CentOS7-Extras-x86_64
URL: /custom/CentOS_7_Linux_for_x86_64/CentOS7-Extras-x86_64
Content Type: yum
2) Repo Name: CentOS7-Foreman-Client-2.1-x86_64
URL: /custom/CentOS_7_Linux_for_x86_64/CentOS7-Foreman-Client-2_1-x86_64
Content Type: yum
3) Repo Name: CentOS7-OS-x86_64
URL: /custom/CentOS_7_Linux_for_x86_64/CentOS7-OS-x86_64
Content Type: yum
4) Repo Name: CentOS7-Updates-x86_64
URL: /custom/CentOS_7_Linux_for_x86_64/CentOS7-Updates-x86_64
Content Type: yum
And then created two content-views out of those repositories. Published and promoted.
# hammer content-view info --id 3
ID: 3
Name: Server
Label: Server
Composite: false
Description:
Content Host Count: 1
Force Puppet: false
Solve Dependencies: false
Organization: MyOrg
Yum Repositories:
1) ID: 4
Name: CentOS7-Updates-x86_64
Label: CentOS7-Updates-x86_64
2) ID: 1
Name: CentOS7-Extras-x86_64
Label: CentOS7-Extras-x86_64
3) ID: 2
Name: CentOS7-Foreman-Client-2.1-x86_64
Label: CentOS7-Foreman-Client-2_1-x86_64
4) ID: 3
Name: CentOS7-OS-x86_64
Label: CentOS7-OS-x86_64
Container Image Repositories:
OSTree Repositories:
Puppet Modules:
Lifecycle Environments:
1) ID: 2
Name: Library
2) ID: 3
Name: DEV
Versions:
1) ID: 3
Version: 1.0
Published: 2020/12/17 15:59:51
2) ID: 5
Version: 2.0
Published: 2020/12/17 21:05:59
Components:
Activation Keys:
1) centos_7_server_key_dev
I then assigned this content to my SmartProxy:
# hammer proxy content info --id 2
Lifecycle Environments:
1) Name: DEV
Organization: MyOrg
Content Views:
1) Name: Server
Composite: no
Last Published: 2020/12/17 21:05:59
Content:
Hosts: 1
Products: 1
Yum repos: 4
Container Image repos: 0
Packages: 11312
Package groups: 1
Errata: 0
Puppet modules: 0
I then try to synchronize this data, and of course this is where I hit the road block. Below is taken from production.log upon the failure:
2020-12-18T13:08:42 [I|app|02557b9c] Started POST "/katello/api/capsules/2-ods00298w10-example-net/content/sync" for 127.0.0.1 at 2020-12-18 13:08:42 +0000
2020-12-18T13:08:42 [I|app|02557b9c] Processing by Katello::Api::V2::CapsuleContentController#sync as HTML
2020-12-18T13:08:42 [I|app|02557b9c] Parameters: {"id"=>"2-ods00298w10-example-net", "skip_metadata_check"=>false, "api_version"=>"v2", "capsule_content"=>{"id"=>"2-ods00298w10-example-net", "skip_metadata_check"=>false}}
2020-12-18T13:08:43 [I|bac|02557b9c] Task {label: Actions::Katello::CapsuleContent::Sync, id: 5e5e6043-b01c-4aab-af6c-95ec05012563, execution_plan_id: 432f95a7-8a48-4d2e-88cf-48b26b906538} state changed: planning
2020-12-18T13:08:44 [I|bac|] Task {label: Actions::Katello::CapsuleContent::Sync, id: 5e5e6043-b01c-4aab-af6c-95ec05012563, execution_plan_id: 432f95a7-8a48-4d2e-88cf-48b26b906538} state changed: planned
2020-12-18T13:08:44 [I|app|] Rendering /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/views/katello/api/v2/common/async.json.rabl within katello/api/v2/layouts/resource
2020-12-18T13:08:44 [I|app|] Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/views/katello/api/v2/common/async.json.rabl within katello/api/v2/layouts/resource (Duration: 51.4ms | Allocations: 13921)
2020-12-18T13:08:44 [I|app|] Completed 202 Accepted in 1404ms (Views: 51.4ms | ActiveRecord: 258.7ms | Allocations: 146479)
2020-12-18T13:08:44 [I|bac|] Task {label: Actions::Katello::CapsuleContent::Sync, id: 5e5e6043-b01c-4aab-af6c-95ec05012563, execution_plan_id: 432f95a7-8a48-4d2e-88cf-48b26b906538} state changed: running
2020-12-18T13:12:48 [E|bac|] PLP0000: Importer indicated a failed response (Katello::Errors::PulpError)
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/pulp/abstract_async_task.rb:119:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/pulp/abstract_async_task.rb:119:in `external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:100:in `poll_external_task_with_rescue'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:22:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/cancellable.rb:14:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/pulp/abstract_async_task.rb:45:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:571:in `block (3 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/execution_wrapper.rb:88:in `wrap'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:17:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_request_id.rb:49:in `restore_current_request_id'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_user.rb:44:in `restore_curent_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/world.rb:31:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:570:in `block (2 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in `block in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:564:in `execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:285:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/director.rb:93:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors.rb:18:in `run_user_code'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
[ sidekiq ]
[ concurrent-ruby ]
2020-12-18T13:12:48 [E|bac|] PLP0000: Importer indicated a failed response (Katello::Errors::PulpError)
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/pulp/abstract_async_task.rb:119:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/pulp/abstract_async_task.rb:119:in `external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:100:in `poll_external_task_with_rescue'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/polling.rb:22:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/cancellable.rb:14:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/pulp/abstract_async_task.rb:45:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:571:in `block (3 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.2/app/lib/actions/middleware/remote_action.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/execution_wrapper.rb:88:in `wrap'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action/progress.rb:17:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_request_id.rb:49:in `restore_current_request_id'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_user.rb:44:in `restore_curent_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:19:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware.rb:32:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/stack.rb:23:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/middleware/world.rb:31:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:570:in `block (2 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:569:in `block in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:472:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:564:in `execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:285:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/director.rb:93:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors.rb:18:in `run_user_code'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
[ sidekiq ]
[ concurrent-ruby ]
2020-12-18T13:12:48 [I|bac|] Task {label: Actions::Katello::CapsuleContent::Sync, id: 5e5e6043-b01c-4aab-af6c-95ec05012563, execution_plan_id: 432f95a7-8a48-4d2e-88cf-48b26b906538} state changed: stopped result: warning
2020-12-18T13:12:48 [I|bac|] Task {label: Actions::Katello::CapsuleContent::Sync, id: 5e5e6043-b01c-4aab-af6c-95ec05012563, execution_plan_id: 432f95a7-8a48-4d2e-88cf-48b26b906538} state changed: stopped result: warning
And also from the SmartProxy syslog:
Dec 18 13:08:53 ods00298w10 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[63eaa0e3-e9c0-4031-b8cc-fc95af75afe5]
Dec 18 13:08:53 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Downloading metadata from https://hqret-foreman.example.net/pulp/repos/MyORG/DEV/Store_Server/custom/CentOS_7_Linux_for_x86_64/CentOS7-OS-x86_64/.
Dec 18 13:08:55 ods00298w10 pulp: urllib3.connectionpool:INFO: [63eaa0e3] Starting new HTTPS connection (1): hqret-foreman.example.net
Dec 18 13:08:56 ods00298w10 pulp: nectar.downloaders.threaded:INFO: [63eaa0e3] Download succeeded: https://hqret-foreman.example.net/pulp/repos/MyORG/DEV/Store_Server/custom/CentOS_7_Linux_for_x86_64/CentOS7-OS-x86_64/.
Dec 18 13:08:56 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Downloading metadata from https://hqret-foreman.example.net/pulp/repos/MyORG/DEV/Store_Server/custom/CentOS_7_Linux_for_x86_64/CentOS7-OS-x86_64/.
Dec 18 13:08:57 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Parsing metadata.
Dec 18 13:08:57 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Downloading metadata from https://hqret-foreman.example.net/pulp/repos/MyORG/DEV/Store_Server/custom/CentOS_7_Linux_for_x86_64/CentOS7-OS-x86_64/.
Dec 18 13:08:58 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Parsing metadata.
Dec 18 13:08:58 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Downloading metadata from https://hqret-foreman.example.net/pulp/repos/MyORG/DEV/Store_Server/custom/CentOS_7_Linux_for_x86_64/CentOS7-OS-x86_64/.
Dec 18 13:08:59 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Parsing metadata.
Dec 18 13:08:59 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Downloading metadata files.
Dec 18 13:09:03 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Generating metadata databases.
Dec 18 13:09:41 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: [63eaa0e3] Determining which units need to be downloaded.
Dec 18 13:11:56 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.purge:INFO: [63eaa0e3] Removing missing units.
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) 'NoneType' object has no attribute 'findall'
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) Traceback (most recent call last):
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 294, in run
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) self.update_content(metadata_files, url)
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 627, in update_content
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) purge.purge_unwanted_units(metadata_files, self.conduit, self.config, catalog)
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/purge.py", line 46, in purge_unwanted_units
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) remove_missing_groups(metadata_files, conduit)
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/purge.py", line 165, in remove_missing_groups
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) remote_named_tuples = get_remote_units(file_function, group.GROUP_TAG, process_func)
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/purge.py", line 277, in get_remote_units
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) for unit in package_info_generator:
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", line 64, in package_list_generator
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) package_info = process_func(element)
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/group.py", line 32, in process_group_element
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) packagelist.findall('packagereq'))
Dec 18 13:12:31 ods00298w10 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [63eaa0e3] (4172-32000) AttributeError: 'NoneType' object has no attribute 'findall'
Dec 18 13:12:31 ods00298w10 pulp: pulp.server.async.tasks:INFO: [63eaa0e3] Task failed : [63eaa0e3-e9c0-4031-b8cc-fc95af75afe5]
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) Task pulp.server.managers.repo.sync.sync[63eaa0e3-e9c0-4031-b8cc-fc95af75afe5] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) Traceback (most recent call last):
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) R = retval = fun(*args, **kwargs)
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 688, in __call__
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) return super(Task, self).__call__(*args, **kwargs)
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 110, in __call__
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) return super(PulpTask, self).__call__(*args, **kwargs)
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) return self.run(*args, **kwargs)
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 854, in sync
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Dec 18 13:12:31 ods00298w10 pulp: celery.app.trace:ERROR: [63eaa0e3] (4172-32000) PulpExecutionException: Importer indicated a failed response
It was really this SmartProxy log that made me think this is perhaps related to this reported issue.
Finally, overall health of system seems to be okay, according to hammer ping:
# hammer ping
database:
Status: ok
Server Response: Duration: 0ms
candlepin:
Status: ok
Server Response: Duration: 30ms
candlepin_events:
Status: ok
message: 1 Processed, 0 Failed
Server Response: Duration: 0ms
candlepin_auth:
Status: ok
Server Response: Duration: 25ms
katello_events:
Status: ok
message: 0 Processed, 0 Failed
Server Response: Duration: 0ms
pulp:
Status: ok
Server Response: Duration: 565ms
pulp_auth:
Status: ok
Server Response: Duration: 71ms
foreman_tasks:
Status: ok
Server Response: Duration: 3ms
Please let me know if anything else I can provide that may assist with the troubleshooting effort.
1 Like
Hi guys,
Sorry we had our holiday shutdown right after this issue got created. We are back in the office now, I will resume working on this.
3 Likes
Got the Katello box stood up and synced the same repos you did, going to get the capsule installed and sync over the content view that I made that is similar to yours and hope to replicate the issue.
Here is my content view:
Here are the repos I have synced:
2 Likes
Looks good, I have posted some stacktraces and extra info in this thread. Bug: Smart Proxies do not sync (Katello 3.15 through 3.18 RC2)
That should point you at some some files to look at and what to look for 
I was able to reproduce the issue, here is the error from the smart proxy logs:
Jan 5 17:05:14 capsule pulp: pulp.server.content.sources.event:ERROR: (11650-70432) error: error reading package header
Jan 5 17:10:05 capsule pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Jan 5 17:10:05 capsule rsyslogd: imjournal: 21955 messages lost due to rate-limiting
Jan 5 17:10:05 capsule pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[cbdabb87-ab2a-4243-b363-501c690043ef]
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) 'NoneType' object has no attribute 'findall'
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) Traceback (most recent call last):
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 294, in run
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) self.update_content(metadata_files, url)
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 627, in update_content
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) purge.purge_unwanted_units(metadata_files, self.conduit, self.config, catalog)
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/purge.py", line 46, in purge_unwanted_units
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) remove_missing_groups(metadata_files, conduit)
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/purge.py", line 165, in remove_missing_groups
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) remote_named_tuples = get_remote_units(file_function, group.GROUP_TAG, process_func)
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/purge.py", line 277, in get_remote_units
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) for unit in package_info_generator:
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", line 64, in package_list_generator
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) package_info = process_func(element)
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/group.py", line 32, in process_group_element
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) packagelist.findall('packagereq'))
Jan 5 17:10:09 capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [090a5d20] (11650-49792) AttributeError: 'NoneType' object has no attribute 'findall'
Jan 5 17:10:10 capsule pulp: pulp.server.async.tasks:INFO: [090a5d20] Task failed : [090a5d20-97fa-4a2c-8ba2-f729683684bc]
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) Task pulp.server.managers.repo.sync.sync[090a5d20-97fa-4a2c-8ba2-f729683684bc] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) Traceback (most recent call last):
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 367, in trace_task
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) R = retval = fun(*args, **kwargs)
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 688, in __call__
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) return super(Task, self).__call__(*args, **kwargs)
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 110, in __call__
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) return super(PulpTask, self).__call__(*args, **kwargs)
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 622, in __protected_call__
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) return self.run(*args, **kwargs)
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 854, in sync
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Jan 5 17:10:11 capsule pulp: celery.app.trace:ERROR: [090a5d20] (11650-49792) PulpExecutionException: Importer indicated a failed response
Here is a screenshot of the failed task when trying to sync the same repos you have in your content view:
Let me know if that looks the same as far as the task page goes.
If so I will reach out to the @Pulp team and see if they have any insights as to what is going on as I look from the Katello side.
Chris
2 Likes
I am very happy you could reproduce the issue. Just as a reminder, my error message is slightly different, it is in the phase āRemoving missing unitsā.
Thank you! This is exactly the message I receive so itās a reproducable problem, thatās good!
I also wrote a hacky patch to workaround the issue:
Glad this is the same issue you are hitting.
@Pulp What do you think of this fix, will this break anything long term? Since the change is in Pulp I am not sure if there if a fix we can do in Katello. @Justin_Sherrill is this something that we can do a fix in Katello for is or this all in Pulp?
Please open a bug in the upstream pulp tracker. There is something fishy ongoing with the remove_missing logic. I am afraid adding a try/except might bubble up in some other place with incorrect logic.
If an issue was created in the upstream pulp bug tracker, would someone post the link to it here? Thanks.
Likewise, if Iām the first one to open the issue, Iāll do the same.
Hi guys,
Did one of you already log a bug in the Pulp upstream tracker? I donāt have an account there
Cheers!
I havenāt. Iām still in the process of familiarizing myself with their project and their issue tracker.
Right now I donāt feel like Iām even in a knowledgeable enough position to create an issue with the right information in it.
Itās even quite possible an issue already exists that addresses our problem, I just wouldnāt know it if I saw it.
@jjeffers, that would be awesome. Youāre definitely more familiar with the Pulp codebase than I am. And as a Katello dev, your word probably carries more weight.
If you create the issue, would you drop a link to it here? I can certainly follow it and chime in with additional data if necessary.
https://pulp.plan.io/issues/8176
As far as I know the Pulp team is receptive to users adding new issues, and they do a good job with issue triage. They take all input seriously, so donāt be shy.
1 Like
Awesome, thanks for picking this up, I will create an account for any future things
3 Likes
Just wondering if we have found a work around for this? Getting the same thing for my CentOS repositories. Here I was thinking it was something that I did that my Smart Proxy would not sync.
Thanks!