Katello 4.2.0.pre.master: content sync from katello server to proxy fails

yaser · July 1, 2021, 11:31pm

Problem:
trying to sync contents of proxy server with katello, it always fails with these errors in task output, there are some different errors also in proxy itself which will be posted here:

Actions::Pulp3::Orchestration::Repository::RefreshRepos
Input:
{“smart_proxy_id”=>2,
“environment_id”=>nil,
“content_view_id”=>nil,
“repository_id”=>nil,
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{}
Exception:
PulpDebClient::ApiError: Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Thu, 01 Jul 2021 23:11:16 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“GET, POST, HEAD, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“45”, “correlation-id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”, “access-control-expose-headers”=>“Correlation-ID”, “via”=>"1.1 sydinftst04, “connection”=>“close”}
Response body: {“distributions”:[“This field is required.”]}
Backtrace:
/opt/theforeman/tfm/root/usr/share/gems/gems/pulp_deb_client-2.11.1/lib/pulp_deb_client/api_client.rb:81:in call_api' /opt/theforeman/tfm/root/usr/share/gems/gems/pulp_deb_client-2.11.1/lib/pulp_deb_client/api/remotes_apt_api.rb:79:in create_with_http_info’
/opt/theforeman/tfm/root/usr/share/gems/gems/pulp_deb_client-2.11.1/lib/pulp_deb_client/api/remotes_apt_api.rb:28:in create' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.pre.master/app/services/katello/pulp3/repository_mirror.rb:103:in create_remote’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.pre.master/app/services/katello/pulp3/repository_mirror.rb:29:in refresh_entities' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.pre.master/app/services/katello/pulp3/repository.rb:118:in refresh_mirror_entities’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.pre.master/app/lib/actions/katello/capsule_content/refresh_repos.rb:44:in `block in invoke_external_task’
Actions::Pulp3::CapsuleContent::Sync
Input:
{“repository_id”=>1,
“smart_proxy_id”=>2,
“options”=>{“skip_metadata_check”=>false},
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{“smart_proxy_history_id”=>45}
Exception:
ArgumentError: Missing the required parameter ‘rpm_rpm_repository_href’ when calling RepositoriesRpmApi.sync

Actions::Pulp3::CapsuleContent::GenerateMetadata
Input:
{“repository_id”=>1,
“smart_proxy_id”=>2,
“options”=>{“skip_metadata_check”=>false, “contents_changed”=>true},
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{}
Exception:
NoMethodError: undefined method `latest_version_href’ for nil:NilClass
Actions::Pulp3::CapsuleContent::RefreshDistribution
Input:
{“repository_id”=>1,
“smart_proxy_id”=>2,
“options”=>
{“tasks”=>Step(9).output,
“use_repository_version”=>false,
“contents_changed”=>true},
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{}
Action:
Actions::Pulp3::CapsuleContent::Sync
Input:
{“repository_id”=>3,
“smart_proxy_id”=>2,
“options”=>{“skip_metadata_check”=>false},
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{“smart_proxy_history_id”=>44}
Exception:
PulpDebClient::ApiError: Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Thu, 01 Jul 2021 23:11:17 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“POST, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“82”, “correlation-id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”, “access-control-expose-headers”=>“Correlation-ID”, “via”=>“1.1 sydinftst04.”, “connection”=>“close”}
Response body: {“remote”:[“This field is required since a remote is not set on the repository.”]}

Actions::Pulp3::CapsuleContent::GenerateMetadata
Input:
{“repository_id”=>3,
“smart_proxy_id”=>2,
“options”=>{“skip_metadata_check”=>false, “contents_changed”=>true},
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{}
Exception:
PulpDebClient::ApiError: Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Thu, 01 Jul 2021 23:11:17 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“GET, POST, HEAD, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“85”, “correlation-id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”, “access-control-expose-headers”=>“Correlation-ID”, “via”=>“1.1 sydinftst04”, “connection”=>“close”}
Response body: {“non_field_errors”:[“one of simple or structured publishing mode must be selected”]}

Actions::Pulp3::CapsuleContent::RefreshDistribution
Input:
{“repository_id”=>3,
“smart_proxy_id”=>2,
“options”=>
{“tasks”=>Step(15).output,
“use_repository_version”=>false,
“contents_changed”=>true},
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{}
Action:
Actions::Pulp3::CapsuleContent::Sync
Input:
{“repository_id”=>9,
“smart_proxy_id”=>2,
“options”=>{“skip_metadata_check”=>false},
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{“smart_proxy_history_id”=>47}
Exception:
ArgumentError: Missing the required parameter ‘rpm_rpm_repository_href’ when calling RepositoriesRpmApi.sync

Actions::Pulp3::CapsuleContent::GenerateMetadata
Input:
{“repository_id”=>9,
“smart_proxy_id”=>2,
“options”=>{“skip_metadata_check”=>false, “contents_changed”=>true},
“remote_user”=>“admin”,
“remote_cp_user”=>“admin”,
“current_request_id”=>“a2160e2a-f4b1-4388-ab1b-7c84d254e236”,
“current_timezone”=>“Australia/Sydney”,
“current_organization_id”=>1,
“current_location_id”=>2,
“current_user_id”=>4}
Output:
{}
Exception:
NoMethodError: undefined method `latest_version_href’ for nil:NilClass

THE ERROR IN FOREMAN PROXY:

Jul 2 09:06:37 sydinftst04 smart-proxy[998]: 10.69.101.8 - - [02/Jul/2021:09:06:37 AEST] “GET /puppet/ca HTTP/1.1” 200 250
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: - → /puppet/ca
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: 10.69.101.8 - - [02/Jul/2021:09:06:37 AEST] “GET /puppet/environments/common/classes HTTP/1.1” 200 2
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: - → /puppet/environments/common/classes
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: 10.69.101.8 - - [02/Jul/2021:09:06:37 AEST] “GET /puppet/ca/autosign HTTP/1.1” 200 2
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: - → /puppet/ca/autosign
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: 10.69.101.8 - - [02/Jul/2021:09:06:37 AEST] “GET /puppet/ca HTTP/1.1” 200 250
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: - → /puppet/ca
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: 10.69.101.8 - - [02/Jul/2021:09:06:37 AEST] “GET /puppet/ca/autosign HTTP/1.1” 200 2
Jul 2 09:06:37 sydinftst04 smart-proxy[998]: - → /puppet/ca/autosign
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:48 +0000] “GET /pulp/api/v3/status HTTP/1.1” 301 0 “-” “rest-client/2.0.2 (linux-gnu x86_64) ruby/2.7.1p83”
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:48 +0000] “GET /pulp/api/v3/status/ HTTP/1.1” 200 1483 “-” “rest-client/2.0.2 (linux-gnu x86_64) ruby/2.7.1p83”
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:48 +0000] “GET /pulp/api/v3/contentguards/certguard/rhsm/ HTTP/1.1” 200 2780 “-” “OpenAPI-Generator/1.0.3/ruby”
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: django.request:ERROR: Internal Server Error: /pulp/api/v3/contentguards/certguard/rhsm/
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: Traceback (most recent call last):
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return self.cursor.execute(sql, params)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint “core_contentguard_name_key”
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: DETAIL: Key (name)=(RHSMCertGuard) already exists.
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: The above exception was the direct cause of the following exception:
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: Traceback (most recent call last):
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/core/handlers/exception.py”, line 34, in inner
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: response = get_response(request)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/core/handlers/base.py”, line 115, in _get_response
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: response = self.process_exception_by_middleware(e, request)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/core/handlers/base.py”, line 113, in _get_response
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: response = wrapped_callback(request, *callback_args, **callback_kwargs)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/views/decorators/csrf.py”, line 54, in wrapped_view
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return view_func(*args, **kwargs)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/viewsets.py”, line 125, in view
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return self.dispatch(request, *args, **kwargs)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/views.py”, line 509, in dispatch
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: response = self.handle_exception(exc)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/views.py”, line 469, in handle_exception
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: self.raise_uncaught_exception(exc)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/views.py”, line 480, in raise_uncaught_exception
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: raise exc
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/views.py”, line 506, in dispatch
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: response = handler(request, *args, **kwargs)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/mixins.py”, line 19, in create
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: self.perform_create(serializer)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/mixins.py”, line 24, in perform_create
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: serializer.save()
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/serializers.py”, line 205, in save
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: self.instance = self.create(validated_data)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/pulpcore/app/serializers/base.py”, line 157, in create
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: instance = super().create(validated_data)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/rest_framework/serializers.py”, line 939, in create
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: instance = ModelClass._default_manager.create(**validated_data)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 422, in create
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: obj.save(force_insert=True, using=self.db)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/pulpcore/app/models/base.py”, line 149, in save
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return super().save(*args, **kwargs)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py”, line 134, in save
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: save(*args, **kwargs)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 744, in save
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: force_update=force_update, update_fields=update_fields)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 779, in save_base
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: parent_inserted = self._save_parents(cls, using, update_fields)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 810, in _save_parents
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: force_insert=parent_inserted,
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 873, in _save_table
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/base.py”, line 911, in _do_insert
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: using=using, raw=raw)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/manager.py”, line 82, in manager_method
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return getattr(self.get_queryset(), name)(*args, **kwargs)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/query.py”, line 1186, in _insert
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return query.get_compiler(using=using).execute_sql(return_id)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/models/sql/compiler.py”, line 1377, in execute_sql
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: cursor.execute(sql, params)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 67, in execute
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 76, in _execute_with_wrappers
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return executor(sql, params, many, context)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return self.cursor.execute(sql, params)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/utils.py”, line 89, in exit
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: raise dj_exc_value.with_traceback(traceback) from exc_value
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: File “/usr/lib/python3.6/site-packages/django/db/backends/utils.py”, line 84, in _execute
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return self.cursor.execute(sql, params)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: return self.cursor.execute(sql, params)
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: django.db.utils.IntegrityError: duplicate key value violates unique constraint “core_contentguard_name_key”
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: DETAIL: Key (name)=(RHSMCertGuard) already exists.
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:48 +0000] “POST /pulp/api/v3/contentguards/certguard/rhsm/ HTTP/1.1” 500 27 “-” “OpenAPI-Generator/1.0.3/ruby”
Jul 2 09:06:48 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:48 +0000] “GET /pulp/api/v3/contentguards/certguard/rhsm/ HTTP/1.1” 200 2780 “-” “OpenAPI-Generator/1.0.3/ruby”
Jul 2 09:06:49 sydinftst04 smart-proxy[998]: 10.69.101.8 - - [02/Jul/2021:09:06:49 AEST] “PUT /container_gateway/repository_list/ HTTP/1.1” 200 0
Jul 2 09:06:49 sydinftst04 smart-proxy[998]: - → /container_gateway/repository_list/
Jul 2 09:06:49 sydinftst04 smart-proxy[998]: 10.69.101.8 - - [02/Jul/2021:09:06:49 AEST] “GET /container_gateway/users HTTP/1.1” 200 12
Jul 2 09:06:49 sydinftst04 smart-proxy[998]: - → /container_gateway/users
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:50 +0000] “GET /pulp/api/v3/repositories/ansible/ansible/?limit=2000&offset=0 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/0.7.1/ruby”
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:50 +0000] “GET /pulp/api/v3/repositories/deb/apt/?limit=2000&offset=0 HTTP/1.1” 200 486 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:50 +0000] “GET /pulp/api/v3/repositories/container/container/?limit=2000&offset=0 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/2.4.0/ruby”
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:50 +0000] “GET /pulp/api/v3/repositories/file/file/?limit=2000&offset=0 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/1.6.1/ruby”
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:50 +0000] “GET /pulp/api/v3/repositories/rpm/rpm/?limit=2000&offset=0 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/3.10.0/ruby”
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:50 +0000] “GET /pulp/api/v3/remotes/deb/apt/?name=1087b477-4558-4d3f-8a0a-bb2ad5d1fa57 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:50 +0000] “GET /pulp/api/v3/remotes/deb/apt/?name=1087b477-4558-4d3f-8a0a-bb2ad5d1fa57 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: django.request:WARNING: Bad Request: /pulp/api/v3/remotes/deb/apt/
Jul 2 09:06:50 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:50 +0000] “POST /pulp/api/v3/remotes/deb/apt/ HTTP/1.1” 400 45 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 2 09:06:51 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:51 +0000] “GET /pulp/api/v3/remotes/rpm/rpm/?name=1-Centos_7-DEV-b7b8a03b-345b-4a04-9d6d-1368f710a489 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/3.10.0/ruby”
Jul 2 09:06:51 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:51 +0000] “GET /pulp/api/v3/remotes/deb/apt/?name=1087b477-4558-4d3f-8a0a-bb2ad5d1fa57 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 2 09:06:51 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:51 +0000] “GET /pulp/api/v3/remotes/rpm/rpm/?name=b7b8a03b-345b-4a04-9d6d-1368f710a489 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/3.10.0/ruby”
Jul 2 09:06:51 sydinftst04 pulpcore-api[1000]: pulp [8202eab5-7889-45a8-91b9-74bfb6d515cc]: - - [01/Jul/2021:23:06:51 +0000] “GET /pulp/api/v3/repositories/rpm/rpm/?limit=2000&name=1-Centos_7-DEV-b7b8a03b-345b-4a04-9d6d-1368f710a489&offset=0 HTTP/1.1” 200 52 “-” "

Expected outcome:
data from katello server gets synced to proxy server

Foreman and Proxy versions:
Name Features Status Version
sydinftst01 Pulpcore, Puppet CA, Puppet, Logs, and Registration 2.6.0-develop
sydinftst04 Container_Gateway, Pulpcore, Templates, Puppet CA, Puppet, Logs, and Registration 2.6.0-develop

server:
foreman-tasks 5.0.0
foreman_ansible 6.4.1
foreman_bootdisk 17.1.0
foreman_discovery 17.0.0
foreman_hooks 0.3.17
foreman_monitoring 2.1.0
foreman_openscap 4.3.2
foreman_remote_execution 4.6.0
foreman_setup 7.0.0
foreman_templates 9.1.0
katello 4.2.0.pre.master
Pulpcore 3.1.0
Supported Content Types

ansible_collection
deb
docker
file
yum

Registration

Version

2.6.0

###proxy version###

Container_Gateway

Version

1.0.5

Pulpcore

Version

3.1.0

Supported Content Types

ansible_collection
deb
docker
file
yum

Registration

Version

2.6.0

Templates

Version

2.6.0

Foreman and Proxy plugin versions:

Distribution and version:
centos7. CentOS Linux release 7.9.2009 (Core)
Other relevant data:

yaser · July 5, 2021, 4:09am

i have downgraded katello 4.2 to 4.1 (foreman 2.5.1, pulpcore 3.0.0) in both katello and foreman proxy, then removed U18 repo to see if it’s working with only centos repos, now it’s doing something (i can see artefact created in /var/lib/pulp of proxy server) but at the end it’s still failing:

Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: pulp [8f3f8d5c-bae0-4f0e-89c4-ec16be24b0dc]: rq.worker:ERROR: Traceback (most recent call last):
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 975, in perform_job
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: rv = job.perform()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 696, in perform
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: self._result = self._execute()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 719, in _execute
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: return self.func(*self.args, **self.kwargs)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: dv.create()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 147, in create
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: loop.run_until_complete(pipeline)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: return future.result()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await asyncio.gather(*futures)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in call
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await self.run()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 441, in run
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await self.parse_content()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 540, in parse_content
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await self.parse_packages(await asyncio.gather(*package_repodata_downloaders))
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 561, in parse_packages
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await self._parse_packages(packages)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 582, in parse_packages
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: (, pkg) = packages.popitem(last=False)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: TypeError: popitem() takes no keyword arguments
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: Traceback (most recent call last):
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 975, in perform_job
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: rv = job.perform()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 696, in perform
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: self._result = self._execute()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/rq/job.py”, line 719, in _execute
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: return self.func(*self.args, **self.kwargs)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: dv.create()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 147, in create
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: loop.run_until_complete(pipeline)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: return future.result()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await asyncio.gather(*futures)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in call
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await self.run()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 441, in run
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await self.parse_content()
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 540, in parse_content
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await self.parse_packages(await asyncio.gather(*package_repodata_downloaders))
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 561, in parse_packages
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: await self._parse_packages(packages)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 582, in parse_packages
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: (, pkg) = packages.popitem(last=False)
Jul 5 14:01:20 sydinftst04 pulpcore-worker-1[69004]: TypeError: popitem() takes no keyword arguments

yaser · July 5, 2021, 6:36am

i’ve done some testings in katello 4.1 (foreman 2.51, pulpcore 3.0.0). Centos repos are working fine and i could synchronise centos7/8 repos to foreman-proxy. but testing Ubuntu18 main component failed with below error:

Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Mon, 05 Jul 2021 05:41:48 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“GET, POST, HEAD, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“45”, “correlation-id”=>“994bc74b-c89d-4724-96b6-eefeadbd9a05”, “access-control-expose-headers”=>“Correlation-ID”, “via”=>“1.1 sydinftst04.”, “connection”=>“close”}
Response body: {“distributions”:[“This field is required.”]}Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Mon, 05 Jul 2021 05:41:49 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“POST, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“82”, “correlation-id”=>“994bc74b-c89d-4724-96b6-eefeadbd9a05”, “access-control-expose-headers”=>“Correlation-ID”, “via”=>“1.1 sydinftst04”, “connection”=>“close”}
Response body: {“remote”:[“This field is required since a remote is not set on the repository.”]}Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Mon, 05 Jul 2021 05:41:49 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“GET, POST, HEAD, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“85”, “correlation-id”=>“994bc74b-c89d-4724-96b6-eefeadbd9a05”, “access-control-expose-headers”=>“Correlation-ID”, “via”=>“1.1 sydinftst04”, “connection”=>“close”}
Response body: {“non_field_errors”:[“one of simple or structured publishing mode must be selected”]}

errors in proxy:

Jul 5 15:41:47 sydinftst04 smart-proxy[69434]: - → /container_gateway/users
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “GET /pulp/api/v3/contentguards/certguard/rhsm/?name=RHSMCertGuard HTTP/1.1” 200 2780 “-” “OpenAPI-Generator/1.0.3/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “PATCH /pulp/api/v3/contentguards/certguard/rhsm/32f3e02d-a7b0-41c1-a7fc-2ee40606e566/ HTTP/1.1” 200 2728 “-” “OpenAPI-Generator/1.0.3/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “GET /pulp/api/v3/repositories/ansible/ansible/?limit=2000&offset=0 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/0.7.3/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “GET /pulp/api/v3/repositories/deb/apt/?limit=2000&offset=0 HTTP/1.1” 200 934 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “GET /pulp/api/v3/repositories/container/container/?limit=2000&offset=0 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/2.4.0/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “GET /pulp/api/v3/repositories/file/file/?limit=2000&offset=0 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/1.6.1/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “GET /pulp/api/v3/repositories/rpm/rpm/?limit=2000&offset=0 HTTP/1.1” 200 3072 “-” “OpenAPI-Generator/3.10.0/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “GET /pulp/api/v3/remotes/deb/apt/?name=1-U18_CV-DEV-6bb564a7-d315-43b1-8f4e-b3c12cded986 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “GET /pulp/api/v3/remotes/deb/apt/?name=1-U18_CV-DEV-6bb564a7-d315-43b1-8f4e-b3c12cded986 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: django.request:WARNING: Bad Request: /pulp/api/v3/remotes/deb/apt/
Jul 5 15:41:48 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:48 +0000] “POST /pulp/api/v3/remotes/deb/apt/ HTTP/1.1” 400 45 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 5 15:41:49 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:49 +0000] “GET /pulp/api/v3/remotes/deb/apt/?name=1-U18_CV-DEV-6bb564a7-d315-43b1-8f4e-b3c12cded986 HTTP/1.1” 200 52 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 5 15:41:49 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:49 +0000] “GET /pulp/api/v3/repositories/deb/apt/?limit=2000&name=1-U18_CV-DEV-6bb564a7-d315-43b1-8f4e-b3c12cded986&offset=0 HTTP/1.1” 200 499 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 5 15:41:49 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: django.request:WARNING: Bad Request: /pulp/api/v3/repositories/deb/apt/6b622ebb-0206-41e3-b52e-50c1b930c465/sync/
Jul 5 15:41:49 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:49 +0000] “POST /pulp/api/v3/repositories/deb/apt/6b622ebb-0206-41e3-b52e-50c1b930c465/sync/ HTTP/1.1” 400 82 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 5 15:41:49 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:49 +0000] “GET /pulp/api/v3/repositories/deb/apt/?limit=2000&name=1-U18_CV-DEV-6bb564a7-d315-43b1-8f4e-b3c12cded986&offset=0 HTTP/1.1” 200 499 “-” “OpenAPI-Generator/2.11.1/ruby”
Jul 5 15:41:49 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: django.request:WARNING: Bad Request: /pulp/api/v3/publications/deb/apt/
Jul 5 15:41:49 sydinftst04 pulpcore-api[69049]: pulp [994bc74b-c89d-4724-96b6-eefeadbd9a05]: - - [05/Jul/2021:05:41:49 +0000] “POST /pulp/api/v3/publications/deb/apt/ HTTP/1.1” 400 85 “-” “OpenAPI-Generator/2.11.1/ruby”

Mamba · July 5, 2021, 5:10pm

+1
I’m trying to sync debian buster content from a katello 4.0.1-1 server to a smart proxy and encountering similar problems.

In the logs on the proxy I’m getting

psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "core_contentguard_name_key"

In the logs on the main foreman server I’m getting

Response body: {"distributions":["This field is required."]}Error message: the server returns an error
Response body: {"remote":["This field is required since a remote is not set on the repository."]}Error message: the server returns an error
Response body: {"non_field_errors":["one of simple or structured publishing mode must be selected"]}

yaser · July 7, 2021, 6:26am

@ATIX team?
deb repository form doesnt have any field called “distributions” and “/pulp/api/v3/distributions/deb/apt/” api and below api needs that field. can someone help in this case???

pulp_deb_client/api/distributions_apt_api.rb