Problem:
After migrating from Foreman2.3 /Katello 3.18 to Foreman2.4 /Katello 4.0 some of my repos do not sync and fail with:
Katello::Errors::Pulp3Error: No declared artifact with relative path "pgdg-redhat-repo-42.0-11.noarch.rpm" for content "<Package: pgdg-redhat-repo>"
My Repos which do not sync:
- Index of /pub/repos/yum/12/redhat/rhel-7-x86_64/ → on demand
- https://packages.grafana.com/oss/rpm → on demand
I also tried to generate a new Repo, which also fails with the same error on sync
Running foreman-rake katello:delete_orphaned_content RAILS_ENV=production did not help.
May anyone has a solution or a workaround for this issue.
The full is attached below:
Expected outcome:
Repo sync success
Foreman and Proxy versions:
- candlepin-3.2.11-1.el7.noarch
- candlepin-selinux-3.2.11-1.el7.noarch
- foreman-2.4.0-1.el7.noarch
- foreman-cli-2.4.0-1.el7.noarch
- foreman-debug-2.4.0-1.el7.noarch
- foreman-dynflow-sidekiq-2.4.0-1.el7.noarch
- foreman-installer-2.4.0-1.el7.noarch
- foreman-installer-katello-2.4.0-1.el7.noarch
- foreman-postgresql-2.4.0-1.el7.noarch
- foreman-proxy-2.4.0-1.el7.noarch
- foreman-release-2.4.0-1.el7.noarch
- foreman-selinux-2.4.0-1.el7.noarch
- foreman-service-2.4.0-1.el7.noarch
- foreman-vmware-2.4.0-1.el7.noarch
- katello-4.0.0-1.el7.noarch
- katello-ca-consumer-ssibol-foreman.ssi.bfc.bolcom.net-1.0-2.noarch
- katello-certs-tools-2.7.3-1.el7.noarch
- katello-client-bootstrap-1.7.6-1.el7.noarch
- katello-common-4.0.0-1.el7.noarch
- katello-debug-4.0.0-1.el7.noarch
- katello-default-ca-1.0-1.noarch
- katello-host-tools-3.5.1-2.el7sat.noarch
- katello-host-tools-fact-plugin-3.5.1-2.el7sat.noarch
- katello-repos-4.0.0-1.el7.noarch
- katello-selinux-4.0.0-1.el7.noarch
- katello-server-ca-1.0-8.noarch
- katello-service-3.14.1-1.el7.noarch
- pulp-client-1.0-3.noarch
- pulpcore-selinux-1.2.4-1.el7.x86_64
- python-gofer-qpid-2.12.5-3.el7.noarch
- python2-qpid-1.37.0-5.el7.noarch
- python2-qpid-proton-0.33.0-1.el7.x86_64
- python2-qpid-qmf-1.39.0-1.el7.x86_64
- python3-pulp-2to3-migration-0.11.0-1.el7.noarch
- python3-pulp-certguard-1.1.0-1.el7.noarch
- python3-pulp-container-2.2.1-1.el7.noarch
- python3-pulp-file-1.5.0-1.el7.noarch
- python3-pulp-rpm-3.10.0-1.el7.noarch
- python3-pulpcore-3.9.1-2.el7.noarch
- qpid-cpp-client-1.39.0-1.el7.x86_64
- qpid-cpp-client-devel-1.39.0-1.el7.x86_64
- qpid-cpp-server-1.39.0-1.el7.x86_64
- qpid-cpp-server-linearstore-1.39.0-1.el7.x86_64
- qpid-dispatch-router-1.15.0-1.el7.x86_64
- qpid-proton-c-0.33.0-1.el7.x86_64
- qpid-qmf-1.39.0-1.el7.x86_64
- qpid-tools-1.39.0-1.el7.noarch
- qpid_router_katello_agent-qpid-router-client-1.0-1.noarch
- rubygem-foreman_maintain-0.7.9-1.el7.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-apache-1.0-8.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-foreman-client-1.0-3.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-foreman-proxy-1.0-8.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-foreman-proxy-client-1.0-3.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-puppet-client-1.0-1.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-qpid-broker-1.0-3.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-qpid-client-cert-1.0-1.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-qpid-router-client-1.0-1.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-qpid-router-server-1.0-3.noarch
- ssibol-foreman.ssi.bfc.bolcom.net-tomcat-1.0-1.noarch
- tfm-rubygem-actioncable-6.0.3.4-1.el7.noarch
- tfm-rubygem-actionmailbox-6.0.3.4-1.el7.noarch
- tfm-rubygem-actionmailer-6.0.3.4-1.el7.noarch
- tfm-rubygem-actionpack-6.0.3.4-1.el7.noarch
- tfm-rubygem-actiontext-6.0.3.4-1.el7.noarch
- tfm-rubygem-actionview-6.0.3.4-1.el7.noarch
- tfm-rubygem-activejob-6.0.3.4-1.el7.noarch
- tfm-rubygem-activemodel-6.0.3.4-1.el7.noarch
- tfm-rubygem-activerecord-6.0.3.4-1.el7.noarch
- tfm-rubygem-activerecord-import-1.0.0-2.el7.noarch
- tfm-rubygem-activerecord-session_store-1.1.1-4.el7.noarch
- tfm-rubygem-activestorage-6.0.3.4-1.el7.noarch
- tfm-rubygem-activesupport-6.0.3.4-1.el7.noarch
- tfm-rubygem-addressable-2.6.0-2.el7.noarch
- tfm-rubygem-algebrick-0.7.3-7.el7.noarch
- tfm-rubygem-amazing_print-1.1.0-1.el7.noarch
- tfm-rubygem-ancestry-3.0.7-1.el7.noarch
- tfm-rubygem-anemone-0.7.2-17.el7.noarch
- tfm-rubygem-angular-rails-templates-1.1.0-1.el7.noarch
- tfm-rubygem-ansi-1.5.0-2.el7.noarch
- tfm-rubygem-apipie-bindings-0.4.0-1.el7.noarch
- tfm-rubygem-apipie-dsl-2.3.0-1.el7.noarch
- tfm-rubygem-apipie-params-0.0.5-4.el7.noarch
- tfm-rubygem-apipie-rails-0.5.17-3.el7.noarch
- tfm-rubygem-arel-9.0.0-4.el7.noarch
- tfm-rubygem-audited-4.9.0-3.el7.noarch
- tfm-rubygem-awesome_print-1.8.0-5.el7.noarch
- tfm-rubygem-bcrypt-3.1.12-3.el7.x86_64
- tfm-rubygem-builder-3.2.4-1.el7.noarch
- tfm-rubygem-bundler_ext-0.4.1-5.el7.noarch
- tfm-rubygem-clamp-1.1.2-6.el7.noarch
- tfm-rubygem-coffee-rails-5.0.0-1.el7.noarch
- tfm-rubygem-coffee-script-2.4.1-4.el7.noarch
- tfm-rubygem-coffee-script-source-1.12.2-4.el7.noarch
- tfm-rubygem-concurrent-ruby-1.1.6-2.el7.noarch
- tfm-rubygem-concurrent-ruby-edge-0.6.0-2.fm2_1.el7.noarch
- tfm-rubygem-connection_pool-2.2.2-2.el7.noarch
- tfm-rubygem-crass-1.0.6-1.el7.noarch
- tfm-rubygem-css_parser-1.4.7-4.el7.noarch
- tfm-rubygem-daemons-1.2.3-6.el7.noarch
- tfm-rubygem-deacon-1.0.0-4.el7.noarch
- tfm-rubygem-deep_cloneable-3.0.0-3.el7.noarch
- tfm-rubygem-deface-1.5.3-2.el7.noarch
- tfm-rubygem-domain_name-0.5.20160310-4.el7sat.noarch
- tfm-rubygem-dynflow-1.4.7-1.fm2_3.el7.noarch
- tfm-rubygem-erubi-1.9.0-1.el7.noarch
- tfm-rubygem-ethon-0.12.0-1.el7.noarch
- tfm-rubygem-excon-0.76.0-1.el7.noarch
- tfm-rubygem-execjs-2.7.0-4.el7.noarch
- tfm-rubygem-facter-4.0.44-1.el7.x86_64
- tfm-rubygem-faraday-0.17.3-1.el7.noarch
- tfm-rubygem-fast_gettext-1.4.1-4.el7.noarch
- tfm-rubygem-ffi-1.12.2-1.el7.x86_64
- tfm-rubygem-fog-core-2.1.0-3.el7.noarch
- tfm-rubygem-fog-vsphere-3.5.0-1.el7.noarch
- tfm-rubygem-foreman-tasks-4.0.1-1.fm2_4.el7.noarch
- tfm-rubygem-foreman-tasks-core-0.3.4-1.fm2_1.el7.noarch
- tfm-rubygem-foreman_ansible-6.2.0-1.fm2_4.el7.noarch
- tfm-rubygem-foreman_ansible_core-4.0.0-1.fm2_4.el7.noarch
- tfm-rubygem-foreman_remote_execution-4.3.0-1.fm2_4.el7.noarch
- tfm-rubygem-foreman_remote_execution_core-1.4.0-1.el7.noarch
- tfm-rubygem-foreman_virt_who_configure-0.5.5-1.fm2_4.el7.noarch
- tfm-rubygem-formatador-0.2.1-12.el7.noarch
- tfm-rubygem-friendly_id-5.3.0-1.el7.noarch
- tfm-rubygem-fx-0.5.0-1.el7.noarch
- tfm-rubygem-get_process_mem-0.2.1-4.el7.noarch
- tfm-rubygem-gettext_i18n_rails-1.8.0-2.el7.noarch
- tfm-rubygem-gitlab-sidekiq-fetcher-0.6.0-1.el7.noarch
- tfm-rubygem-globalid-0.4.2-1.el7.noarch
- tfm-rubygem-graphql-1.8.14-2.el7.noarch
- tfm-rubygem-graphql-batch-0.3.10-2.el7.noarch
- tfm-rubygem-gssapi-1.2.0-7.el7.noarch
- tfm-rubygem-hammer_cli-2.4.0-1.el7.noarch
- tfm-rubygem-hammer_cli_foreman-2.4.0-1.el7.noarch
- tfm-rubygem-hammer_cli_foreman_ansible-0.3.3-1.fm2_4.el7.noarch
- tfm-rubygem-hammer_cli_foreman_bootdisk-0.3.0-1.el7.noarch
- tfm-rubygem-hammer_cli_foreman_docker-0.0.7-1.el7.noarch
- tfm-rubygem-hammer_cli_foreman_remote_execution-0.2.1-1.fm2_4.el7.noarch
- tfm-rubygem-hammer_cli_foreman_tasks-0.0.15-1.fm2_2.el7.noarch
- tfm-rubygem-hammer_cli_katello-1.0-1.el7.noarch
- tfm-rubygem-hashie-3.6.0-2.el7.noarch
- tfm-rubygem-highline-2.0.3-1.el7.noarch
- tfm-rubygem-hocon-1.3.1-1.el7.noarch
- tfm-rubygem-http-cookie-1.0.2-5.el7sat.noarch
- tfm-rubygem-i18n-1.8.2-1.el7.noarch
- tfm-rubygem-ipaddress-0.8.0-12.el7.noarch
- tfm-rubygem-jwt-2.2.1-2.el7.noarch
- tfm-rubygem-kafo-6.2.1-1.el7.noarch
- tfm-rubygem-kafo_parsers-1.1.0-3.el7.noarch
- tfm-rubygem-kafo_wizards-0.0.2-1.el7.noarch
- tfm-rubygem-katello-4.0.0-1.el7.noarch
- tfm-rubygem-ldap_fluff-0.5.0-1.el7.noarch
- tfm-rubygem-little-plugger-1.1.4-2.el7.noarch
- tfm-rubygem-locale-2.0.9-14.el7.noarch
- tfm-rubygem-logging-2.3.0-1.el7.noarch
- tfm-rubygem-loofah-2.4.0-1.el7.noarch
- tfm-rubygem-mail-2.7.1-1.el7.noarch
- tfm-rubygem-marcel-0.3.3-1.el7.noarch
- tfm-rubygem-method_source-0.9.2-2.el7.noarch
- tfm-rubygem-mime-types-3.2.2-4.el7.noarch
- tfm-rubygem-mime-types-data-3.2018.0812-4.el7.noarch
- tfm-rubygem-mimemagic-0.3.5-1.el7.noarch
- tfm-rubygem-mini_mime-1.0.2-1.el7.noarch
- tfm-rubygem-mini_portile2-2.4.0-1.el7.noarch
- tfm-rubygem-multi_json-1.14.1-2.el7.noarch
- tfm-rubygem-multipart-post-2.0.0-2.el7.noarch
- tfm-rubygem-mustermann-1.0.2-4.el7.noarch
- tfm-rubygem-net-ldap-0.17.0-1.el7.noarch
- tfm-rubygem-net-ping-2.0.1-4.el7.noarch
- tfm-rubygem-net-scp-1.2.1-4.el7.noarch
- tfm-rubygem-net-ssh-4.2.0-2.el7.noarch
- tfm-rubygem-netrc-0.11.0-5.el7.noarch
- tfm-rubygem-nio4r-2.5.4-1.el7.x86_64
- tfm-rubygem-nokogiri-1.10.9-1.el7.x86_64
- tfm-rubygem-oauth-0.5.4-4.el7.noarch
- tfm-rubygem-optimist-3.0.0-2.el7.noarch
- tfm-rubygem-parse-cron-0.1.4-4.fm2_1.el7.noarch
- tfm-rubygem-passenger-4.0.18-10.13.el7.x86_64
- tfm-rubygem-passenger-native-4.0.18-10.13.el7.x86_64
- tfm-rubygem-passenger-native-libs-4.0.18-10.13.el7.x86_64
- tfm-rubygem-pg-1.1.4-3.el7.x86_64
- tfm-rubygem-polyglot-0.3.5-2.el7.noarch
- tfm-rubygem-powerbar-2.0.1-2.el7sat.noarch
- tfm-rubygem-promise.rb-0.7.4-2.el7.noarch
- tfm-rubygem-public_suffix-3.0.3-2.el7.noarch
- tfm-rubygem-pulp_2to3_migration_client-0.7.0-1.el7.noarch
- tfm-rubygem-pulp_ansible_client-0.6.0-1.el7.noarch
- tfm-rubygem-pulp_certguard_client-1.0.3-1.el7.noarch
- tfm-rubygem-pulp_container_client-2.2.0-1.el7.noarch
- tfm-rubygem-pulp_deb_client-2.8.0-1.el7.noarch
- tfm-rubygem-pulp_docker_client-4.0.0b6.dev01562331743-1.el7.noarch
- tfm-rubygem-pulp_file_client-1.5.0-1.el7.noarch
- tfm-rubygem-pulp_rpm_client-3.10.0-1.el7.noarch
- tfm-rubygem-pulpcore_client-3.9.0-1.el7.noarch
- tfm-rubygem-puma-5.1.1-1.el7.x86_64
- tfm-rubygem-qpid_messaging-1.38.0-1.el7.x86_64
- tfm-rubygem-qpid_proton-0.33.0-1.el7.x86_64
- tfm-rubygem-rabl-0.14.3-1.el7.noarch
- tfm-rubygem-rack-2.2.3-1.el7.noarch
- tfm-rubygem-rack-cors-1.0.2-2.el7.noarch
- tfm-rubygem-rack-jsonp-1.3.1-9.el7.noarch
- tfm-rubygem-rack-protection-2.1.0-1.el7.noarch
- tfm-rubygem-rack-test-1.1.0-4.el7.noarch
- tfm-rubygem-rails-6.0.3.4-1.el7.noarch
- tfm-rubygem-rails-dom-testing-2.0.3-6.el7.noarch
- tfm-rubygem-rails-html-sanitizer-1.3.0-1.el7.noarch
- tfm-rubygem-rails-i18n-6.0.0-2.el7.noarch
- tfm-rubygem-railties-6.0.3.4-1.el7.noarch
- tfm-rubygem-rainbow-2.2.1-3.el7.noarch
- tfm-rubygem-rake-compiler-1.0.7-3.el7.noarch
- tfm-rubygem-rb-inotify-0.9.7-5.el7.noarch
- tfm-rubygem-rbvmomi-2.2.0-3.el7.noarch
- tfm-rubygem-record_tag_helper-1.0.1-3.el7.noarch
- tfm-rubygem-redfish_client-0.5.2-1.el7.noarch
- tfm-rubygem-redis-4.1.2-2.el7.noarch
- tfm-rubygem-responders-3.0.0-3.el7.noarch
- tfm-rubygem-rest-client-2.0.2-3.el7.noarch
- tfm-rubygem-rkerberos-0.1.5-19.el7.x86_64
- tfm-rubygem-roadie-3.4.0-3.el7.noarch
- tfm-rubygem-roadie-rails-2.1.1-2.el7.noarch
- tfm-rubygem-robotex-1.0.0-21.el7.noarch
- tfm-rubygem-rsec-0.4.3-4.el7.noarch
- tfm-rubygem-ruby-libvirt-0.7.1-1.el7.x86_64
- tfm-rubygem-ruby2ruby-2.4.2-3.el7.noarch
- tfm-rubygem-ruby_parser-3.10.1-3.el7.noarch
- tfm-rubygem-rubyipmi-0.10.0-6.el7.noarch
- tfm-rubygem-runcible-2.13.1-1.el7.noarch
- tfm-rubygem-safemode-1.3.6-1.el7.noarch
- tfm-rubygem-scoped_search-4.1.9-1.el7.noarch
- tfm-rubygem-sd_notify-0.1.0-1.el7.noarch
- tfm-rubygem-secure_headers-6.3.0-2.el7.noarch
- tfm-rubygem-sequel-5.7.1-3.el7.noarch
- tfm-rubygem-server_sent_events-0.1.2-1.el7.noarch
- tfm-rubygem-sexp_processor-4.10.0-6.el7.noarch
- tfm-rubygem-sidekiq-5.2.7-3.el7.noarch
- tfm-rubygem-sinatra-2.1.0-1.el7.noarch
- tfm-rubygem-smart_proxy_ansible-3.0.1-6.fm2_2.el7.noarch
- tfm-rubygem-smart_proxy_dynflow-0.3.0-2.fm2_4.el7.noarch
- tfm-rubygem-smart_proxy_dynflow_core-0.3.2-1.fm2_4.el7.noarch
- tfm-rubygem-smart_proxy_pulp-2.1.0-3.fm2_2.el7.noarch
- tfm-rubygem-smart_proxy_remote_execution_ssh-0.3.1-1.fm2_4.el7.noarch
- tfm-rubygem-sprockets-4.0.2-1.el7.noarch
- tfm-rubygem-sprockets-rails-3.2.1-6.el7.noarch
- tfm-rubygem-sqlite3-1.3.13-6.el7.x86_64
- tfm-rubygem-sshkey-1.9.0-4.el7.noarch
- tfm-rubygem-statsd-instrument-2.1.4-3.el7.noarch
- tfm-rubygem-stomp-1.4.9-1.el7.noarch
- tfm-rubygem-thor-1.0.1-2.el7.noarch
- tfm-rubygem-thread_safe-0.3.6-5.el7.noarch
- tfm-rubygem-tilt-2.0.8-4.el7.noarch
- tfm-rubygem-typhoeus-1.3.1-1.el7.noarch
- tfm-rubygem-tzinfo-1.2.6-1.el7.noarch
- tfm-rubygem-unf-0.1.3-8.el7.noarch
- tfm-rubygem-unf_ext-0.0.7.2-3.el7.x86_64
- tfm-rubygem-unicode-0.4.4.4-3.el7.x86_64
- tfm-rubygem-unicode-display_width-1.0.5-5.el7sat.noarch
- tfm-rubygem-validates_lengths_from_database-0.5.0-7.el7.noarch
- tfm-rubygem-webpack-rails-0.9.8-5.el7.noarch
- tfm-rubygem-websocket-driver-0.7.1-1.el7.x86_64
- tfm-rubygem-websocket-extensions-0.1.5-1.el7.noarch
- tfm-rubygem-will_paginate-3.1.7-3.el7.noarch
- tfm-rubygem-xmlrpc-0.3.0-2.el7.noarch
- tfm-rubygem-zeitwerk-2.2.2-1.el7.noarch
- tfm-runtime-6.1-4.el7.x86_64
Foreman and Proxy plugin versions:
Distribution and version:
Rhel 7.9
Other relevant data:
Foreman task sync log:
{“pulp_tasks”=> [{“pulp_href”=>"/pulp/api/v3/tasks/7ef900f6-6e4e-482d-ae0c-f393aacb32bf/", “pulp_created”=>“2021-05-18T14:08:29.674+00:00”, “state”=>“failed”, “name”=>“pulp_rpm.app.tasks.synchronizing.synchronize”, “logging_cid”=>“c1f1e51359a24ca6b4855b084c0cd079”, “started_at”=>“2021-05-18T14:08:30.718+00:00”, “finished_at”=>“2021-05-18T14:08:40.640+00:00”, “error”=> {“traceback”=> " File “/usr/lib/python3.6/site-packages/rq/worker.py”, line 975, in perform_job\n" + " rv = job.perform()\n" + " File “/usr/lib/python3.6/site-packages/rq/job.py”, line 696, in perform\n" + " self._result = self._execute()\n" + " File “/usr/lib/python3.6/site-packages/rq/job.py”, line 719, in _execute\n" + " return self.func(*self.args, **self.kwargs)\n" + " File “/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py”, line 269, in synchronize\n" + " dv.create()\n" + " File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py”, line 148, in create\n" + " loop.run_until_complete(pipeline)\n" + " File “/usr/lib64/python3.6/asyncio/base_events.py”, line 484, in run_until_complete\n" + " return future.result()\n" + " File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 225, in create_pipeline\n" + " await asyncio.gather(*futures)\n" + " File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py”, line 43, in **call**\n" + " await self.run()\n" + " File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py”, line 263, in run\n" + " RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))\n" + " File “/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py”, line 316, in _needed_remote_artifacts\n" + " msg.format(rp=content_artifact.relative_path, c=d_content.content)\n", “description”=> “No declared artifact with relative path “pgdg-redhat-repo-42.0-11.noarch.rpm” for content “<Package: pgdg-redhat-repo>””}, “worker”=>"/pulp/api/v3/workers/25ce6979-f5d5-4a05-9e21-160934741f44/", “child_tasks”=>, “progress_reports”=> [{“message”=>“Parsed Packages”, “code”=>“parsing.packages”, “state”=>“completed”, “total”=>540, “done”=>540}, {“message”=>“Downloading Metadata Files”, “code”=>“downloading.metadata”, “state”=>“completed”, “done”=>4}, {“message”=>“Downloading Artifacts”, “code”=>“downloading.artifacts”, “state”=>“canceled”, “done”=>0}, {“message”=>“Associating Content”, “code”=>“associating.content”, “state”=>“canceled”, “done”=>0}, {“message”=>“Un-Associating Content”, “code”=>“unassociating.content”, “state”=>“canceled”, “done”=>0}, {“message”=>“Parsed Comps”, “code”=>“parsing.comps”, “state”=>“completed”, “total”=>1, “done”=>1}], “created_resources”=>, “reserved_resources_record”=> ["/pulp/api/v3/repositories/rpm/rpm/8ea73b81-63f3-4064-a9af-c9d19d083060/", “/pulp/api/v3/remotes/rpm/rpm/01f78dd7-200f-460a-a47f-23510c99ade0/”]}], “create_version”=>true, “task_groups”=>, “poll_attempts”=>{“total”=>11, “failed”=>1}}
/var/log/messages:
May 19 07:55:56 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: d2f0c95d79fa4fbfb225c3c93c491967 May 19 07:55:56 foreman pulpcore-api: - - [19/May/2021:05:55:56 +0000] "GET /pulp/api/v3/status HTTP/1.1" 301 0 "-" "rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157" May 19 07:55:56 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: 32b730f1a8754e3da09faf68c1cfacef May 19 07:55:56 foreman pulpcore-api: - - [19/May/2021:05:55:56 +0000] "GET /pulp/api/v3/status/ HTTP/1.1" 200 4054 "-" "rest-client/2.0.2 (linux-gnu x86_64) ruby/2.5.5p157" May 19 07:55:57 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: 4727215826544302a4a291fc6a87118c May 19 07:55:57 foreman pulpcore-resource-manager: pulp [None]: rq.worker:INFO: resource-manager: 2c0e4c35-3ff9-4562-b7d4-b6ed3efc92fa May 19 07:55:57 foreman pulpcore-api: - - [19/May/2021:05:55:57 +0000] "POST /pulp/api/v3/repositories/rpm/rpm/8ea73b81-63f3-4064-a9af-c9d19d083060/sync/ HTTP/1.1" 202 67 "-" "OpenAPI-Generator/3.10.0/ruby" May 19 07:55:57 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: 57372fff3f824bc9a1172e95f158ac57 May 19 07:55:57 foreman pulpcore-api: - - [19/May/2021:05:55:57 +0000] "GET /pulp/api/v3/tasks/acbf6a85-a2b1-43fa-b211-314b3d8e6ed9/ HTTP/1.1" 200 574 "-" "OpenAPI-Generator/3.9.0/ruby" May 19 07:55:57 foreman pulpcore-resource-manager: pulp [4727215826544302a4a291fc6a87118c]: rq.worker:INFO: resource-manager: Job OK (2c0e4c35-3ff9-4562-b7d4-b6ed3efc92fa) May 19 07:55:57 foreman pulpcore-worker-8: pulp [None]: rq.worker:INFO: 1361@foreman.example.com: acbf6a85-a2b1-43fa-b211-314b3d8e6ed9 May 19 07:55:57 foreman pulpcore-worker-8: pulp [4727215826544302a4a291fc6a87118c]: pulp_rpm.app.tasks.synchronizing:INFO: Synchronizing: repository=75d31a7e-28c8-41bc-be6b-c63180040824 remote=5f358220f1028544342eb26f-75d31a7e-28c8-41bc-be6b-c63180040824 May 19 07:55:57 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: f04c873e78e345229725c7db9b7fcc29 May 19 07:55:57 foreman pulpcore-api: - - [19/May/2021:05:55:57 +0000] "GET /pulp/api/v3/tasks/acbf6a85-a2b1-43fa-b211-314b3d8e6ed9/ HTTP/1.1" 200 655 "-" "OpenAPI-Generator/3.9.0/ruby" May 19 07:55:58 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: 06daa573d4c14ba3913b8fb8ab4abd8b May 19 07:55:58 foreman pulpcore-api: - - [19/May/2021:05:55:58 +0000] "GET /pulp/api/v3/tasks/acbf6a85-a2b1-43fa-b211-314b3d8e6ed9/ HTTP/1.1" 200 655 "-" "OpenAPI-Generator/3.9.0/ruby" May 19 07:55:59 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: d622f13dd9954b25934ddec3a0759509 May 19 07:55:59 foreman pulpcore-api: - - [19/May/2021:05:55:59 +0000] "GET /pulp/api/v3/tasks/acbf6a85-a2b1-43fa-b211-314b3d8e6ed9/ HTTP/1.1" 200 1246 "-" "OpenAPI-Generator/3.9.0/ruby" May 19 07:55:59 foreman systemd: serial-getty@ttyS0.service holdoff time over, scheduling restart. May 19 07:55:59 foreman systemd: Stopped Serial Getty on ttyS0. May 19 07:55:59 foreman systemd: Started Serial Getty on ttyS0. May 19 07:55:59 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: a03aae47c9b04e6385efb9f8ec714334 May 19 07:55:59 foreman pulpcore-api: - - [19/May/2021:05:55:59 +0000] "GET /pulp/api/v3/tasks/acbf6a85-a2b1-43fa-b211-314b3d8e6ed9/ HTTP/1.1" 200 1361 "-" "OpenAPI-Generator/3.9.0/ruby" May 19 07:56:00 foreman pulpcore-worker-8: pulp [4727215826544302a4a291fc6a87118c]: rq.worker:ERROR: Traceback (most recent call last): May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 975, in perform_job May 19 07:56:00 foreman pulpcore-worker-8: rv = job.perform() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/rq/job.py", line 696, in perform May 19 07:56:00 foreman pulpcore-worker-8: self._result = self._execute() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/rq/job.py", line 719, in _execute May 19 07:56:00 foreman pulpcore-worker-8: return self.func(*self.args, **self.kwargs) May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 269, in synchronize May 19 07:56:00 foreman pulpcore-worker-8: dv.create() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create May 19 07:56:00 foreman pulpcore-worker-8: loop.run_until_complete(pipeline) May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete May 19 07:56:00 foreman pulpcore-worker-8: return future.result() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline May 19 07:56:00 foreman pulpcore-worker-8: await asyncio.gather(*futures) May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__ May 19 07:56:00 foreman pulpcore-worker-8: await self.run() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 263, in run May 19 07:56:00 foreman pulpcore-worker-8: RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch)) May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 316, in _needed_remote_artifacts May 19 07:56:00 foreman pulpcore-worker-8: msg.format(rp=content_artifact.relative_path, c=d_content.content) May 19 07:56:00 foreman pulpcore-worker-8: ValueError: No declared artifact with relative path "pgdg-redhat-repo-42.0-11.noarch.rpm" for content "<Package: pgdg-redhat-repo>" May 19 07:56:00 foreman pulpcore-worker-8: Traceback (most recent call last): May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 975, in perform_job May 19 07:56:00 foreman pulpcore-worker-8: rv = job.perform() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/rq/job.py", line 696, in perform May 19 07:56:00 foreman pulpcore-worker-8: self._result = self._execute() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/rq/job.py", line 719, in _execute May 19 07:56:00 foreman pulpcore-worker-8: return self.func(*self.args, **self.kwargs) May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 269, in synchronize May 19 07:56:00 foreman pulpcore-worker-8: dv.create() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/declarative_version.py", line 148, in create May 19 07:56:00 foreman pulpcore-worker-8: loop.run_until_complete(pipeline) May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete May 19 07:56:00 foreman pulpcore-worker-8: return future.result() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline May 19 07:56:00 foreman pulpcore-worker-8: await asyncio.gather(*futures) May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__ May 19 07:56:00 foreman pulpcore-worker-8: await self.run() May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 263, in run May 19 07:56:00 foreman pulpcore-worker-8: RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch)) May 19 07:56:00 foreman pulpcore-worker-8: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 316, in _needed_remote_artifacts May 19 07:56:00 foreman pulpcore-worker-8: msg.format(rp=content_artifact.relative_path, c=d_content.content) May 19 07:56:00 foreman pulpcore-worker-8: ValueError: No declared artifact with relative path "pgdg-redhat-repo-42.0-11.noarch.rpm" for content "<Package: pgdg-redhat-repo>" May 19 07:56:00 foreman pulpcore-worker-8: pulp [None]: rq.worker:INFO: 1361@foreman.example.com: 61856726-c4c7-4274-b637-32c7730b5d1c May 19 07:56:00 foreman pulpcore-worker-8: pulp [None]: rq.worker:INFO: 1361@foreman.example.com: Job OK (61856726-c4c7-4274-b637-32c7730b5d1c) May 19 07:56:00 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: 2d734bdab816443399919046656e58d7 May 19 07:56:00 foreman pulpcore-api: - - [19/May/2021:05:56:00 +0000] "GET /pulp/api/v3/tasks/acbf6a85-a2b1-43fa-b211-314b3d8e6ed9/ HTTP/1.1" 200 2924 "-" "OpenAPI-Generator/3.9.0/ruby" May 19 07:56:00 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: 074eb41994b443f9b3a43880a23e90c5 May 19 07:56:00 foreman pulpcore-api: - - [19/May/2021:05:56:00 +0000] "PATCH /pulp/api/v3/distributions/rpm/rpm/da7b128d-0e47-40b2-9bc5-298dad3f435c/ HTTP/1.1" 202 67 "-" "OpenAPI-Generator/3.10.0/ruby" May 19 07:56:00 foreman pulpcore-resource-manager: pulp [None]: rq.worker:INFO: resource-manager: eb31b108-b98e-4b8b-bea7-31076d38381e May 19 07:56:00 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: b7bcd65fc82b48cba495467d4e0de8a7 May 19 07:56:00 foreman pulpcore-api: - - [19/May/2021:05:56:00 +0000] "GET /pulp/api/v3/tasks/0d9fb0a4-9758-4af5-81a6-e73255ef6c0f/ HTTP/1.1" 200 450 "-" "OpenAPI-Generator/3.9.0/ruby" May 19 07:56:00 foreman pulpcore-resource-manager: pulp [074eb41994b443f9b3a43880a23e90c5]: rq.worker:INFO: resource-manager: Job OK (eb31b108-b98e-4b8b-bea7-31076d38381e) May 19 07:56:00 foreman pulpcore-worker-2: pulp [None]: rq.worker:INFO: 1346@foreman.example.com: 0d9fb0a4-9758-4af5-81a6-e73255ef6c0f May 19 07:56:00 foreman pulpcore-worker-2: pulp [074eb41994b443f9b3a43880a23e90c5]: rq.worker:INFO: 1346@foreman.example.com: Job OK (0d9fb0a4-9758-4af5-81a6-e73255ef6c0f) May 19 07:56:00 foreman pulpcore-worker-2: pulp [None]: rq.worker:INFO: 1346@foreman.example.com: d002f572-8432-43a2-839f-0fbfbcaa6bf8 May 19 07:56:00 foreman pulpcore-worker-2: pulp [None]: rq.worker:INFO: 1346@foreman.example.com: Job OK (d002f572-8432-43a2-839f-0fbfbcaa6bf8) May 19 07:56:01 foreman pulpcore-api: pulp [None]: django_guid:INFO: Header `Correlation-ID` was not found in the incoming request. Generated new GUID: 558bb4d5dca84f7ea4c67f54cb7449e3 May 19 07:56:01 foreman pulpcore-api: - - [19/May/2021:05:56:01 +0000] "GET /pulp/api/v3/tasks/0d9fb0a4-9758-4af5-81a6-e73255ef6c0f/ HTTP/1.1" 200 558 "-" "OpenAPI-Generator/3.9.0/ruby"