Problem:
When running the Pulp 2to3 migration preparation, I get the following error:
# date; foreman-maintain content prepare; date
Wed May 19 14:58:36 CEST 2021
Running Prepare content for Pulp 3
================================================================================
Prepare content for Pulp 3:
Rubocop not loaded.
Rubocop not loaded.
Starting task.
2021-05-19 15:01:18 +0200: Distribution creation 0/49Migration failed, You will want to investigate: https://foreman.example.com/foreman_tasks/tasks/b2d45fda-11a7-4998-9db9-205368616d98
rake aborted!
ForemanTasks::TaskError: Task b2d45fda-11a7-4998-9db9-205368616d98: Katello::Errors::Pulp3Error: 4 subtask(s) failed for task group /pulp/api/v3/task-groups/6f3fac92-fadf-41ec-8193-17c0ebc5f1bd/.
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.2.1/lib/katello/tasks/pulp3_migration.rake:33:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:pulp3_migration
(See full trace by running task with --trace)
[FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 1
--------------------------------------------------------------------------------
Scenario [Prepare content for Pulp 3] failed.
The following steps ended up in failing state:
[content-prepare]
Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="content-prepare"
Wed May 19 15:01:28 CEST 2021
The number of failed subtasks also varies slightly from 3 to 5.
Expected outcome:
content prepare task finishing without errors
Foreman and Proxy versions:
Foreman 2.3.3
Katello 3.18.2
Foreman and Proxy plugin versions:
candlepin-3.1.22-1.el7.noarch
candlepin-selinux-3.1.22-1.el7.noarch
deployc001.srv.muenchen.de-foreman-client-1.0-1.noarch
deployc001.srv.muenchen.de-foreman-proxy-1.0-1.noarch
deployc001.srv.muenchen.de-foreman-proxy-client-1.0-1.noarch
deployc001.srv.muenchen.de-qpid-broker-1.0-2.noarch
deployc001.srv.muenchen.de-qpid-client-cert-1.0-1.noarch
deployc001.srv.muenchen.de-qpid-router-client-1.0-1.noarch
deployc001.srv.muenchen.de-qpid-router-server-1.0-1.noarch
foreman-2.3.3-1.el7.noarch
foreman-cli-2.3.3-1.el7.noarch
foreman-debug-2.3.3-1.el7.noarch
foreman-dynflow-sidekiq-2.3.3-1.el7.noarch
foreman-installer-2.3.3-1.el7.noarch
foreman-installer-katello-2.3.3-1.el7.noarch
foreman-postgresql-2.3.3-1.el7.noarch
foreman-proxy-2.3.3-1.el7.noarch
foreman-release-2.3.3-1.el7.noarch
foreman-release-scl-7-3.el7.noarch
foreman-selinux-2.3.3-1.el7.noarch
foreman-service-2.3.3-1.el7.noarch
foreman-vmware-2.3.3-1.el7.noarch
katello-3.18.2-1.el7.noarch
katello-agent-3.5.4-1.el7.noarch
katello-certs-tools-2.7.3-1.el7.noarch
katello-client-bootstrap-1.7.5-1.el7.noarch
katello-common-3.18.2-1.el7.noarch
katello-debug-3.18.2-1.el7.noarch
katello-default-ca-1.0-1.noarch
katello-host-tools-3.5.4-1.el7.noarch
katello-host-tools-fact-plugin-3.5.4-1.el7.noarch
katello-repos-3.18.2-1.el7.noarch
katello-selinux-3.5.0-1.el7.noarch
katello-server-ca-1.0-1.noarch
katello-service-3.14.1-1.el7.noarch
pulp-admin-client-2.21.5-1.el7.noarch
pulp-client-1.0-1.noarch
pulp-deb-plugins-1.10.2-1.el7.noarch
pulp-docker-plugins-3.2.9-1.el7.noarch
pulp-katello-1.0.3-1.el7.noarch
pulp-puppet-plugins-2.21.5-1.el7.noarch
pulp-puppet-tools-2.21.5-1.el7.noarch
pulp-rpm-admin-extensions-2.21.5-1.el7.noarch
pulp-rpm-plugins-2.21.5-1.el7.noarch
pulp-selinux-2.21.5-1.el7.noarch
pulp-server-2.21.5-1.el7.noarch
pulpcore-selinux-1.2.3-2.el7.x86_64
python-gofer-qpid-2.12.5-3.el7.noarch
python-isodate-0.5.0-4.pulp.el7.noarch
python-pulp-bindings-2.21.5-1.el7.noarch
python-pulp-client-lib-2.21.5-1.el7.noarch
python-pulp-common-2.21.5-1.el7.noarch
python-pulp-deb-common-1.10.2-1.el7.noarch
python-pulp-docker-common-3.2.9-1.el7.noarch
python-pulp-oid_validation-2.21.5-1.el7.noarch
python-pulp-puppet-common-2.21.5-1.el7.noarch
python-pulp-repoauth-2.21.5-1.el7.noarch
python-pulp-rpm-common-2.21.5-1.el7.noarch
python-pulp-streamer-2.21.5-1.el7.noarch
python2-qpid-1.37.0-5.el7.noarch
python2-qpid-proton-0.33.0-1.el7.x86_64
python2-qpid-qmf-1.39.0-1.el7.x86_64
python3-pulp-2to3-migration-0.11.0-1.el7.noarch
python3-pulp-certguard-1.0.3-1.el7.noarch
python3-pulp-container-2.1.0-1.el7.noarch
python3-pulp-deb-2.7.0-1.el7.noarch
python3-pulp-file-1.3.0-1.el7.noarch
python3-pulp-rpm-3.10.0-1.el7.noarch
python3-pulpcore-3.7.5-1.el7.noarch
qpid-cpp-client-1.39.0-1.el7.x86_64
qpid-cpp-client-devel-1.39.0-1.el7.x86_64
qpid-cpp-server-1.39.0-1.el7.x86_64
qpid-cpp-server-linearstore-1.39.0-1.el7.x86_64
qpid-dispatch-router-1.15.0-1.el7.x86_64
qpid-proton-c-0.33.0-1.el7.x86_64
qpid-qmf-1.39.0-1.el7.x86_64
qpid-tools-1.39.0-1.el7.noarch
rubygem-foreman_maintain-0.7.5-1.el7.noarch
tfm-ror51-rubygem-actioncable-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-actionmailer-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-actionpack-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-actionview-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-activejob-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-activemodel-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-activerecord-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-activesupport-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-arel-8.0.0-2.el7.centos.noarch
tfm-ror51-rubygem-builder-3.2.3-2.el7.centos.noarch
tfm-ror51-rubygem-coffee-rails-4.2.2-2.el7.centos.noarch
tfm-ror51-rubygem-coffee-script-2.4.1-2.el7.centos.noarch
tfm-ror51-rubygem-coffee-script-source-1.12.2-2.el7.centos.noarch
tfm-ror51-rubygem-concurrent-ruby-1.0.5-4.el7.centos.noarch
tfm-ror51-rubygem-crass-1.0.2-2.el7.centos.noarch
tfm-ror51-rubygem-erubi-1.7.0-2.el7.centos.noarch
tfm-ror51-rubygem-execjs-2.7.0-2.el7.centos.noarch
tfm-ror51-rubygem-globalid-0.4.1-2.el7.centos.noarch
tfm-ror51-rubygem-i18n-0.9.1-3.el7.centos.noarch
tfm-ror51-rubygem-loofah-2.1.1-3.el7.centos.noarch
tfm-ror51-rubygem-mail-2.7.0-3.el7.centos.noarch
tfm-ror51-rubygem-method_source-0.9.0-2.el7.centos.noarch
tfm-ror51-rubygem-mime-types-3.1-2.el7.centos.noarch
tfm-ror51-rubygem-mime-types-data-3.2016.0521-2.el7.centos.noarch
tfm-ror51-rubygem-mini_mime-1.0.0-2.el7.centos.noarch
tfm-ror51-rubygem-multi_json-1.12.2-2.el7.centos.noarch
tfm-ror51-rubygem-mustermann-1.0.1-2.el7.centos.noarch
tfm-ror51-rubygem-nio4r-2.1.0-2.el7.centos.x86_64
tfm-ror51-rubygem-nokogiri-1.8.1-2.el7.centos.x86_64
tfm-ror51-rubygem-rack-2.0.3-2.el7.centos.noarch
tfm-ror51-rubygem-rack-protection-2.0.0-2.el7.centos.noarch
tfm-ror51-rubygem-rack-test-0.7.0-2.el7.centos.noarch
tfm-ror51-rubygem-rails-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-rails-dom-testing-2.0.3-2.el7.centos.noarch
tfm-ror51-rubygem-rails-html-sanitizer-1.0.3-3.el7.centos.noarch
tfm-ror51-rubygem-railties-5.1.6-1.el7.centos.noarch
tfm-ror51-rubygem-sinatra-2.0.0-2.el7.centos.noarch
tfm-ror51-rubygem-sprockets-3.7.1-2.el7.centos.noarch
tfm-ror51-rubygem-sprockets-rails-3.2.1-2.el7.centos.noarch
tfm-ror51-rubygem-sqlite3-1.3.13-2.el7.centos.x86_64
tfm-ror51-rubygem-thor-0.20.0-2.el7.centos.noarch
tfm-ror51-rubygem-thread_safe-0.3.6-2.el7.centos.noarch
tfm-ror51-rubygem-tilt-2.0.8-2.el7.centos.noarch
tfm-ror51-rubygem-turbolinks-2.5.4-2.el7.centos.noarch
tfm-ror51-rubygem-tzinfo-1.2.4-3.el7.centos.noarch
tfm-ror51-rubygem-websocket-driver-0.6.5-3.el7.centos.x86_64
tfm-ror51-rubygem-websocket-extensions-0.1.2-2.el7.centos.noarch
tfm-ror51-runtime-1.1-2.el7.centos.x86_64
tfm-rubygem-actioncable-6.0.3.4-1.el7.noarch
tfm-rubygem-actionmailbox-6.0.3.4-1.el7.noarch
tfm-rubygem-actionmailer-6.0.3.4-1.el7.noarch
tfm-rubygem-actionpack-6.0.3.4-1.el7.noarch
tfm-rubygem-actiontext-6.0.3.4-1.el7.noarch
tfm-rubygem-actionview-6.0.3.4-1.el7.noarch
tfm-rubygem-activejob-6.0.3.4-1.el7.noarch
tfm-rubygem-activemodel-6.0.3.4-1.el7.noarch
tfm-rubygem-activerecord-6.0.3.4-1.el7.noarch
tfm-rubygem-activerecord-import-1.0.0-2.el7.noarch
tfm-rubygem-activerecord-session_store-1.1.1-4.el7.noarch
tfm-rubygem-activestorage-6.0.3.4-1.el7.noarch
tfm-rubygem-activesupport-6.0.3.4-1.el7.noarch
tfm-rubygem-addressable-2.6.0-2.el7.noarch
tfm-rubygem-algebrick-0.7.3-7.el7.noarch
tfm-rubygem-amazing_print-1.1.0-1.el7.noarch
tfm-rubygem-ancestry-3.0.7-1.el7.noarch
tfm-rubygem-anemone-0.7.2-17.el7.noarch
tfm-rubygem-angular-rails-templates-1.1.0-1.el7.noarch
tfm-rubygem-ansi-1.5.0-2.el7.noarch
tfm-rubygem-apipie-bindings-0.4.0-1.el7.noarch
tfm-rubygem-apipie-dsl-2.3.0-1.el7.noarch
tfm-rubygem-apipie-params-0.0.5-4.el7.noarch
tfm-rubygem-apipie-rails-0.5.17-3.el7.noarch
tfm-rubygem-arel-9.0.0-4.el7.noarch
tfm-rubygem-audited-4.9.0-3.el7.noarch
tfm-rubygem-awesome_print-1.8.0-5.el7.noarch
tfm-rubygem-bcrypt-3.1.12-3.el7.x86_64
tfm-rubygem-builder-3.2.4-1.el7.noarch
tfm-rubygem-bundler_ext-0.4.1-5.el7.noarch
tfm-rubygem-clamp-1.1.2-6.el7.noarch
tfm-rubygem-coffee-rails-5.0.0-1.el7.noarch
tfm-rubygem-coffee-script-2.4.1-4.el7.noarch
tfm-rubygem-coffee-script-source-1.12.2-4.el7.noarch
tfm-rubygem-concurrent-ruby-1.1.6-2.el7.noarch
tfm-rubygem-concurrent-ruby-edge-0.6.0-2.fm2_1.el7.noarch
tfm-rubygem-connection_pool-2.2.2-2.el7.noarch
tfm-rubygem-crass-1.0.6-1.el7.noarch
tfm-rubygem-css_parser-1.4.7-4.el7.noarch
tfm-rubygem-daemons-1.2.3-6.el7.noarch
tfm-rubygem-deacon-1.0.0-4.el7.noarch
tfm-rubygem-declarative-0.0.10-2.el7.noarch
tfm-rubygem-declarative-option-0.1.0-2.el7.noarch
tfm-rubygem-deep_cloneable-3.0.0-3.el7.noarch
tfm-rubygem-deface-1.5.3-2.el7.noarch
tfm-rubygem-diffy-3.0.1-5.el7.noarch
tfm-rubygem-domain_name-0.5.20160310-4.el7.noarch
tfm-rubygem-dynflow-1.4.7-1.fm2_3.el7.noarch
tfm-rubygem-erubi-1.9.0-1.el7.noarch
tfm-rubygem-ethon-0.12.0-1.el7.noarch
tfm-rubygem-excon-0.76.0-1.el7.noarch
tfm-rubygem-execjs-2.7.0-4.el7.noarch
tfm-rubygem-facter-2.4.0-7.el7.x86_64
tfm-rubygem-faraday-0.17.3-1.el7.noarch
tfm-rubygem-faraday_middleware-0.13.1-2.el7.noarch
tfm-rubygem-fast_gettext-1.4.1-4.el7.noarch
tfm-rubygem-ffi-1.12.2-1.el7.x86_64
tfm-rubygem-fog-aws-3.6.5-1.el7.noarch
tfm-rubygem-fog-core-2.1.0-3.el7.noarch
tfm-rubygem-fog-digitalocean-0.4.0-2.el7.noarch
tfm-rubygem-fog-google-1.11.0-1.el7.noarch
tfm-rubygem-fog-json-1.2.0-3.el7.noarch
tfm-rubygem-fog-openstack-1.0.8-3.el7.noarch
tfm-rubygem-fog-ovirt-1.2.5-1.el7.noarch
tfm-rubygem-fog-vsphere-3.4.0-1.el7.noarch
tfm-rubygem-fog-xenserver-1.0.0-2.el7.noarch
tfm-rubygem-fog-xml-0.1.2-8.el7.noarch
tfm-rubygem-foreman-tasks-3.0.5-1.fm2_3.el7.noarch
tfm-rubygem-foreman-tasks-core-0.3.4-1.fm2_1.el7.noarch
tfm-rubygem-foreman_bootdisk-17.0.2-2.fm2_2.el7.noarch
tfm-rubygem-foreman_discovery-16.3.4-1.fm2_3.el7.noarch
tfm-rubygem-foreman_expire_hosts-7.0.4-1.fm2_3.el7.noarch
tfm-rubygem-foreman_hooks-0.3.17-1.fm2_3.el7.noarch
tfm-rubygem-foreman_remote_execution-4.2.2-1.fm2_3.el7.noarch
tfm-rubygem-foreman_remote_execution_core-1.4.0-1.el7.noarch
tfm-rubygem-foreman_scc_manager-1.8.8-1.fm2_3.el7.noarch
tfm-rubygem-foreman_snapshot_management-1.7.1-1.fm2_1.el7.noarch
tfm-rubygem-foreman_templates-9.0.1-1.fm2_2.el7.noarch
tfm-rubygem-formatador-0.2.1-12.el7.noarch
tfm-rubygem-friendly_id-5.3.0-1.el7.noarch
tfm-rubygem-fx-0.5.0-1.el7.noarch
tfm-rubygem-get_process_mem-0.2.1-4.el7.noarch
tfm-rubygem-gettext_i18n_rails-1.8.0-2.el7.noarch
tfm-rubygem-git-1.5.0-1.el7.noarch
tfm-rubygem-gitlab-sidekiq-fetcher-0.6.0-1.el7.noarch
tfm-rubygem-globalid-0.4.2-1.el7.noarch
tfm-rubygem-google-api-client-0.33.2-1.el7.noarch
tfm-rubygem-google-cloud-env-1.3.3-1.el7.noarch
tfm-rubygem-googleauth-0.13.1-1.el7.noarch
tfm-rubygem-graphql-1.8.14-2.el7.noarch
tfm-rubygem-graphql-batch-0.3.10-2.el7.noarch
tfm-rubygem-gssapi-1.2.0-7.el7.noarch
tfm-rubygem-hammer_cli-2.3.0-1.el7.noarch
tfm-rubygem-hammer_cli_foreman-2.3.1-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_bootdisk-0.3.0-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_docker-0.0.7-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_tasks-0.0.15-1.fm2_2.el7.noarch
tfm-rubygem-hammer_cli_katello-0.24.3-1.el7.noarch
tfm-rubygem-hashie-3.6.0-2.el7.noarch
tfm-rubygem-highline-1.7.8-5.el7.noarch
tfm-rubygem-http-cookie-1.0.2-4.el7.noarch
tfm-rubygem-httpclient-2.8.3-3.el7.noarch
tfm-rubygem-i18n-1.8.2-1.el7.noarch
tfm-rubygem-infoblox-3.0.0-3.el7.noarch
tfm-rubygem-ipaddress-0.8.0-12.el7.noarch
tfm-rubygem-jquery-ui-rails-6.0.1-1.el7.noarch
tfm-rubygem-jwt-2.2.1-2.el7.noarch
tfm-rubygem-kafo-6.1.2-1.el7.noarch
tfm-rubygem-kafo_parsers-1.1.0-3.el7.noarch
tfm-rubygem-kafo_wizards-0.0.1-4.el7.noarch
tfm-rubygem-katello-3.18.2.1-1.el7.noarch
tfm-rubygem-ldap_fluff-0.4.7-5.el7.noarch
tfm-rubygem-little-plugger-1.1.4-2.el7.noarch
tfm-rubygem-locale-2.0.9-14.el7.noarch
tfm-rubygem-logging-2.3.0-1.el7.noarch
tfm-rubygem-loofah-2.4.0-1.el7.noarch
tfm-rubygem-mail-2.7.1-1.el7.noarch
tfm-rubygem-marcel-0.3.3-1.el7.noarch
tfm-rubygem-memoist-0.16.0-2.el7.noarch
tfm-rubygem-method_source-0.9.2-2.el7.noarch
tfm-rubygem-mime-types-3.2.2-4.el7.noarch
tfm-rubygem-mime-types-data-3.2018.0812-4.el7.noarch
tfm-rubygem-mimemagic-0.3.5-1.el7.noarch
tfm-rubygem-mini_mime-1.0.2-1.el7.noarch
tfm-rubygem-mini_portile2-2.4.0-1.el7.noarch
tfm-rubygem-multi_json-1.14.1-2.el7.noarch
tfm-rubygem-multipart-post-2.0.0-2.el7.noarch
tfm-rubygem-mustermann-1.0.2-4.el7.noarch
tfm-rubygem-net-ldap-0.16.1-2.el7.noarch
tfm-rubygem-net-ping-2.0.1-4.el7.noarch
tfm-rubygem-net-scp-1.2.1-4.el7.noarch
tfm-rubygem-net-ssh-4.2.0-2.el7.noarch
tfm-rubygem-netrc-0.11.0-5.el7.noarch
tfm-rubygem-nio4r-2.5.4-1.el7.x86_64
tfm-rubygem-nokogiri-1.10.9-1.el7.x86_64
tfm-rubygem-oauth-0.5.4-4.el7.noarch
tfm-rubygem-optimist-3.0.0-2.el7.noarch
tfm-rubygem-os-1.0.0-2.el7.noarch
tfm-rubygem-ovirt-engine-sdk-4.3.0-1.el7.x86_64
tfm-rubygem-parse-cron-0.1.4-4.fm2_1.el7.noarch
tfm-rubygem-passenger-4.0.18-10.13.el7.x86_64
tfm-rubygem-passenger-native-4.0.18-10.13.el7.x86_64
tfm-rubygem-passenger-native-libs-4.0.18-10.13.el7.x86_64
tfm-rubygem-pg-1.1.4-3.el7.x86_64
tfm-rubygem-polyglot-0.3.5-2.el7.noarch
tfm-rubygem-powerbar-2.0.1-2.el7.noarch
tfm-rubygem-promise.rb-0.7.4-2.el7.noarch
tfm-rubygem-public_suffix-3.0.3-2.el7.noarch
tfm-rubygem-pulp_2to3_migration_client-0.7.0-1.el7.noarch
tfm-rubygem-pulp_ansible_client-0.4.2-1.el7.noarch
tfm-rubygem-pulp_certguard_client-1.0.3-1.el7.noarch
tfm-rubygem-pulp_container_client-2.1.0-1.el7.noarch
tfm-rubygem-pulp_deb_client-2.7.0-1.el7.noarch
tfm-rubygem-pulp_docker_client-4.0.0b6.dev01562331743-1.el7.noarch
tfm-rubygem-pulp_file_client-1.3.0-1.el7.noarch
tfm-rubygem-pulp_rpm_client-3.9.0-1.el7.noarch
tfm-rubygem-pulpcore_client-3.7.1-1.el7.noarch
tfm-rubygem-puma-4.3.6-1.el7.x86_64
tfm-rubygem-puma-plugin-systemd-0.1.5-1.el7.noarch
tfm-rubygem-puppetdb_foreman-5.0.0-2.fm2_1.el7.noarch
tfm-rubygem-qpid_messaging-1.38.0-1.el7.x86_64
tfm-rubygem-rabl-0.14.3-1.el7.noarch
tfm-rubygem-rack-2.2.3-1.el7.noarch
tfm-rubygem-rack-cors-1.0.2-2.el7.noarch
tfm-rubygem-rack-jsonp-1.3.1-9.el7.noarch
tfm-rubygem-rack-protection-2.0.3-4.el7.noarch
tfm-rubygem-rack-test-1.1.0-4.el7.noarch
tfm-rubygem-rails-6.0.3.4-1.el7.noarch
tfm-rubygem-rails-dom-testing-2.0.3-6.el7.noarch
tfm-rubygem-rails-html-sanitizer-1.3.0-1.el7.noarch
tfm-rubygem-rails-i18n-6.0.0-2.el7.noarch
tfm-rubygem-railties-6.0.3.4-1.el7.noarch
tfm-rubygem-rainbow-2.2.1-3.el7.noarch
tfm-rubygem-rake-compiler-1.0.7-3.el7.noarch
tfm-rubygem-rb-inotify-0.9.7-5.el7.noarch
tfm-rubygem-rbovirt-0.1.7-4.el7.noarch
tfm-rubygem-rbvmomi-2.2.0-3.el7.noarch
tfm-rubygem-record_tag_helper-1.0.1-3.el7.noarch
tfm-rubygem-redfish_client-0.5.2-1.el7.noarch
tfm-rubygem-redis-4.1.2-2.el7.noarch
tfm-rubygem-representable-3.0.4-2.el7.noarch
tfm-rubygem-responders-3.0.0-3.el7.noarch
tfm-rubygem-rest-client-2.0.2-3.el7.noarch
tfm-rubygem-retriable-3.1.2-2.el7.noarch
tfm-rubygem-rkerberos-0.1.5-19.el7.x86_64
tfm-rubygem-roadie-3.4.0-3.el7.noarch
tfm-rubygem-roadie-rails-2.1.1-2.el7.noarch
tfm-rubygem-robotex-1.0.0-21.el7.noarch
tfm-rubygem-rsec-0.4.3-4.el7.noarch
tfm-rubygem-ruby-libvirt-0.7.1-1.el7.x86_64
tfm-rubygem-ruby2ruby-2.4.2-3.el7.noarch
tfm-rubygem-ruby_parser-3.10.1-3.el7.noarch
tfm-rubygem-rubyipmi-0.10.0-6.el7.noarch
tfm-rubygem-runcible-2.13.1-1.el7.noarch
tfm-rubygem-safemode-1.3.6-1.el7.noarch
tfm-rubygem-scoped_search-4.1.9-1.el7.noarch
tfm-rubygem-sd_notify-0.1.0-1.el7.noarch
tfm-rubygem-secure_headers-6.3.0-2.el7.noarch
tfm-rubygem-sequel-5.7.1-3.el7.noarch
tfm-rubygem-server_sent_events-0.1.2-1.el7.noarch
tfm-rubygem-sexp_processor-4.10.0-6.el7.noarch
tfm-rubygem-sidekiq-5.2.7-3.el7.noarch
tfm-rubygem-signet-0.14.0-1.el7.noarch
tfm-rubygem-sinatra-2.0.3-4.el7.noarch
tfm-rubygem-smart_proxy_dhcp_infoblox-0.0.16-4.fm2_2.el7.noarch
tfm-rubygem-smart_proxy_dynflow-0.3.0-2.fm2_3.el7.noarch
tfm-rubygem-smart_proxy_dynflow_core-0.3.2-1.fm2_3.el7.noarch
tfm-rubygem-smart_proxy_pulp-2.1.0-3.fm2_2.el7.noarch
tfm-rubygem-smart_proxy_remote_execution_ssh-0.3.1-1.fm2_3.el7.noarch
tfm-rubygem-sprockets-4.0.2-1.el7.noarch
tfm-rubygem-sprockets-rails-3.2.1-6.el7.noarch
tfm-rubygem-sqlite3-1.3.13-6.el7.x86_64
tfm-rubygem-sshkey-1.9.0-4.el7.noarch
tfm-rubygem-statsd-instrument-2.1.4-3.el7.noarch
tfm-rubygem-stomp-1.4.9-1.el7.noarch
tfm-rubygem-thor-1.0.1-2.el7.noarch
tfm-rubygem-thread_safe-0.3.6-5.el7.noarch
tfm-rubygem-tilt-2.0.8-4.el7.noarch
tfm-rubygem-trollop-2.1.2-3.el7.noarch
tfm-rubygem-typhoeus-1.3.1-1.el7.noarch
tfm-rubygem-tzinfo-1.2.6-1.el7.noarch
tfm-rubygem-uber-0.1.0-2.el7.noarch
tfm-rubygem-unf-0.1.3-8.el7.noarch
tfm-rubygem-unf_ext-0.0.7.2-3.el7.x86_64
tfm-rubygem-unicode-0.4.4.4-3.el7.x86_64
tfm-rubygem-unicode-display_width-1.0.5-4.el7.noarch
tfm-rubygem-useragent-0.16.8-4.el7.noarch
tfm-rubygem-validates_lengths_from_database-0.5.0-7.el7.noarch
tfm-rubygem-webpack-rails-0.9.8-5.el7.noarch
tfm-rubygem-websocket-driver-0.7.1-1.el7.x86_64
tfm-rubygem-websocket-extensions-0.1.5-1.el7.noarch
tfm-rubygem-wicked-1.3.3-2.el7.noarch
tfm-rubygem-will_paginate-3.1.7-3.el7.noarch
tfm-rubygem-xmlrpc-0.3.0-2.el7.noarch
tfm-rubygem-zeitwerk-2.2.2-1.el7.noarch
tfm-rubygem-zest-0.0.4-1.el7.noarch
tfm-runtime-6.1-4.el7.x86_64
Distribution and version:
RHEL7
Other relevant data:
I am stuck at this stage after facing another problem beforehand (the “error 403” problem with the same task) that I could solve by restoring superuser flag for the admin user within Pulp3 (as mentioned in many other topics).
Now I am stuck with the above error and all solutions/workarounds I could find did not help.
What I have tried so far:
- Resetting the Pulp3 DB and starting over again:
sudo systemctl stop pulpcore* --all
sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' /usr/bin/pulpcore-manager shell -c "import pulpcore; pulpcore.app.models.ReservedResource.objects.all().delete()"
sudo systemctl restart pulpcore* --all
- Removing deb and docker support from Katello:
foreman-installer --foreman-proxy-content-enable-docker=false --katello-enable-docker=false
foreman-installer --katello-use-pulp-2-for-deb=false --katello-enable-deb=false --foreman-proxy-content-enable-deb=false
- Removing orphaned content:
foreman-rake katello:delete_orphaned_content
- Applying the Pulp migration patch from this pulp issue since I saw errors regarding unsupported zck metadata files in /var/log/messages. The issue also suggested to run
foreman-maintain content migration-resetafter applying the patch, but foreman-maintain told me that subcommand does not exist. - Reset the Pulp3 database again and restart once more
After all this, I am still at the same stage with the same error, and no clue how to continue from here. As a bonus, even after applying the mentioned Pulp patch, I am still getting errors regarding unsupported zck metadata, but it looks like they do not appear on every run.
May 19 16:07:43 myforeman pulpcore-worker-6: pulp: rq.worker:ERROR: Traceback (most recent call last):
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 19 16:07:43 myforeman pulpcore-worker-6: rv = job.perform()
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 19 16:07:43 myforeman pulpcore-worker-6: self._result = self._execute()
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 19 16:07:43 myforeman pulpcore-worker-6: return self.func(*self.args, **self.kwargs)
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 224, in complex_repo_migration
May 19 16:07:43 myforeman pulpcore-worker-6: migrated_repo.pulp3_repository_version
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 371, in migrate_repo_distributor
May 19 16:07:43 myforeman pulpcore-worker-6: pulp2dist, repo_version)
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 78, in migrate_to_pulp3
May 19 16:07:43 myforeman pulpcore-worker-6: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 320, in publish
May 19 16:07:43 myforeman pulpcore-worker-6: metadata_signing_service=metadata_signing_service,
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 551, in create_repomd_xml
May 19 16:07:43 myforeman pulpcore-worker-6: record.fill(checksum_type)
May 19 16:07:43 myforeman pulpcore-worker-6: OSError: Error while computing stat of compressed content of updateinfo.xml.zck:Cannot open a file updateinfo.xml.zck: createrepo_c wasn't compiled with zchunk support
May 19 16:07:43 myforeman pulpcore-worker-6: Traceback (most recent call last):
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 19 16:07:43 myforeman pulpcore-worker-6: rv = job.perform()
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 19 16:07:43 myforeman pulpcore-worker-6: self._result = self._execute()
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 19 16:07:43 myforeman pulpcore-worker-6: return self.func(*self.args, **self.kwargs)
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 224, in complex_repo_migration
May 19 16:07:43 myforeman pulpcore-worker-6: migrated_repo.pulp3_repository_version
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 371, in migrate_repo_distributor
May 19 16:07:43 myforeman pulpcore-worker-6: pulp2dist, repo_version)
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 78, in migrate_to_pulp3
May 19 16:07:43 myforeman pulpcore-worker-6: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 320, in publish
May 19 16:07:43 myforeman pulpcore-worker-6: metadata_signing_service=metadata_signing_service,
May 19 16:07:43 myforeman pulpcore-worker-6: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 551, in create_repomd_xml
May 19 16:07:43 myforeman pulpcore-worker-6: record.fill(checksum_type)
May 19 16:07:43 myforeman pulpcore-worker-6: OSError: Error while computing stat of compressed content of updateinfo.xml.zck:Cannot open a file updateinfo.xml.zck: createrepo_c wasn't compiled with zchunk support
May 19 16:07:43 myforeman pulpcore-worker-6: pulp: rq.worker:INFO: 52506@myforeman.example.com: 17bd3c6e-5826-4c55-bceb-f693964c075b
May 19 16:07:43 myforeman pulpcore-worker-6: pulp: rq.worker:INFO: 52506@myforeman.example.com: Job OK (17bd3c6e-5826-4c55-bceb-f693964c075b)
An adittional error I am seeing on every run of foreman-maintain content prepare is this one (repeated several time, probably once per affected repo):
May 19 16:07:37 myforeman journal: cr_repomd_record_fill: File icons.tar.gz doesn't exists
May 19 16:07:37 myforeman pulpcore-worker-1: pulp: rq.worker:ERROR: Traceback (most recent call last):
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 19 16:07:37 myforeman pulpcore-worker-1: rv = job.perform()
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 19 16:07:37 myforeman pulpcore-worker-1: self._result = self._execute()
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 19 16:07:37 myforeman pulpcore-worker-1: return self.func(*self.args, **self.kwargs)
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 224, in complex_repo_migration
May 19 16:07:37 myforeman pulpcore-worker-1: migrated_repo.pulp3_repository_version
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 371, in migrate_repo_distributor
May 19 16:07:37 myforeman pulpcore-worker-1: pulp2dist, repo_version)
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 78, in migrate_to_pulp3
May 19 16:07:37 myforeman pulpcore-worker-1: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 320, in publish
May 19 16:07:37 myforeman pulpcore-worker-1: metadata_signing_service=metadata_signing_service,
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 551, in create_repomd_xml
May 19 16:07:37 myforeman pulpcore-worker-1: record.fill(checksum_type)
May 19 16:07:37 myforeman pulpcore-worker-1: OSError: File icons.tar.gz doesn't exists or not a regular file
May 19 16:07:37 myforeman pulpcore-worker-1: Traceback (most recent call last):
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
May 19 16:07:37 myforeman pulpcore-worker-1: rv = job.perform()
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
May 19 16:07:37 myforeman pulpcore-worker-1: self._result = self._execute()
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
May 19 16:07:37 myforeman pulpcore-worker-1: return self.func(*self.args, **self.kwargs)
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 224, in complex_repo_migration
May 19 16:07:37 myforeman pulpcore-worker-1: migrated_repo.pulp3_repository_version
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 371, in migrate_repo_distributor
May 19 16:07:37 myforeman pulpcore-worker-1: pulp2dist, repo_version)
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/repository.py", line 78, in migrate_to_pulp3
May 19 16:07:37 myforeman pulpcore-worker-1: publish(repo_version.pk, checksum_types=checksum_types, sqlite_metadata=sqlite)
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 320, in publish
May 19 16:07:37 myforeman pulpcore-worker-1: metadata_signing_service=metadata_signing_service,
May 19 16:07:37 myforeman pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 551, in create_repomd_xml
May 19 16:07:37 myforeman pulpcore-worker-1: record.fill(checksum_type)
May 19 16:07:37 myforeman pulpcore-worker-1: OSError: File icons.tar.gz doesn't exists or not a regular file
May 19 16:07:37 myforeman pulpcore-worker-1: pulp: rq.worker:INFO: 52510@myforeman.example.com: ca43cbc1-cd67-46ac-a9aa-91ccc7bd05e9
May 19 16:07:38 myforeman pulpcore-worker-1: pulp: rq.worker:INFO: 52510@myforeman.example.com: Job OK (ca43cbc1-cd67-46ac-a9aa-91ccc7bd05e9)
Plus, I am also getting errors claiming some rpm files would be corrupt (example):
May 19 16:06:14 myforeman pulpcore-worker-2: pulp: pulp_2to3_migration.app.plugin.content:WARNING: The content located in /var/lib/pulp/content/units/rpm/f2/ae2bd661555fe6b1866216b5746bf6cb054fe067d79aa38f6fa385ed1fdfc5/rh-perl524-perl-CPAN-2.11-379.el7.noarch.rpm is missing or corrupted. It was skipped during Pulp 2t
o3 migration.
Yet, on manuel examination of the file they seem to be just fine rpm files:
# rpm -qip /var/lib/pulp/content/units/rpm/f2/ae2bd661555fe6b1866216b5746bf6cb054fe067d79aa38f6fa385ed1fdfc5/rh-perl524-perl-CPAN-2.11-379.el7.noarch.rpm
Name : rh-perl524-perl-CPAN
Epoch : 0
Version : 2.11
Release : 379.el7
Architecture: noarch
Install Date: (not installed)
Group : Development/Languages
Size : 824208
License : GPL+ or Artistic
Signature : (none)
Source RPM : rh-perl524-perl-5.24.0-379.el7.src.rpm
Build Date : Sat 22 Oct 2016 11:29:35 AM CEST
Build Host : c1be.rdu2.centos.org
Relocations : (not relocatable)
Packager : CBS <cbs@centos.org>
Vendor : CentOS
URL : http://www.perl.org/
Summary : Query, download and build perl modules from CPAN sites
Description :
The CPAN module automates or at least simplifies the make and install of
perl modules and extensions. It includes some primitive searching
capabilities and knows how to use LWP, HTTP::Tiny, Net::FTP and certain
external download clients to fetch distributions from the net.
Any pointers to what to try out would be greatly appreciated!
If you need any more information/logs, let me know.
Regards
