Pulp 3 Migration fails after upgrade to Katello 3.17

Support
1

Problem:
I have attempted the pulp migration the last several upgrades, but it never succeeds. After the upgrade from Katello 3.16 to 3.17, I attempted the pulp 3 migration. I got the following error:

 foreman-maintain content switchover

Running Switch support for certain content from Pulp 2 to Pulp 3

Switch support for certain content from Pulp 2 to Pulp 3:
Performing final content migration before switching content
[FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 1:
Migration failed, You will want to investigate: https://katellod02p.d.carfax.us/foreman_tasks/tasks/311e7c0a-4f6b-48c2-ace0-9b2b28bac636
rake aborted!
ForemanTasks::TaskError: Task 311e7c0a-4f6b-48c2-ace0-9b2b28bac636: Katello::Errors::Pulp3Error: invalid literal for int() with base 10: ‘1601956550.79’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.17.1/lib/katello/tasks/pulp3_migration.rake:17:in block (2 levels) in <top (required)>' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in <top (required)>’
Tasks: TOP => katello:pulp3_migration
(See full trace by running task with --trace)

Scenario [Switch support for certain content from Pulp 2 to Pulp 3] failed.

The following steps ended up in failing state:

[content-switchover]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist=“content-switchover”

Expected outcome:
Successful migration

Foreman and Proxy versions:
Foreman 2.2.2
Katello 2.17

Foreman and Proxy plugin versions:
nstalled Packages

candlepin-3.1.22-1.el7.noarch
candlepin-selinux-3.1.22-1.el7.noarch
foreman-2.2.2-1.el7.noarch
foreman-cli-2.2.2-1.el7.noarch
foreman-debug-2.2.2-1.el7.noarch
foreman-dynflow-sidekiq-2.2.2-1.el7.noarch
foreman-ec2-2.2.2-1.el7.noarch
foreman-installer-2.2.2-1.el7.noarch
foreman-installer-katello-2.2.2-1.el7.noarch
foreman-postgresql-2.2.2-1.el7.noarch
foreman-proxy-2.2.2-1.el7.noarch
foreman-release-2.2.2-1.el7.noarch
foreman-release-scl-7-3.el7.noarch
foreman-selinux-2.2.2-1.el7.noarch
foreman-service-2.2.2-1.el7.noarch
foreman-vmware-2.2.2-1.el7.noarch
katello-3.17.1-1.el7.noarch
katello-ca-consumer-katrepod02p.d.carfax.us-1.0-3.noarch
katello-certs-tools-2.7.1-2.el7.noarch
katello-client-bootstrap-1.7.5-1.el7.noarch
katello-common-3.17.1-1.el7.noarch
katello-debug-3.17.1-1.el7.noarch
katello-default-ca-1.0-1.noarch
katello-host-tools-fact-plugin-3.0.0-3.el7.noarch
katello-repos-3.17.1-2.el7.noarch
katello-selinux-3.4.0-1.el7.noarch
katello-server-ca-1.0-4.noarch
katello-service-3.14.1-1.el7.noarch
katellod02p.d.carfax.us-apache-1.0-4.noarch
katellod02p.d.carfax.us-foreman-client-1.0-1.noarch
katellod02p.d.carfax.us-foreman-proxy-1.0-2.noarch
katellod02p.d.carfax.us-foreman-proxy-client-1.0-1.noarch
katellod02p.d.carfax.us-qpid-broker-1.0-2.noarch
katellod02p.d.carfax.us-qpid-client-cert-1.0-1.noarch
katellod02p.d.carfax.us-qpid-router-client-1.0-1.noarch
katellod02p.d.carfax.us-qpid-router-server-1.0-1.noarch
katellod02p.d.carfax.us-tomcat-1.0-1.noarch
pulp-admin-client-2.21.5-1.el7.noarch
pulp-client-1.0-1.noarch
pulp-deb-plugins-1.10.2-1.el7.noarch
pulp-docker-plugins-3.2.8-1.el7.noarch
pulp-katello-1.0.3-1.el7.noarch
pulp-puppet-plugins-2.21.5-1.el7.noarch
pulp-puppet-tools-2.21.5-1.el7.noarch
pulp-rpm-handlers-2.21.5-1.el7.noarch
pulp-rpm-plugins-2.21.5-1.el7.noarch
pulp-selinux-2.21.5-1.el7.noarch
pulp-server-2.21.5-1.el7.noarch
pulpcore-selinux-1.2.3-2.el7.x86_64
python-gofer-qpid-2.12.5-3.el7.noarch
python-pulp-agent-lib-2.21.5-1.el7.noarch
python-pulp-bindings-2.21.5-1.el7.noarch
python-pulp-client-lib-2.21.5-1.el7.noarch
python-pulp-common-2.21.5-1.el7.noarch
python-pulp-deb-common-1.10.2-1.el7.noarch
python-pulp-docker-common-3.2.8-1.el7.noarch
python-pulp-oid_validation-2.21.5-1.el7.noarch
python-pulp-puppet-common-2.21.5-1.el7.noarch
python-pulp-repoauth-2.21.5-1.el7.noarch
python-pulp-rpm-common-2.21.5-1.el7.noarch
python-pulp-streamer-2.21.5-1.el7.noarch
python2-qpid-1.37.0-5.el7.noarch
python2-qpid-proton-0.32.0-2.el7.x86_64
python2-qpid-qmf-1.39.0-1.el7.x86_64
python3-pulp-2to3-migration-0.6.0-2.el7.noarch
python3-pulp-certguard-1.0.3-1.el7.noarch
python3-pulp-container-2.1.0-1.el7.noarch
python3-pulp-file-1.3.0-1.el7.noarch
python3-pulp-rpm-3.7.0-1.el7.noarch
python3-pulpcore-3.7.3-1.el7.noarch
qpid-cpp-client-1.39.0-1.el7.x86_64
qpid-cpp-client-devel-1.39.0-1.el7.x86_64
qpid-cpp-server-1.39.0-1.el7.x86_64
qpid-cpp-server-linearstore-1.39.0-1.el7.x86_64
qpid-dispatch-router-1.14.0-1.el7.x86_64
qpid-proton-c-0.32.0-2.el7.x86_64
qpid-qmf-1.39.0-1.el7.x86_64
qpid-tools-1.39.0-1.el7.noarch
rubygem-foreman_maintain-0.7.1-1.el7.noarch
tfm-rubygem-actioncable-6.0.3.1-1.el7.noarch
tfm-rubygem-actionmailbox-6.0.3.1-1.el7.noarch
tfm-rubygem-actionmailer-6.0.3.1-1.el7.noarch
tfm-rubygem-actionpack-6.0.3.1-1.el7.noarch
tfm-rubygem-actiontext-6.0.3.1-1.el7.noarch
tfm-rubygem-actionview-6.0.3.1-1.el7.noarch
tfm-rubygem-activejob-6.0.3.1-1.el7.noarch
tfm-rubygem-activemodel-6.0.3.1-1.el7.noarch
tfm-rubygem-activerecord-6.0.3.1-1.el7.noarch
tfm-rubygem-activerecord-import-1.0.0-2.el7.noarch
tfm-rubygem-activerecord-session_store-1.1.1-4.el7.noarch
tfm-rubygem-activestorage-6.0.3.1-1.el7.noarch
tfm-rubygem-activesupport-6.0.3.1-1.el7.noarch
tfm-rubygem-addressable-2.6.0-2.el7.noarch
tfm-rubygem-algebrick-0.7.3-7.el7.noarch
tfm-rubygem-amazing_print-1.1.0-1.el7.noarch
tfm-rubygem-ancestry-3.0.7-1.el7.noarch
tfm-rubygem-anemone-0.7.2-17.el7.noarch
tfm-rubygem-angular-rails-templates-1.1.0-1.el7.noarch
tfm-rubygem-ansi-1.5.0-2.el7.noarch
tfm-rubygem-apipie-bindings-0.4.0-1.el7.noarch
tfm-rubygem-apipie-dsl-2.2.9-1.el7.noarch
tfm-rubygem-apipie-params-0.0.5-4.el7.noarch
tfm-rubygem-apipie-rails-0.5.17-3.el7.noarch
tfm-rubygem-arel-9.0.0-4.el7.noarch
tfm-rubygem-audited-4.9.0-3.el7.noarch
tfm-rubygem-awesome_print-1.8.0-5.el7.noarch
tfm-rubygem-bcrypt-3.1.12-3.el7.x86_64
tfm-rubygem-builder-3.2.4-1.el7.noarch
tfm-rubygem-bundler_ext-0.4.1-5.el7.noarch
tfm-rubygem-chef-api-0.6.0-4.el7.noarch
tfm-rubygem-clamp-1.1.2-6.el7.noarch
tfm-rubygem-coffee-rails-5.0.0-1.el7.noarch
tfm-rubygem-coffee-script-2.4.1-4.el7.noarch
tfm-rubygem-coffee-script-source-1.12.2-4.el7.noarch
tfm-rubygem-concurrent-ruby-1.1.6-2.el7.noarch
tfm-rubygem-concurrent-ruby-edge-0.6.0-2.fm2_1.el7.noarch
tfm-rubygem-connection_pool-2.2.2-2.el7.noarch
tfm-rubygem-crass-1.0.6-1.el7.noarch
tfm-rubygem-css_parser-1.4.7-4.el7.noarch
tfm-rubygem-daemons-1.2.3-6.el7.noarch
tfm-rubygem-deacon-1.0.0-4.el7.noarch
tfm-rubygem-declarative-0.0.10-2.el7.noarch
tfm-rubygem-declarative-option-0.1.0-2.el7.noarch
tfm-rubygem-deep_cloneable-3.0.0-3.el7.noarch
tfm-rubygem-deface-1.5.3-2.el7.noarch
tfm-rubygem-domain_name-0.5.20160310-4.el7.noarch
tfm-rubygem-dynflow-1.4.7-1.fm2_2.el7.noarch
tfm-rubygem-erubi-1.9.0-1.el7.noarch
tfm-rubygem-ethon-0.12.0-1.el7.noarch
tfm-rubygem-excon-0.76.0-1.el7.noarch
tfm-rubygem-execjs-2.7.0-4.el7.noarch
tfm-rubygem-facter-2.4.0-7.el7.x86_64
tfm-rubygem-faraday-0.15.4-2.el7.noarch
tfm-rubygem-fast_gettext-1.4.1-4.el7.noarch
tfm-rubygem-ffi-1.12.2-1.el7.x86_64
tfm-rubygem-fog-aws-3.6.5-1.el7.noarch
tfm-rubygem-fog-core-2.1.0-3.el7.noarch
tfm-rubygem-fog-digitalocean-0.4.0-2.el7.noarch
tfm-rubygem-fog-google-1.8.2-2.el7.noarch
tfm-rubygem-fog-json-1.2.0-3.el7.noarch
tfm-rubygem-fog-openstack-1.0.8-3.el7.noarch
tfm-rubygem-fog-ovirt-1.2.5-1.el7.noarch
tfm-rubygem-fog-vsphere-3.4.0-1.el7.noarch
tfm-rubygem-fog-xenserver-1.0.0-2.el7.noarch
tfm-rubygem-fog-xml-0.1.2-8.el7.noarch
tfm-rubygem-foreman-tasks-3.0.1-1.fm2_2.el7.noarch
tfm-rubygem-foreman-tasks-core-0.3.4-1.fm2_1.el7.noarch
tfm-rubygem-foreman_chef-0.10.0-1.fm2_2.el7.noarch
tfm-rubygem-foreman_discovery-16.2.0-1.fm2_2.el7.noarch
tfm-rubygem-foreman_docker-5.0.0-3.fm2_1.el7.noarch
tfm-rubygem-foreman_remote_execution-4.1.0-1.fm2_2.el7.noarch
tfm-rubygem-foreman_remote_execution_core-1.3.1-1.el7.noarch
tfm-rubygem-formatador-0.2.1-12.el7.noarch
tfm-rubygem-friendly_id-5.3.0-1.el7.noarch
tfm-rubygem-fx-0.5.0-1.el7.noarch
tfm-rubygem-get_process_mem-0.2.1-4.el7.noarch
tfm-rubygem-gettext_i18n_rails-1.8.0-2.el7.noarch
tfm-rubygem-gitlab-sidekiq-fetcher-0.6.0-1.el7.noarch
tfm-rubygem-globalid-0.4.2-1.el7.noarch
tfm-rubygem-google-api-client-0.23.9-3.el7.noarch
tfm-rubygem-googleauth-0.6.7-3.el7.noarch
tfm-rubygem-graphql-1.8.14-2.el7.noarch
tfm-rubygem-graphql-batch-0.3.10-2.el7.noarch
tfm-rubygem-gssapi-1.2.0-7.el7.noarch
tfm-rubygem-hammer_cli-2.2.1-1.el7.noarch
tfm-rubygem-hammer_cli_foreman-2.2.0-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_bootdisk-0.3.0-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_discovery-1.0.2-1.fm2_1.el7.noarch
tfm-rubygem-hammer_cli_foreman_docker-0.0.7-1.el7.noarch
tfm-rubygem-hammer_cli_foreman_remote_execution-0.1.2-1.fm2_2.el7.noarch
tfm-rubygem-hammer_cli_foreman_tasks-0.0.15-1.fm2_2.el7.noarch
tfm-rubygem-hammer_cli_katello-0.23.2-1.el7.noarch
tfm-rubygem-hashie-3.6.0-2.el7.noarch
tfm-rubygem-highline-1.7.8-5.el7.noarch
tfm-rubygem-http-cookie-1.0.2-4.el7.noarch
tfm-rubygem-httpclient-2.8.3-3.el7.noarch
tfm-rubygem-i18n-1.8.2-1.el7.noarch
tfm-rubygem-ipaddress-0.8.0-12.el7.noarch
tfm-rubygem-jwt-2.2.1-2.el7.noarch
tfm-rubygem-kafo-5.0.1-1.el7.noarch
tfm-rubygem-kafo_parsers-1.1.0-3.el7.noarch
tfm-rubygem-kafo_wizards-0.0.1-4.el7.noarch
tfm-rubygem-katello-3.17.1-1.el7.noarch
tfm-rubygem-ldap_fluff-0.4.7-5.el7.noarch
tfm-rubygem-little-plugger-1.1.4-2.el7.noarch
tfm-rubygem-locale-2.0.9-14.el7.noarch
tfm-rubygem-logging-2.2.2-5.el7.noarch
tfm-rubygem-logify-0.2.0-3.el7.noarch
tfm-rubygem-loofah-2.4.0-1.el7.noarch
tfm-rubygem-mail-2.7.1-1.el7.noarch
tfm-rubygem-marcel-0.3.3-1.el7.noarch
tfm-rubygem-memoist-0.16.0-2.el7.noarch
tfm-rubygem-method_source-0.9.2-1.el7.noarch
tfm-rubygem-mime-types-3.2.2-4.el7.noarch
tfm-rubygem-mime-types-data-3.2018.0812-4.el7.noarch
tfm-rubygem-mimemagic-0.3.5-1.el7.noarch
tfm-rubygem-mini_mime-1.0.2-1.el7.noarch
tfm-rubygem-mini_portile2-2.4.0-1.el7.noarch
tfm-rubygem-multi_json-1.14.1-2.el7.noarch
tfm-rubygem-multipart-post-2.0.0-2.el7.noarch
tfm-rubygem-mustermann-1.0.2-4.el7.noarch
tfm-rubygem-net-ldap-0.16.1-2.el7.noarch
tfm-rubygem-net-ping-2.0.1-4.el7.noarch
tfm-rubygem-net-scp-1.2.1-4.el7.noarch
tfm-rubygem-net-ssh-4.2.0-2.el7.noarch
tfm-rubygem-netrc-0.11.0-5.el7.noarch
tfm-rubygem-nio4r-2.5.2-1.el7.x86_64
tfm-rubygem-nokogiri-1.10.9-1.el7.x86_64
tfm-rubygem-oauth-0.5.4-4.el7.noarch
tfm-rubygem-optimist-3.0.0-2.el7.noarch
tfm-rubygem-os-1.0.0-2.el7.noarch
tfm-rubygem-ovirt-engine-sdk-4.3.0-1.el7.x86_64
tfm-rubygem-parse-cron-0.1.4-4.fm2_1.el7.noarch
tfm-rubygem-passenger-4.0.18-10.13.el7.x86_64
tfm-rubygem-passenger-native-4.0.18-10.13.el7.x86_64
tfm-rubygem-passenger-native-libs-4.0.18-10.13.el7.x86_64
tfm-rubygem-pg-1.1.4-3.el7.x86_64
tfm-rubygem-polyglot-0.3.5-2.el7.noarch
tfm-rubygem-powerbar-2.0.1-2.el7.noarch
tfm-rubygem-promise.rb-0.7.4-2.el7.noarch
tfm-rubygem-public_suffix-3.0.3-2.el7.noarch
tfm-rubygem-pulp_2to3_migration_client-0.5.0-1.el7.noarch
tfm-rubygem-pulp_ansible_client-0.3.0-1.el7.noarch
tfm-rubygem-pulp_certguard_client-1.0.2-1.el7.noarch
tfm-rubygem-pulp_container_client-2.0.0-1.el7.noarch
tfm-rubygem-pulp_docker_client-4.0.0b6.dev01562331743-1.el7.noarch
tfm-rubygem-pulp_file_client-1.2.0-1.el7.noarch
tfm-rubygem-pulp_rpm_client-3.6.2-1.el7.noarch
tfm-rubygem-pulpcore_client-3.6.0-1.el7.noarch
tfm-rubygem-puma-4.3.3-4.el7.x86_64
tfm-rubygem-qpid_messaging-1.38.0-1.el7.x86_64
tfm-rubygem-rabl-0.14.3-1.el7.noarch
tfm-rubygem-rack-2.2.3-1.el7.noarch
tfm-rubygem-rack-cors-1.0.2-2.el7.noarch
tfm-rubygem-rack-jsonp-1.3.1-9.el7.noarch
tfm-rubygem-rack-protection-2.0.3-4.el7.noarch
tfm-rubygem-rack-test-1.1.0-4.el7.noarch
tfm-rubygem-rails-6.0.3.1-1.el7.noarch
tfm-rubygem-rails-dom-testing-2.0.3-6.el7.noarch
tfm-rubygem-rails-html-sanitizer-1.3.0-1.el7.noarch
tfm-rubygem-rails-i18n-6.0.0-2.el7.noarch
tfm-rubygem-railties-6.0.3.1-1.el7.noarch
tfm-rubygem-rainbow-2.2.1-3.el7.noarch
tfm-rubygem-rake-compiler-1.0.7-3.el7.noarch
tfm-rubygem-rb-inotify-0.9.7-5.el7.noarch
tfm-rubygem-rbovirt-0.1.7-4.el7.noarch
tfm-rubygem-rbvmomi-2.2.0-3.el7.noarch
tfm-rubygem-record_tag_helper-1.0.1-3.el7.noarch
tfm-rubygem-redis-4.1.2-2.el7.noarch
tfm-rubygem-representable-3.0.4-2.el7.noarch
tfm-rubygem-responders-3.0.0-3.el7.noarch
tfm-rubygem-rest-client-2.0.2-3.el7.noarch
tfm-rubygem-retriable-3.1.2-2.el7.noarch
tfm-rubygem-rkerberos-0.1.5-19.el7.x86_64
tfm-rubygem-roadie-3.4.0-3.el7.noarch
tfm-rubygem-roadie-rails-2.1.1-2.el7.noarch
tfm-rubygem-robotex-1.0.0-21.el7.noarch
tfm-rubygem-rsec-0.4.3-4.el7.noarch
tfm-rubygem-ruby-libvirt-0.7.1-1.el7.x86_64
tfm-rubygem-ruby2ruby-2.4.2-3.el7.noarch
tfm-rubygem-ruby_parser-3.10.1-3.el7.noarch
tfm-rubygem-rubyipmi-0.10.0-6.el7.noarch
tfm-rubygem-runcible-2.13.1-1.el7.noarch
tfm-rubygem-safemode-1.3.5-3.el7.noarch
tfm-rubygem-scoped_search-4.1.9-1.el7.noarch
tfm-rubygem-secure_headers-6.3.0-2.el7.noarch
tfm-rubygem-sequel-5.7.1-3.el7.noarch
tfm-rubygem-sexp_processor-4.10.0-6.el7.noarch
tfm-rubygem-sidekiq-5.2.7-3.el7.noarch
tfm-rubygem-signet-0.11.0-3.el7.noarch
tfm-rubygem-sinatra-2.0.3-4.el7.noarch
tfm-rubygem-smart_proxy_chef-0.2.0-6.fm2_2.el7.noarch
tfm-rubygem-smart_proxy_dynflow-0.2.4-6.fm2_2.el7.noarch
tfm-rubygem-smart_proxy_dynflow_core-0.2.6-1.fm2_2.el7.noarch
tfm-rubygem-smart_proxy_pulp-2.1.0-3.fm2_2.el7.noarch
tfm-rubygem-smart_proxy_remote_execution_ssh-0.3.0-4.fm2_2.el7.noarch
tfm-rubygem-sprockets-4.0.2-1.el7.noarch
tfm-rubygem-sprockets-rails-3.2.1-6.el7.noarch
tfm-rubygem-sqlite3-1.3.13-6.el7.x86_64
tfm-rubygem-sshkey-1.9.0-4.el7.noarch
tfm-rubygem-statsd-instrument-2.1.4-3.el7.noarch
tfm-rubygem-stomp-1.4.9-1.el7.noarch
tfm-rubygem-thor-1.0.1-2.el7.noarch
tfm-rubygem-thread_safe-0.3.6-5.el7.noarch
tfm-rubygem-tilt-2.0.8-4.el7.noarch
tfm-rubygem-trollop-2.1.2-3.el7.noarch
tfm-rubygem-typhoeus-1.3.1-1.el7.noarch
tfm-rubygem-tzinfo-1.2.6-1.el7.noarch
tfm-rubygem-uber-0.1.0-2.el7.noarch
tfm-rubygem-unf-0.1.3-8.el7.noarch
tfm-rubygem-unf_ext-0.0.7.2-3.el7.x86_64
tfm-rubygem-unicode-0.4.4.4-3.el7.x86_64
tfm-rubygem-unicode-display_width-1.0.5-4.el7.noarch
tfm-rubygem-validates_lengths_from_database-0.5.0-7.el7.noarch
tfm-rubygem-webpack-rails-0.9.8-5.el7.noarch
tfm-rubygem-websocket-driver-0.7.1-1.el7.x86_64
tfm-rubygem-websocket-extensions-0.1.5-1.el7.noarch
tfm-rubygem-wicked-1.3.3-2.el7.noarch
tfm-rubygem-will_paginate-3.1.7-3.el7.noarch
tfm-rubygem-x-editable-rails-1.5.5-5.el7.noarch
tfm-rubygem-xmlrpc-0.3.0-2.el7.noarch
tfm-rubygem-zeitwerk-2.2.2-1.el7.noarch
tfm-rubygem-zest-0.0.4-1.el7.noarch
tfm-runtime-6.1-3.el7.x86_64

Distribution and version:
CentOS Linux release 7.9.2009 (Core)
su
Other relevant data:

2

It looks similar to [ContentMigration] migration fails with error "Katello::Errors::Pulp3Error: 'NoneType' object has no attribute 'cast'" - the error message is different but they may be related

@Justin_Sherrill any thoughts on further debugging for this one as well?

3

@tbrooks It appears that this is a traceback in pulp3, could you take a look in journalctl or /var/log/messages and look for a message ’ invalid literal for int()’ and then look for a traceback around it? That would help narrow down the problem.

I would also recommend upgrade to 3.18 before continuing. We plan on targeting all of the migration fixes to that release. Thanks!

1 Like
4

Unfortunately, it has rolled off, so not able to see what happened. I attempted to the upgrade to 3.18, and I got Success! when it finished, but it failed the following tasks. I’m not able to see in the production.log or /var/log/foreman-installer/katello.log which repo is causing the issue.

2021-02-02 11:34:47 [DEBUG ] [root] Failed upgrade task: katello:correct_repositories, see logs for more information.
2021-02-02 11:34:47 [DEBUG ] [root] =============================================
2021-02-02 11:34:47 [DEBUG ] [root] Upgrade Step 2/3: katello:correct_puppet_environments. This may take a long while.
2021-02-02 11:34:47 [DEBUG ] [root] Processing Puppet Environment 1/193: 1-carfax_default_cv-v1_0-puppet-142cc4dd-7d00-480a-862a-c6601759b811 (1)
2021-02-02 11:34:47 [DEBUG ] [root] Failed upgrade task: katello:correct_puppet_environments, see logs for more information.

Here is the backtrace for the first error:
2021-02-02T11:34:36 [W|app|] Failed upgrade task: katello:correct_repositories
2021-02-02T11:34:36 [I|app|] Backtrace for ‘Failed upgrade task: katello:correct_repositories’ error (NoMethodError): undefined method repository_href' for nil:NilClass | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/repository.rake:144:in repo_exists?’
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/repository.rake:64:in block (3 levels) in <top (required)>' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:70:in block (2 levels) in find_each’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:70:in each' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:70:in block in find_each’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:136:in block in find_in_batches' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:238:in block in in_batches’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:222:in loop' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:222:in in_batches’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:135:in find_in_batches' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:69:in find_each’
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/repository.rake:62:in with_index' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/repository.rake:62:in block (2 levels) in <top (required)>’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in block in execute' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in each’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in execute' | /usr/share/foreman/lib/tasks/upgrade.rake:32:in run_task’
| /usr/share/foreman/lib/tasks/upgrade.rake:27:in block (3 levels) in <top (required)>' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/delegation.rb:87:in each’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/delegation.rb:87:in each' | /usr/share/foreman/lib/tasks/upgrade.rake:19:in each_with_index’
| /usr/share/foreman/lib/tasks/upgrade.rake:19:in block (2 levels) in <top (required)>' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in block in execute’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in each' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in execute’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:195:in block in invoke_with_call_chain' | /opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in mon_synchronize’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:188:in invoke_with_call_chain' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:181:in invoke’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:160:in invoke_task' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in block (2 levels) in top_level’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in each' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in block in top_level’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:125:in run_with_threads' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:110:in top_level’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:83:in block in run' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:186:in standard_exception_handling’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:80:in run' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in <top (required)>’
| /opt/rh/rh-ruby25/root/usr/bin/rake:23:in load' | /opt/rh/rh-ruby25/root/usr/bin/rake:23:in ’

And the backtrace for correct_puppet_environments:

2021-02-02T11:34:36 [W|app|] Failed upgrade task: katello:correct_puppet_environments

2021-02-02T11:34:36 [I|app|] Backtrace for ‘Failed upgrade task: katello:correct_puppet_environments’ error (RuntimeError): Cannot find repository type puppet, is it enabled?
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/models/katello/concerns/smart_proxy_extensions.rb:192:in pulp3_repository_type_support?' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/app/models/katello/concerns/smart_proxy_extensions.rb:165:in pulp3_support?’
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/repository.rake:142:in repo_exists?' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/repository.rake:83:in block (3 levels) in <top (required)>’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:70:in block (2 levels) in find_each' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:70:in each’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:70:in block in find_each' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:136:in block in find_in_batches’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:238:in block in in_batches' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:222:in loop’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:222:in in_batches' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:135:in find_in_batches’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/batches.rb:69:in find_each' | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/repository.rake:81:in with_index’
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/repository.rake:81:in block (2 levels) in <top (required)>' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in block in execute’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in each' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in execute’
| /usr/share/foreman/lib/tasks/upgrade.rake:32:in run_task' | /usr/share/foreman/lib/tasks/upgrade.rake:27:in block (3 levels) in <top (required)>’
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/delegation.rb:87:in each' | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/delegation.rb:87:in each’
| /usr/share/foreman/lib/tasks/upgrade.rake:19:in each_with_index' | /usr/share/foreman/lib/tasks/upgrade.rake:19:in block (2 levels) in <top (required)>’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in block in execute' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in each’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in execute' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:195:in block in invoke_with_call_chain’
| /opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in mon_synchronize' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:188:in invoke_with_call_chain’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:181:in invoke' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:160:in invoke_task’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in block (2 levels) in top_level' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in each’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in block in top_level' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:125:in run_with_threads’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:110:in top_level' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:83:in block in run’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:186:in standard_exception_handling' | /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:80:in run’
| /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in <top (required)>' | /opt/rh/rh-ruby25/root/usr/bin/rake:23:in load’
| /opt/rh/rh-ruby25/root/usr/bin/rake:23:in `’

Not sure if I should continue to the pulp migration or not.

5

can you provide your /etc/foreman/plugins/katello.yaml file? (feel free to hide any sensitive info)

I’d also be curious to look at a foreman-debug. I’ll pm you an email address you can send it if possible.

6

Thanks for the email Justin. I just sent you a link for the foreman-debug as well as the yaml file contents.

7

I ran the migration again after the upgrade to 3.18. Here is the traceback for that error:

Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 819, in getint
Feb  2 16:34:30 katellod02p pulpcore-worker-1: fallback=fallback, **kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 809, in _get_conv
Feb  2 16:34:30 katellod02p pulpcore-worker-1: **kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 803, in _get
Feb  2 16:34:30 katellod02p pulpcore-worker-1: return conv(self.get(section, option, **kwargs))
Feb  2 16:34:30 katellod02p pulpcore-worker-1: ValueError: invalid literal for int() with base 10: '1601956550.79'
Feb  2 16:34:30 katellod02p pulpcore-worker-1: During handling of the above exception, another exception occurred:
Feb  2 16:34:30 katellod02p pulpcore-worker-1: Traceback (most recent call last):
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Feb  2 16:34:30 katellod02p pulpcore-worker-1: rv = job.perform()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self._result = self._execute()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Feb  2 16:34:30 katellod02p pulpcore-worker-1: return self.func(*self.args, **self.kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 141, in migrate_from_pulp2
Feb  2 16:34:30 katellod02p pulpcore-worker-1: migrate_content(plan, skip_corrupted=skip_corrupted)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 47, in migrate_content
Feb  2 16:34:30 katellod02p pulpcore-worker-1: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 145, in migrate_content_to_pulp3
Feb  2 16:34:30 katellod02p pulpcore-worker-1: loop.run_until_complete(dm.create())
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Feb  2 16:34:30 katellod02p pulpcore-worker-1: return future.result()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 90, in create
Feb  2 16:34:30 katellod02p pulpcore-worker-1: await pipeline
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Feb  2 16:34:30 katellod02p pulpcore-worker-1: await asyncio.gather(*futures)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Feb  2 16:34:30 katellod02p pulpcore-worker-1: await self.run()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 182, in run
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.migrate_to_pulp3(cmodel, ctype)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 277, in migrate_to_pulp3
Feb  2 16:34:30 katellod02p pulpcore-worker-1: pulp3content, extra_info = pulp_2to3_detail_content.create_pulp3_content()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/pulp_2to3_models.py", line 774, in create_pulp3_content
Feb  2 16:34:30 katellod02p pulpcore-worker-1: treeinfo_serialized = self.get_treeinfo_serialized()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/pulp_2to3_models.py", line 745, in get_treeinfo_serialized
Feb  2 16:34:30 katellod02p pulpcore-worker-1: treeinfo.load(f=os.path.join(self.pulp2content.pulp2_storage_path, namespace))
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 55, in load
Feb  2 16:34:30 katellod02p pulpcore-worker-1: super().load(f)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/common.py", line 263, in load
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.deserialize(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 83, in deserialize
Feb  2 16:34:30 katellod02p pulpcore-worker-1: current.deserialize(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/treeinfo.py", line 319, in deserialize
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.deserialize_1_0(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/treeinfo.py", line 347, in deserialize_1_0
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.build_timestamp = parser.getint(self._section, "build_timestamp")
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 819, in getint
Feb  2 16:34:30 katellod02p pulpcore-worker-1: fallback=fallback, **kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 809, in _get_conv
Feb  2 16:34:30 katellod02p pulpcore-worker-1: **kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 803, in _get
Feb  2 16:34:30 katellod02p pulpcore-worker-1: return conv(self.get(section, option, **kwargs))
Feb  2 16:34:30 katellod02p pulpcore-worker-1: ValueError: invalid literal for int() with base 10: '1601956550.79'
Feb  2 16:34:30 katellod02p pulpcore-worker-1: Traceback (most recent call last):
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 65, in deserialize
Feb  2 16:34:30 katellod02p pulpcore-worker-1: super().deserialize(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/treeinfo.py", line 125, in deserialize
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.tree.deserialize(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/treeinfo.py", line 319, in deserialize
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.deserialize_1_0(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/treeinfo.py", line 347, in deserialize_1_0
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.build_timestamp = parser.getint(self._section, "build_timestamp")
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 819, in getint
Feb  2 16:34:30 katellod02p pulpcore-worker-1: fallback=fallback, **kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 809, in _get_conv
Feb  2 16:34:30 katellod02p pulpcore-worker-1: **kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 803, in _get
Feb  2 16:34:30 katellod02p pulpcore-worker-1: return conv(self.get(section, option, **kwargs))
Feb  2 16:34:30 katellod02p pulpcore-worker-1: ValueError: invalid literal for int() with base 10: '1601956550.79'
Feb  2 16:34:30 katellod02p pulpcore-worker-1: During handling of the above exception, another exception occurred:
Feb  2 16:34:30 katellod02p pulpcore-worker-1: Traceback (most recent call last):
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Feb  2 16:34:30 katellod02p pulpcore-worker-1: rv = job.perform()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self._result = self._execute()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Feb  2 16:34:30 katellod02p pulpcore-worker-1: return self.func(*self.args, **self.kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 141, in migrate_from_pulp2
Feb  2 16:34:30 katellod02p pulpcore-worker-1: migrate_content(plan, skip_corrupted=skip_corrupted)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 47, in migrate_content
Feb  2 16:34:30 katellod02p pulpcore-worker-1: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 145, in migrate_content_to_pulp3
Feb  2 16:34:30 katellod02p pulpcore-worker-1: loop.run_until_complete(dm.create())
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Feb  2 16:34:30 katellod02p pulpcore-worker-1: return future.result()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 90, in create
Feb  2 16:34:30 katellod02p pulpcore-worker-1: await pipeline
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Feb  2 16:34:30 katellod02p pulpcore-worker-1: await asyncio.gather(*futures)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Feb  2 16:34:30 katellod02p pulpcore-worker-1: await self.run()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 182, in run
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.migrate_to_pulp3(cmodel, ctype)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 277, in migrate_to_pulp3
Feb  2 16:34:30 katellod02p pulpcore-worker-1: pulp3content, extra_info = pulp_2to3_detail_content.create_pulp3_content()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/pulp_2to3_models.py", line 774, in create_pulp3_content
Feb  2 16:34:30 katellod02p pulpcore-worker-1: treeinfo_serialized = self.get_treeinfo_serialized()
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/pulp_2to3_models.py", line 745, in get_treeinfo_serialized
Feb  2 16:34:30 katellod02p pulpcore-worker-1: treeinfo.load(f=os.path.join(self.pulp2content.pulp2_storage_path, namespace))
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 55, in load
Feb  2 16:34:30 katellod02p pulpcore-worker-1: super().load(f)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/common.py", line 263, in load
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.deserialize(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulp_rpm/app/kickstart/treeinfo.py", line 83, in deserialize
Feb  2 16:34:30 katellod02p pulpcore-worker-1: current.deserialize(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/treeinfo.py", line 319, in deserialize
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.deserialize_1_0(parser)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/productmd/treeinfo.py", line 347, in deserialize_1_0
Feb  2 16:34:30 katellod02p pulpcore-worker-1: self.build_timestamp = parser.getint(self._section, "build_timestamp")
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 819, in getint
Feb  2 16:34:30 katellod02p pulpcore-worker-1: fallback=fallback, **kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 809, in _get_conv
Feb  2 16:34:30 katellod02p pulpcore-worker-1: **kwargs)
Feb  2 16:34:30 katellod02p pulpcore-worker-1: File "/usr/lib64/python3.6/configparser.py", line 803, in _get
Feb  2 16:34:30 katellod02p pulpcore-worker-1: return conv(self.get(section, option, **kwargs))
Feb  2 16:34:30 katellod02p pulpcore-worker-1: ValueError: invalid literal for int() with base 10: '1601956550.79'
Feb  2 16:34:30 katellod02p pulpcore-worker-1: pulp: rq.worker:INFO: Cleaning registries for queue: 32060@katellod02p.d.carfax.us
Feb  2 16:34:30 katellod02p pulpcore-worker-1: pulp: rq.worker:INFO: 32060@katellod02p.d.carfax.us: 811b317b-2bd0-4822-aa78-42e222428f76
Feb  2 16:34:30 katellod02p pulpcore-worker-1: pulp: rq.worker:INFO: 32060@katellod02p.d.carfax.us: Job OK (811b317b-2bd0-4822-aa78-42e222428f76)
Feb  2 16:34:30 katellod02p pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Feb  2 16:34:31 katellod02p pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[44bff93e-4d19-428f-a7ad-b9c763e316b4]
Feb  2 16:34:31 katellod02p pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[8760144d-c11c-4f41-805a-77d056052f46]
Feb  2 16:34:31 katellod02p pulp: celery.app.trace:INFO: [8760144d] Task pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[8760144d-c11c-4f41-805a-77d056052f46] succeeded in 0.0153680140065s: None
Feb  2 16:34:31 katellod02p pulp: celery.app.trace:INFO: [44bff93e] Task pulp.server.async.tasks._queue_reserved_task[44bff93e-4d19-428f-a7ad-b9c763e316b4] succeeded in 0.0431852839974s: None
Feb  2 16:34:31 katellod02p pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[cb5d183b-8894-4de2-a759-9cc609deb05a]
Feb  2 16:34:31 katellod02p pulp: celery.app.trace:INFO: [cb5d183b] Task pulp.server.async.tasks._release_resource[cb5d183b-8894-4de2-a759-9cc609deb05a] succeeded in 0.0032302979962s: None
Feb  2 16:34:33 katellod02p pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
Feb  2 16:34:33 katellod02p pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[35654f68-2a97-436b-801d-33138a6f6184]
Feb  2 16:34:33 katellod02p pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[18518e6a-05e9-44b1-849f-6e05b81d6058]
Feb  2 16:34:33 katellod02p pulp: celery.app.trace:INFO: [35654f68] Task pulp.server.async.tasks._queue_reserved_task[35654f68-2a97-436b-801d-33138a6f6184] succeeded in 0.0220853369974s: None
Feb  2 16:34:33 katellod02p pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[61daca52-ea63-4d78-9319-5fa4d79ccf20]
Feb  2 16:34:33 katellod02p pulp: celery.app.trace:INFO: [18518e6a] Task pulp.server.managers.consumer.applicability.regenerate_applicability_for_consumers[18518e6a-05e9-44b1-849f-6e05b81d6058] succeeded in 0.0138719550014s: None
Feb  2 16:34:33 katellod02p pulp: celery.app.trace:INFO: [61daca52] Task pulp.server.async.tasks._release_resource[61daca52-ea63-4d78-9319-5fa4d79ccf20] succeeded in 0.00245403499866s: None
8

Just for reference, here is the error on this run:

foreman-maintain content switchover
Running Switch support for certain content from Pulp 2 to Pulp 3
================================================================================
Switch support for certain content from Pulp 2 to Pulp 3:
Performing final content migration before switching content
           [FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 1:
 enabled
Starting task.
Migration failed, You will want to investigate: https://katellod02p.d.carfax.us/foreman_tasks/tasks/11178f76-a22c-4774-96c9-42eda5d09487
rake aborted!
ForemanTasks::TaskError: Task 11178f76-a22c-4774-96c9-42eda5d09487: Katello::Errors::Pulp3Error: invalid literal for int() with base 10: '1601956550.79'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1/lib/katello/tasks/pulp3_migration.rake:31:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:pulp3_migration
(See full trace by running task with --trace)
2021-02-02 16:34:34 -0600: Migrating rpm content to Pulp 3 srpm 4173/4187
--------------------------------------------------------------------------------
Scenario [Switch support for certain content from Pulp 2 to Pulp 3] failed.

The following steps ended up in failing state:

  [content-switchover]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="content-switchover"
9

Awesome! It looks like there may be a kickstart tree with some data that pulp3 isn’t able to migrate properly. I opened an issue here: Issue #8200: ValueError: invalid literal for int() with base 10 during migration (likely treeinfo related) - Migration Plugin - Pulp

We’ll work with the pulp team to get it resolved and backported to 3.18. It may help to know which kickstart tree is causing the problem. Would you be able to try something like:

grep 1601956550 /var/lib/pulp/published/yum/master/yum_distributor/*/.treeinfo  /var/lib/pulp/published/yum/master/yum_distributor/*/treeinfo

and then if that locates a file with that timestamp, provide the full treeinfo file here? that might give us a clue as to where it came from.

10

I don’t get anything back for that search:

grep: /var/lib/pulp/published/yum/master/yum_distributor/*/.treeinfo: No such file or directory
grep: /var/lib/pulp/published/yum/master/yum_distributor/*/treeinfo: No such file or directory

When I look under /var/lib/pulp/published/yum/master/yum_distributor/*, I do see numbers similar to
1601956550.79, but none that actually match.

 ls /var/lib/pulp/published/yum/master/yum_distributor/*/ | grep 160

1608308470.3
1601874613.47
1600980586.86
1609705083.26
1609729119.1
1601923272.73
1609728608.41
1609728611.05
1609728607.58
1606860001.58
1606860001.22
1609728607.17
1609728607.37
1609728607.43
1601923388.72
1607096086.01
1607096085.79
1601922684.1
1609728557.61
1609728552.11
1609728554.52
1606859972.17
1606859971.41
1609728561.56
1609728556.68
1609728555.51
1601923668.28
1607612543.58
1607612543.3
1609728542.82
1609728544.59
1609728542.9
1609728543.37
1609728543.72
1609728543.44
1609728544.19
1609728543.05
1609728542.96
1609728543.81
1601922663.73
1601922664.71
1601922663.12
1601922665.17
1601922667.85
1601922666.18
1601922664.32
1601922667.23
1601922664.82
1601922666.83
1601923180.41
1601923179.89
1601923178.71
1601923179.64
1601923180.24
1601923180.05
1601923178.9
1601923179.79
1601923179.33
1601923180.31
1604023136.62
1604023136.2
1604023135.55
1604023135.49
1604023136.31
1604023135.94
1604023136.02
1604023136.53
1604023136.34
1604023136.73
1606859955.75
1606859956.62
1606859955.79
1606859956.02
1606859956.3
1606859956.08
1606859956.51
1606859956.37
1606859955.94
1606859956.44
1609728021.62
1609728022.15
1609728022.45
1609728021.68
1609728021.76
1609728022.49
1609727954.82
1609727956.78
1606858496.79
1609727957.54
1609727943.9
1609727949.3
1606858487.82
1606858496.17
1606858487.85
1606858545.11
1606858506.52
1606858513.23
1609727928.5
1609727929.61
1609727884.65
1609727937.06
1609727889.17
1609727885.2
1601917252.51
1601917254.39
1601917252.44
1601917320.25
1601917269.27
1601917275.12
1601918269.59
1601918271.28
1601918269.73
1601918322.55
1601918275.95
1601918276.5
1604022864.61
1604022866.39
1604022865.92
1604022912.76
1604022871.75
1604022877.23
1600981247.87
1606859704.45
1609728844.46
1606859704.5
1609728846.74
1600981292.16
1607096152.99
1607096153.34
1600981186.55
1606859665.67
1609728765.9
1606859672.29
1609728765.7
1600981340.6
1600439868.15
1607612612.03
1607612612.12
1600981182.51
1600981181.64
1600981181.71
1600981181.79
1600981182.65
1600981181.59
1600981184.54
1600981184.39
1601587858.57
1601587858.68
1601587858.48
1601587858.63
1601587859.15
1601587858.87
1601587859.64
1601587859.89
1603133612.25
1603133611.99
1603133611.67
1603133611.95
1603133613.0
1603133612.16
1603133612.4
1603133613.07
1604023150.6
1604023150.36
1604023150.67
1604023150.5
1604023150.85
1604023150.92
1604023152.17
1604023152.48
1606859662.63
1606859663.02
1606859662.81
1606859662.86
1606859663.09
1606859663.14
1606859663.26
1606859663.69
1609728735.05
1609728734.97
1609728735.16
1609728735.46
1609728735.74
1609728735.25
1609728736.03
1609728736.32
1601405056.69
1608306076.73
1609728044.25
1609728044.52
1609728044.71
1606858625.78
1609727966.28
1606858689.66
1601584848.1
1601584846.95
1601584852.24
1601584846.72
1601584905.74
1603127299.82
1603127300.92
1603127307.46
1603127299.73
1603127363.74
1604022685.58
1604022688.38
1604022689.79
1604022685.38
1604022740.26
1606858622.35
1606858622.7
1606858626.45
1606858622.45
1606858678.54
1609727934.18
1609727935.63
1609727942.57
1609727934.75
1609727976.14
1609728268.34
1609728235.26
1601585129.81
1603133491.51
1604022809.88
1606858846.23
1609728218.02
1609728255.54
1609728230.4
1606859030.34
1609728216.95
1601917520.45
1604022911.22
1607374103.0
1607374125.07
1607373725.81
1607374141.05
1607373723.87
1607374152.61
1607374167.17
1607373688.75
1607374181.43
1607373686.62
1606859091.87
1604022885.05
1606859090.19
1609728319.89
1609728374.56
1601585230.18
1601585227.96
1601585228.93
1601585227.81
1603127346.44
1603127340.98
1603127342.85
1603127341.63
1604022807.7
1604022807.74
1604022803.17
1604022803.44
1606859131.42
1606859129.95
1606859131.6
1606859130.29
1609728367.5
1609728365.29
1609728365.91
1609728367.73
1603739194.72
1609794913.36
1603739177.96
1609794932.54
1600896063.85
1601324623.27
1603739174.01
1608217107.46
1609779531.38
1609728996.73
1609728997.01
1600433560.17
1609728997.34
1601924133.44
1609728997.08
1606859904.89
1606859904.4
1609728996.81
1600433561.41
1600433560.42
1600433598.4
1601924185.83
1607096252.47
1607096248.96
1600433599.4
1600433598.5
1609728854.43
1609728854.53
1600433355.65
1609728854.82
1601924048.06
1609728857.52
1606859841.07
1606859840.56
1609728854.7
1600433358.39
1600433356.98
1600434093.92
1601924333.7
1607613171.98
1607613171.2
1600434099.29
1600434094.1
1600433346.62
1600433346.68
1600433345.98
1600433347.65
1600433347.08
1600433346.82
1600433346.89
1600433346.06
1600433346.55
1600433347.19
1600433346.11
1600703506.2
1600703510.21
1600703506.21
1600703511.2
1600703511.08
1600703506.57
1600703510.26
1600703505.83
1600703506.32
1600703511.14
1600703506.07
1601924028.24
1601924027.65
1601924026.72
1601924028.99
1601924028.66
1601924027.97
1601924028.34
1601924026.79
1601924028.45
1601924028.86
1601924027.29
1604586446.73
1604586446.84
1604586445.33
1604586447.8
1604586447.56
1604586447.28
1604586447.35
1604586446.58
1604586446.95
1604586446.44
1604586445.62
1606859830.6
1606859830.96
1606859830.38
1606859831.74
1606859831.29
1606859831.08
1606859831.35
1606859830.53
1606859831.16
1606859830.65
1606859830.72
1609728795.82
1609728796.8
1609728795.28
1609728797.18
1609728797.14
1609728796.23
1609728796.83
1609728795.56
1609728796.14
1609728795.61
1609728795.89
1608571697.91
1608571700.79
1608571699.25
1608571698.05
1608571699.26
1608571698.56
1608571697.97
1608571978.77
1608571978.93
1608571979.04
1608571978.55
1608571978.98
1608571978.61
1608571977.6
1609728432.68
1609728689.13
1601918380.23
1609728434.3
1606859570.19
1609728498.34
1600432875.12
1608572008.23
1608572009.25
1608572008.93
1608572008.83
1608572009.14
1608572008.56
1608572008.49
1600432853.84
1600433025.8
1600432855.88
1600432856.69
1600432855.65
1600432900.95
1600432870.19
1600693049.83
1600693218.56
1600693051.17
1600693050.54
1600693050.31
1600693083.58
1600693060.75
1601918315.37
1601918496.46
1601918316.17
1601918315.76
1601918315.97
1601918339.81
1601918322.93
1604586015.33
1604586205.4
1604586016.39
1604586014.81
1604586015.98
1604586059.98
1604586028.72
1606859188.99
1606859432.83
1606859190.29
1606859190.18
1606859569.13
1606859249.89
1606859206.01
1609728426.08
1609728664.72
1609728428.41
1609728429.32
1609728428.11
1609728473.99
1609728430.71
1608564646.93
1608564668.08
1600433118.21
1608564697.13
1600433112.21
1600703141.07
1601923703.82
1601931348.92
1601931383.97
1600980707.76
1601931549.78
1600980705.25
1601923117.67
1609720305.28
1609692059.64
1609710781.71
1609736436.69
1609780280.98
1609736436.61
1609736437.85
1604584808.63
1601874412.0
1609736435.64
1609727877.9
1609736436.04
1609728199.38
1601875155.37
1609691643.89
1609704575.66
1609736436.09
1609720371.05
1609736434.74
1609727880.04
1601874072.1
1601873908.93
1609720305.43
1609691919.27
1601877104.26
1609720371.06
1609970563.83
1609727957.14
1601874638.31
1609695979.74
1609720767.59
1607372532.72
1609704609.48
1609728364.07
1609705311.05
1609704608.06
1607372643.76
1601873020.33
1609720348.9

11

Those are directories that pulp creates, and are okay. try grepping those directories for ’ build_timestamp’

grep -R build_timestamp /var/lib/pulp/published/yum/master/yum_distributor/

see if that returns ‘1601956550.79’ in any files

12

Just wanted to update you that the grep is still running. No timestamp is coming back with a decimal point. Everything returns like this:

1606858545.11/.treeinfo:build_timestamp = 1605735501

Looks like it won’t finish running until tomorrow.

13

It finally finished. Here are the results:

/var/lib/pulp/published/yum/master/yum_distributor/c269d5ae-3538-4092-9f0a-6c3c9d966f48/1609970563.83/.treeinfo:build_timestamp = 1601956550.79
/var/lib/pulp/published/yum/master/yum_distributor/1-carfax_ol_7_kickstart_cv-v1_0-c269d5ae-3538-4092-9f0a-6c3c9d966f48/1610121521.91/.treeinfo:build_timestamp = 1601956550.79
/var/lib/pulp/published/yum/master/yum_distributor/1-carfax_ol_7_kickstart_cv-Library-c269d5ae-3538-4092-9f0a-6c3c9d966f48/1610121531.67/.treeinfo:build_timestamp = 1601956550.79
/var/lib/pulp/published/yum/master/yum_distributor/1-carfax-ol-7-composite-view-v1_0-c269d5ae-3538-4092-9f0a-6c3c9d966f48/1610122061.16/.treeinfo:build_timestamp = 1601956550.79
/var/lib/pulp/published/yum/master/yum_distributor/1-carfax-ol-7-composite-view-Library-c269d5ae-3538-4092-9f0a-6c3c9d966f48/1610122098.99/.treeinfo:build_timestamp = 1601956550.79
/var/lib/pulp/published/yum/master/yum_distributor/1-carfax-ol-7-composite-view-alpha-c269d5ae-3538-4092-9f0a-6c3c9d966f48/1610124301.2/.treeinfo:build_timestamp = 1601956550.79
/var/lib/pulp/published/yum/master/yum_distributor/1-carfax-ol-7-composite-view-beta-c269d5ae-3538-4092-9f0a-6c3c9d966f48/1610124356.88/.treeinfo:build_timestamp = 1601956550.79
/var/lib/pulp/published/yum/master/yum_distributor/1-carfax-ol-7-composite-view-prod-c269d5ae-3538-4092-9f0a-6c3c9d966f48/1610125167.15/.treeinfo:build_timestamp = 1601956550.79
14

Awesome! Can you pick one of those files and paste the full content of the file here? That should give us a clue about where this came from.

15

Here are a couple to look at:

cat /var/lib/pulp/published/yum/master/yum_distributor/c269d5ae-3538-4092-9f0a-6c3c9d966f48/1609970563.83/.treeinfo

[general]
; WARNING.0 = THIS SECTION IS KEPT ONLY FOR COMPATIBILITY REASONS
; WARNING.1 = Consider using other sections (product, base_product, tree, variant-*) instead
arch = x86_64
family = Oracle Linux
name = Oracle Linux 7.9
packagedir = Packages
platforms = x86_64,xen
repository = .
timestamp = 1601956550.79
variant = Server
version = 7.9

[header]
version = 0.3

[images-x86_64]
initrd = images/pxeboot/initrd.img
kernel = images/pxeboot/vmlinuz
upgrade = images/pxeboot/upgrade.img

[images-xen]
initrd = images/pxeboot/initrd.img
kernel = images/pxeboot/vmlinuz
upgrade = images/pxeboot/upgrade.img

[media]
discnum = 1
totaldiscs = 1

[product]
name = Oracle Linux
short = OL
version = 7.9

[stage2]
mainimage = LiveOS/squashfs.img

[tree]
arch = x86_64
build_timestamp = 1601956550.79
platforms = x86_64,xen
variants = Server

[variant-Server]
id = Server
name = Server
packages = Packages
repository = .
type = variant
uid = Server
addons = Server-HighAvailability,Server-ResilientStorage

[addon-Server-HighAvailability]
id = HighAvailability
name = High Availability
packages = addons/HighAvailability
repository = addons/HighAvailability
type = addon
uid = Server-HighAvailability

[addon-Server-ResilientStorage]
id = ResilientStorage
name = Resilient Storage
packages = addons/ResilientStorage
repository = addons/ResilientStorage
type = addon
uid = Server-ResilientStorage

[checksums]
LiveOS/squashfs.img = sha256:293b4768ec103b86e3c8ce97709d03781e96b9588a3e2d5b576a4aea5db50e8c
images/pxeboot/initrd.img = sha256:b616a7c583daa327ee26ba05ab4a37c15e3da70f39a7715b3bbadf5274c3f09f
images/pxeboot/upgrade.img = sha256:a6bd93d26cb541ed46f4deac17b91b93e0ee8edbd6a568a8908fb71284e0779b
images/pxeboot/vmlinuz = sha256:0bc3140324e22ed3355902e12093777f275f3a649a45615665c3cf31b17d90e5
addons/HighAvailability/repodata/repomd.xml = sha256:bb36fe16ddf54ee072c1e72f5f19a973c4585935644d2dec2d2dd75a79997417
addons/ResilientStorage/repodata/repomd.xml = sha256:357669f7bfd91be75767d2132d1e07b14a34f220f1780e7d73535b201b723397

cat /var/lib/pulp/published/yum/master/yum_distributor/1-carfax_ol_7_kickstart_cv-v1_0-c269d5ae-3538-4092-9f0a-6c3c9d966f48/1610121521.91/.treeinfo

[general]
; WARNING.0 = THIS SECTION IS KEPT ONLY FOR COMPATIBILITY REASONS
; WARNING.1 = Consider using other sections (product, base_product, tree, variant-*) instead
arch = x86_64
family = Oracle Linux
name = Oracle Linux 7.9
packagedir = Packages
platforms = x86_64,xen
repository = .
timestamp = 1601956550.79
variant = Server
version = 7.9

[header]
version = 0.3

[images-x86_64]
initrd = images/pxeboot/initrd.img
kernel = images/pxeboot/vmlinuz
upgrade = images/pxeboot/upgrade.img

[images-xen]
initrd = images/pxeboot/initrd.img
kernel = images/pxeboot/vmlinuz
upgrade = images/pxeboot/upgrade.img

[media]
discnum = 1
totaldiscs = 1

[product]
name = Oracle Linux
short = OL
version = 7.9

[stage2]
mainimage = LiveOS/squashfs.img

[tree]
arch = x86_64
build_timestamp = 1601956550.79
platforms = x86_64,xen
variants = Server

[variant-Server]
id = Server
name = Server
packages = Packages
repository = .
type = variant
uid = Server
addons = Server-HighAvailability,Server-ResilientStorage

[addon-Server-HighAvailability]
id = HighAvailability
name = High Availability
packages = addons/HighAvailability
repository = addons/HighAvailability
type = addon
uid = Server-HighAvailability

[addon-Server-ResilientStorage]
id = ResilientStorage
name = Resilient Storage
packages = addons/ResilientStorage
repository = addons/ResilientStorage
type = addon
uid = Server-ResilientStorage

[checksums]
LiveOS/squashfs.img = sha256:293b4768ec103b86e3c8ce97709d03781e96b9588a3e2d5b576a4aea5db50e8c
images/pxeboot/initrd.img = sha256:b616a7c583daa327ee26ba05ab4a37c15e3da70f39a7715b3bbadf5274c3f09f
images/pxeboot/upgrade.img = sha256:a6bd93d26cb541ed46f4deac17b91b93e0ee8edbd6a568a8908fb71284e0779b
images/pxeboot/vmlinuz = sha256:0bc3140324e22ed3355902e12093777f275f3a649a45615665c3cf31b17d90e5
addons/HighAvailability/repodata/repomd.xml = sha256:bb36fe16ddf54ee072c1e72f5f19a973c4585935644d2dec2d2dd75a79997417
addons/ResilientStorage/repodata/repomd.xml = sha256:357669f7bfd91be75767d2132d1e07b14a34f220f1780e7d73535b201b723397
16

@tbrooks, @Justin_Sherrill , thanks a lot it really helped.
There is a bug in one of the dependencies called productmd.
You can try to patch it and see if it works.

1 Like
17

By “see if it works”, I assumed that you meant rerun the migration which I tried. However, I’m still getting the error:
Katello::Errors::Pulp3Error: invalid literal for int() with base 10: ‘1601956550.79’

Is there something else I should do?

18

I’m having to roll back to a snapshot due to this bug.

https://community.theforeman.org/t/bug-host-group-settings-not-saved-foreman-2-2-katello-3-17/21353/42

19

@Justin_Sherrill, @goosemania I plan on taking a stab at upgrading again. So, I will be trying the pulp migration once again. Last time around, I wasn’t clear on what to do after I have applied the patch. I have edited /usr/share/foreman/app/models/host/managed.rb and applied the patch. How should I proceed when the file /var/lib/pulp/published/yum/master/yum_distributor/c269d5ae-3538-4092-9f0a-6c3c9d966f48/1609970563.83/.treeinfo still has the “timestamp = 1601956550.79” in the file. Can the .79 be simply removed?

Thanks,

Tim

20

With the patch to productmd, I’d expect it to just work.
If it fails the same or any other way, please provide traceback from pulp logs (use journalctl to look for them).
Thanks!

1 Like