1.19 > 1.20 upgrade issues

Tomasz_Zajaczkowski · November 22, 2018, 11:34am

Hi All,

I am having some issues upgrading 1.19 to 1.20… (with katello 3.8 > 3.9) running on CentOS Linux release 7.5.1804 (Core) and I need some assistance as I am not sure where the problem is…

‘foreman-installer (–scenario katello) --upgrade’ throws the error below:

Upgrade Step: migrate_foreman...
...
StandardError: An error has occurred, this and all later migrations canceled:

PG::UndefinedTable: ERROR:  relation "katello_root_repositories" does not exist
LINE 1: ...itories".* FROM "katello_repositories" INNER JOIN "katello_r...
                                                             ^
: SELECT "katello_repositories".* FROM "katello_repositories" INNER JOIN "katello_root_repositories" ON "katello_repositories"."root_id" = "katello_root_repositories"."id" WHERE "katello_root_repositories"."product_id" = $1

Before upgrade system was working fine but web gui now says: ‘ERF64-6496 [Foreman::MaintenanceException]: There are migrations pending in the system.’

Any help would be much appreciated.
Tomasz

tbrisker · November 22, 2018, 11:49am

can you try to run foreman-rake db:migrate? if it fails again, please paste the full error and stack trace that follows it

Tomasz_Zajaczkowski · November 22, 2018, 11:59am

please see below:

 foreman-rake db:migrate  --trace --verbose
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
** Invoke db:migrate (first_time)
** Invoke db:load_config (first_time)
** Invoke environment (first_time)
** Execute environment
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/content_facet_host_extensions.rb:7: warning: already initialized constant Katello::Concerns::ContentFacetHostExtensions::ERRATA_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/content_facet_host_extensions.rb:7: warning: previous definition of ERRATA_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/content_facet_host_extensions.rb:14: warning: already initialized constant Katello::Concerns::ContentFacetHostExtensions::TRACE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/content_facet_host_extensions.rb:14: warning: previous definition of TRACE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:13: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::SUBSCRIPTION_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:13: warning: previous definition of SUBSCRIPTION_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:21: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::SLA_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:21: warning: previous definition of SLA_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:26: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::USAGE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:26: warning: previous definition of USAGE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:31: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::ROLE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:31: warning: previous definition of ROLE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:36: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::ADDONS_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:36: warning: previous definition of ADDONS_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:41: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::PURPOSE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/concerns/subscription_facet_host_extensions.rb:41: warning: previous definition of PURPOSE_STATUS_MAP was here
** Execute db:load_config
** Invoke plugin:refresh_migrations (first_time)
** Invoke environment
** Execute plugin:refresh_migrations
** Execute db:migrate
== 20180814202747 AddRecurringLogicToSyncPlan: migrating ======================
-- add_column(:katello_sync_plans, :foreman_tasks_recurring_logic_id, :integer)
   -> 0.0008s
-- add_column(:katello_sync_plans, :cron_expression, :string)
   -> 0.0003s
-- add_foreign_key(:katello_sync_plans, :foreman_tasks_recurring_logics, {:name=>"katello_sync_plan_foreman_tasks_recurring_logic_fk", :column=>"foreman_tasks_recurring_logic_id"})
   -> 0.0017s
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

PG::UndefinedTable: ERROR:  relation "katello_root_repositories" does not exist
LINE 1: ...itories".* FROM "katello_repositories" INNER JOIN "katello_r...
                                                             ^
: SELECT "katello_repositories".* FROM "katello_repositories" INNER JOIN "katello_root_repositories" ON "katello_repositories"."root_id" = "katello_root_repositories"."id" WHERE "katello_root_repositories"."product_id" = $1
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:669:in `prepare'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:669:in `block in prepare_statement'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:664:in `prepare_statement'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_cache'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `execute_and_clear'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:81:in `exec_query'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:469:in `select_prepared'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:55:in `select_all'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:101:in `select_all'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:41:in `find_by_sql'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/statement_cache.rb:108:in `execute'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_association.rb:314:in `find_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/has_many_through_association.rb:194:in `find_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_association.rb:271:in `load_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_proxy.rb:46:in `load_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_proxy.rb:1032:in `records'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:498:in `blank?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/core_ext/object/blank.rb:27:in `present?'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/product.rb:127:in `enabled?'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/sync_plan.rb:40:in `block in product_enabled'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/sync_plan.rb:39:in `product_enabled'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:426:in `block in make_lambda'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:198:in `block (2 levels) in halting'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:606:in `block (2 levels) in default_terminator'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:605:in `catch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:605:in `block in default_terminator'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:199:in `block in halting'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `block in invoke_before'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `invoke_before'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:131:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:816:in `_run_validate_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations.rb:409:in `run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations/callbacks.rb:118:in `block in run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:816:in `_run_validation_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations/callbacks.rb:118:in `run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations.rb:339:in `valid?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:67:in `valid?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:84:in `perform_validations'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:52:in `save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:315:in `block in save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:252:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:315:in `save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/suppressor.rb:48:in `save!'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:9:in `block (2 levels) in change'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:100:in `as'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:106:in `as_anonymous_admin'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:7:in `block in change'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `block in find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:222:in `loop'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:222:in `in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:69:in `find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:11:in `find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:6:in `change'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:814:in `exec_migration'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:798:in `block (2 levels) in migrate'
/opt/rh/rh-ruby25/root/usr/share/ruby/benchmark.rb:293:in `measure'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:797:in `block in migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:414:in `with_connection'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:796:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:977:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1292:in `block in execute_migration_in_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `block in ddl_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `block in transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `ddl_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1291:in `execute_migration_in_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1263:in `block in migrate_without_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `migrate_without_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `block in migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1363:in `with_advisory_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1036:in `up'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1011:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:172:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/railties/databases.rake:60:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `block in execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:195:in `block in invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:188:in `invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:181:in `invoke'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:160:in `invoke_task'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `block in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:125:in `run_with_threads'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:110:in `top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:83:in `block in run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:80:in `run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `load'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `<main>'

Caused by:
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "katello_root_repositories" does not exist
LINE 1: ...itories".* FROM "katello_repositories" INNER JOIN "katello_r...
                                                             ^
: SELECT "katello_repositories".* FROM "katello_repositories" INNER JOIN "katello_root_repositories" ON "katello_repositories"."root_id" = "katello_root_repositories"."id" WHERE "katello_root_repositories"."product_id" = $1
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:669:in `prepare'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:669:in `block in prepare_statement'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:664:in `prepare_statement'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_cache'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `execute_and_clear'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:81:in `exec_query'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:469:in `select_prepared'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:55:in `select_all'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:101:in `select_all'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:41:in `find_by_sql'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/statement_cache.rb:108:in `execute'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_association.rb:314:in `find_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/has_many_through_association.rb:194:in `find_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_association.rb:271:in `load_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_proxy.rb:46:in `load_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_proxy.rb:1032:in `records'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:498:in `blank?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/core_ext/object/blank.rb:27:in `present?'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/product.rb:127:in `enabled?'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/sync_plan.rb:40:in `block in product_enabled'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/sync_plan.rb:39:in `product_enabled'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:426:in `block in make_lambda'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:198:in `block (2 levels) in halting'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:606:in `block (2 levels) in default_terminator'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:605:in `catch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:605:in `block in default_terminator'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:199:in `block in halting'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `block in invoke_before'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `invoke_before'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:131:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:816:in `_run_validate_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations.rb:409:in `run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations/callbacks.rb:118:in `block in run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:816:in `_run_validation_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations/callbacks.rb:118:in `run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations.rb:339:in `valid?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:67:in `valid?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:84:in `perform_validations'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:52:in `save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:315:in `block in save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:252:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:315:in `save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/suppressor.rb:48:in `save!'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:9:in `block (2 levels) in change'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:100:in `as'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:106:in `as_anonymous_admin'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:7:in `block in change'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `block in find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:222:in `loop'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:222:in `in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:69:in `find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:11:in `find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:6:in `change'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:814:in `exec_migration'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:798:in `block (2 levels) in migrate'
/opt/rh/rh-ruby25/root/usr/share/ruby/benchmark.rb:293:in `measure'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:797:in `block in migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:414:in `with_connection'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:796:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:977:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1292:in `block in execute_migration_in_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `block in ddl_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `block in transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `ddl_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1291:in `execute_migration_in_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1263:in `block in migrate_without_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `migrate_without_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `block in migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1363:in `with_advisory_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1036:in `up'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1011:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:172:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/railties/databases.rake:60:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `block in execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:195:in `block in invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:188:in `invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:181:in `invoke'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:160:in `invoke_task'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `block in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:125:in `run_with_threads'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:110:in `top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:83:in `block in run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:80:in `run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `load'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `<main>'

Caused by:
PG::UndefinedTable: ERROR:  relation "katello_root_repositories" does not exist
LINE 1: ...itories".* FROM "katello_repositories" INNER JOIN "katello_r...
                                                             ^
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:669:in `prepare'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:669:in `block in prepare_statement'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:664:in `prepare_statement'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_cache'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `execute_and_clear'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:81:in `exec_query'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:469:in `select_prepared'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:55:in `select_all'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/query_cache.rb:101:in `select_all'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:41:in `find_by_sql'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/statement_cache.rb:108:in `execute'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_association.rb:314:in `find_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/has_many_through_association.rb:194:in `find_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_association.rb:271:in `load_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_proxy.rb:46:in `load_target'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/associations/collection_proxy.rb:1032:in `records'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:498:in `blank?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/core_ext/object/blank.rb:27:in `present?'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/product.rb:127:in `enabled?'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/sync_plan.rb:40:in `block in product_enabled'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/app/models/katello/sync_plan.rb:39:in `product_enabled'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:426:in `block in make_lambda'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:198:in `block (2 levels) in halting'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:606:in `block (2 levels) in default_terminator'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:605:in `catch'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:605:in `block in default_terminator'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:199:in `block in halting'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `block in invoke_before'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:513:in `invoke_before'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:131:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:816:in `_run_validate_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations.rb:409:in `run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations/callbacks.rb:118:in `block in run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/callbacks.rb:816:in `_run_validation_callbacks'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations/callbacks.rb:118:in `run_validations!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activemodel-5.2.1/lib/active_model/validations.rb:339:in `valid?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:67:in `valid?'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:84:in `perform_validations'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/validations.rb:52:in `save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:315:in `block in save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:252:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:315:in `save!'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/suppressor.rb:48:in `save!'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:9:in `block (2 levels) in change'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:100:in `as'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:106:in `as_anonymous_admin'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:7:in `block in change'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:70:in `block in find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:222:in `loop'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:222:in `in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/batches.rb:69:in `find_each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:11:in `find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.9.0/db/migrate/20180814202747_add_recurring_logic_to_sync_plan.rb:6:in `change'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:814:in `exec_migration'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:798:in `block (2 levels) in migrate'
/opt/rh/rh-ruby25/root/usr/share/ruby/benchmark.rb:293:in `measure'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:797:in `block in migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:414:in `with_connection'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:796:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:977:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1292:in `block in execute_migration_in_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `block in ddl_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `block in transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `ddl_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1291:in `execute_migration_in_transaction'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1263:in `block in migrate_without_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `migrate_without_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `block in migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1363:in `with_advisory_lock'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1036:in `up'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1011:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:172:in `migrate'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/railties/databases.rake:60:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `block in execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in `execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:195:in `block in invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in `mon_synchronize'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:188:in `invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:181:in `invoke'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:160:in `invoke_task'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in `block in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:125:in `run_with_threads'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:110:in `top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:83:in `block in run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:80:in `run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `load'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `<main>'
Tasks: TOP => db:migrate

mrabrde · November 23, 2018, 2:57pm

Hello,

i also get the same error while upgrading from foreman 1.19 -> 1.20 with katello 3.8 -> 3.9.
My server is also running on CentOS 7.5.1804.

Regards

Michael

sajha · November 26, 2018, 12:16am

The issue is reproducible for sync plans with redhat products. We are tracking the fix for this with Bug #25544: Upgrade issue 3.8 -> 3.9 - Katello - Foreman

Zhunting · November 26, 2018, 2:25pm

Thanks for finding this and thanks @sajha for a quick fix. When the PR is merged a 3.9.1 release will follow soon.

kevev · December 3, 2018, 2:50pm

Is there a workaround? I am stuck with an partially upgraded server.

mrabrde · December 3, 2018, 3:32pm

Hello,

download the files

https://projects.theforeman.org/projects/katello/repository/revisions/dd6df32019fdcd30da9fa7a5e57e3d6bbe372bd9

and maintain it on your system.

Regards

Michael

kevev · December 4, 2018, 9:10pm

Thank You, as that resolved my upgrade issue.

I ran into one more issue, but was able to fix it myself:

Upgrade Step: Running installer…
‘qpid-config --ssl-certificate /etc/pki/katello/certs/foreman01.rbfcu.local-qpid-broker.crt --ssl-key /etc/pki/katello/private/foreman01.rbfcu.local-qpid-broker.key -b amqps://localhost:5671 add queue katello_event_queue --durable’ returned 1 instead of one of [0]

The fix was to do umask 0027 and then re-run the upgrade. I found the tip here:
https://bugzilla.redhat.com/show_bug.cgi?id=1119910