Update Server

Hi All

After running yum update on Foreman/Katello server I get this error on the WEB UI?

Service unavailabl ERF64-6496 [Foreman::MaintenanceException]: There are migrations pending in the system.

I have tried restarting the services foreman-maintain service restart, all services show as running with no errors, have also tried a server reboot, same error, have I missed something I need to do when updating the core server.

CentOS 7.8
Foreman 2.0
Katello 3.15

foreman-installer --scenario katello --upgrade
?

Thanks that fixed it, but now I have another problem, there is a paused task
called Content migration which is stuck at paused have tried to resume and to
cancel, but neither work, the error is:
Katello::Errors::Pulp3MigrationError: Pulp 3 migration cannot run. Types file, docker have already been migrated.

Hi Stephen,
When I’m not able to clean the task using the foreman web or the dynaflow console, I use the following options. Of course, use it on your own responsibility.

option 1: foreman-rake foreman_tasks:cleanup

foreman-rake foreman_tasks:cleanup TASK_SEARCH='label ~ *' STATES='running,pending,stopped' VERBOSE=true
foreman-rake katello:delete_orphaned_content --trace
foreman-rake katello:reimport --trace

option 2: remove the tasks from the database ( https://cstan.io/?p=8976&lang=en )

tasks=$( sudo su - postgres -c ‘psql -d foreman -c "select id from foreman_tasks_tasks where state != ‘’‘stopped’’’;"’ | grep ‘[^-].*-.*-.*-.*-.*’)
for task in $Tasks; do
echo “ForemanTasks::Task.find(”${task}").destroy"
done | foreman-rake console

While I’m not a member of any of the dev teams, I hope these options could be useful for you.

Hi

Many thanks, the remove from the database has sorted it.

I’m having the exact same problem after updating from 2.0.0 to 2.0.1.

System was working fine, but after a yum update (that upgraded to 2.0.1), and after a reboot, i’m seeing the same error message. I tried foreman-rake db:migrate only to fail with this trace:
20140522185700 ChangeTemplatekindToBootdisk: migrating ===================
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

Invalid single-table inheritance type: ProvisioningTemplate is not a subclass of ChangeTemplatekindToBootdisk::FakeConfigTemplate
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/inheritance.rb:233:in `find_sti_class'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/inheritance.rb:207:in `discriminate_class_for_record'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/persistence.rb:69:in `instantiate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:52:in `block (2 levels) in find_by_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/result.rb:57:in `block in each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/result.rb:57:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/result.rb:57:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:52:in `map'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:52:in `block in find_by_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:51:in `find_by_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:554:in `block in exec_queries'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:578:in `skip_query_cache_if_necessary'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:541:in `exec_queries'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:414:in `load'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:200:in `records'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `|'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-16.0.0/db/migrate/20140522185700_change_templatekind_to_bootdisk.rb:18:in `up'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:777:in `up'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:817:in `exec_migration'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:798:in `block (2 levels) in migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:797:in `block in migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:414:in `with_connection'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:796:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:977:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1292:in `block in execute_migration_in_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `block in ddl_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `block in transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `ddl_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1291:in `execute_migration_in_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1263:in `block in migrate_without_lock'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `migrate_without_lock'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `block in migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1363:in `with_advisory_lock'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1036:in `up'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1011:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:172:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/railties/databases.rake:60:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'

Caused by:
ActiveRecord::SubclassNotFound: Invalid single-table inheritance type: ProvisioningTemplate is not a subclass of ChangeTemplatekindToBootdisk::FakeConfigTemplate
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/inheritance.rb:233:in `find_sti_class'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/inheritance.rb:207:in `discriminate_class_for_record'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/persistence.rb:69:in `instantiate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:52:in `block (2 levels) in find_by_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/result.rb:57:in `block in each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/result.rb:57:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/result.rb:57:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:52:in `map'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:52:in `block in find_by_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/querying.rb:51:in `find_by_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:554:in `block in exec_queries'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:578:in `skip_query_cache_if_necessary'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:541:in `exec_queries'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:414:in `load'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation.rb:200:in `records'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `|'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-16.0.0/db/migrate/20140522185700_change_templatekind_to_bootdisk.rb:18:in `up'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:777:in `up'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:817:in `exec_migration'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:798:in `block (2 levels) in migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:797:in `block in migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:414:in `with_connection'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:796:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:977:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1292:in `block in execute_migration_in_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `block in ddl_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `block in transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/connection_adapters/abstract/database_statements.rb:254:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/transactions.rb:212:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1343:in `ddl_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1291:in `execute_migration_in_transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1263:in `block in migrate_without_lock'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1262:in `migrate_without_lock'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `block in migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1363:in `with_advisory_lock'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1210:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1036:in `up'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/migration.rb:1011:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/tasks/database_tasks.rb:172:in `migrate'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/railties/databases.rake:60:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => db:migrate

Not really sure how to proceed now or how to fix this.
I also tried cleaning up the tasks, but still didn’t work.
Just to point out that i’m not using katello.

OK, just for completeness, removing foreman_bootdisk plugin fixes the problem, so, this may be “borked” somehow?

Hello, this is something different. It’s just happens that the UI error is the same, “pending migration” is a common error.

Are you sure about this statement? Because what I see in logs is inability to perform migration in bootdisk plugin that is from 2014. In other words, this would never happen if this was 2.0.0 -> 2.0.1 upgrade.

To be on the safe side, I have reviewed core patches backported into 2.0.1 and I see no changes related to provisioning templates.

What I see here are symptoms of upgrading across multiple versions. We don’t either test or support these kind of upgrades.

1 Like

Except if you enabled bootdisk plugin before/during the upgrade.

@Marek_Hulan I remember you were seeing something similar?

No, this was not what has been done. I’m sure 2.0.0 was running fine before the 2.0.1 upgrade.
The plugin was also not enabled for during the 2.0.1 upgrade, it was already there.

It might have been some ruby upgrade the made the migration code “invalid” and not directly related to the plugin itself, and that only ended up manifesting after the 2.0.1 upgrade, but this instance was, for sure, running 2.0.0 just fine. (and is now runing 2.0.1 after removing the bootdisk plugin)