foreman-nightly-rpm-pipeline 1484 failed

[jenkins]

Foreman RPM nightly pipeline failed:

https://ci.theforeman.org/job/foreman-nightly-rpm-pipeline/1484/

foreman-pipeline-foreman-nightly-almalinux8-install (failed) (remote job)
foreman-pipeline-foreman-nightly-centos7-install (failed) (remote job)
foreman-pipeline-foreman-nightly-centos8-stream-install (failed) (remote job)
foreman-pipeline-foreman-nightly-centos7-upgrade (failed) (remote job)
foreman-pipeline-foreman-nightly-almalinux8-upgrade (failed) (remote job)
foreman-pipeline-foreman-nightly-centos8-stream-upgrade (failed) (remote job)

[ehelms]

The failure appears to be coming from foreman_puppet plugin:

@Ron_Lavi @nadjaheitmann - could y’all take a look?

2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: PG::UndefinedTable: ERROR:  relation "application_records" does not exist
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: LINE 1: DELETE FROM "application_records" WHERE "application_records...
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns:                     ^
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `async_exec_params'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:671:in `block in exec_no_cache'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activesupport-6.1.6/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:670:in `exec_no_cache'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/postgresql_adapter.rb:649:in `execute_and_clear'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/postgresql/database_statements.rb:72:in `exec_delete'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/abstract/database_statements.rb:185:in `delete'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/connection_adapters/abstract/query_cache.rb:22:in `delete'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/activerecord-6.1.6/lib/active_record/relation.rb:594:in `delete_all'
2022-05-27 14:09:05 [INFO  ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: /usr/share/gems/gems/foreman_puppet-4.0.0/db/migrate_foreman/20161205142618_delete_orphaned_smart_class_parameters.foreman_puppet.rb:4:in `up'

[Ron_Lavi]

Seem like it’s related to the rails 6.1 upgrade,
adding it to our backlog.

[evgeni]

it isn’t Rails related, see the discussion with Ondrej in foreman-nightly-rpm-pipeline 1485 failed