Fail to sync repo after upgrade to Katello 4.1 - undefined method `last_contents_changed'

Problem:

After upgrading to Katello 4.1 there are some issues with sync of repos.
I think that someone forgot to declare a method correctly

Expected outcome:

Correct sync of repo

Foreman and Proxy versions:

Foreman 2.5 / Katello 4.1

Foreman and Proxy plugin versions:

  • candlepin-4.0.4-1.el8.noarch
  • candlepin-selinux-4.0.4-1.el8.noarch
  • foreman-2.5.0-1.el8.noarch
  • foreman-cli-2.5.0-1.el8.noarch
  • foreman-debug-2.5.0-1.el8.noarch
  • foreman-dynflow-sidekiq-2.5.0-1.el8.noarch
  • foreman-installer-2.5.0-1.el8.noarch
  • foreman-installer-katello-2.5.0-1.el8.noarch
  • foreman-postgresql-2.5.0-1.el8.noarch
  • foreman-proxy-2.5.0-1.el8.noarch
  • foreman-release-2.5.0-1.el8.noarch
  • foreman-service-2.5.0-1.el8.noarch
  • katello-4.1.0-1.el8.noarch
  • katello-certs-tools-2.7.3-1.el8.noarch
  • katello-client-bootstrap-1.7.6-1.el8.noarch
  • katello-common-4.1.0-1.el8.noarch
  • katello-debug-4.1.0-1.el8.noarch
  • katello-default-ca-1.0-1.noarch
  • katello-repos-4.1.0-1.el8.noarch
  • katello-server-ca-1.0-1.noarch
  • katello.comtec.lan-apache-1.0-1.noarch
  • katello.comtec.lan-foreman-client-1.0-1.noarch
  • katello.comtec.lan-foreman-proxy-1.0-1.noarch
  • katello.comtec.lan-foreman-proxy-client-1.0-1.noarch
  • katello.comtec.lan-puppet-client-1.0-1.noarch
  • pulp-client-1.0-1.noarch
  • python3-pulp-ansible-0.7.3-1.el8.noarch
  • python3-pulp-certguard-1.2.0-1.el8.noarch
  • python3-pulp-container-2.5.3-1.el8.noarch
  • python3-pulp-deb-2.11.2-1.el8.noarch
  • python3-pulp-file-1.6.0-1.el8.noarch
  • python3-pulp-rpm-3.11.0-1.el8.noarch
  • python3-pulpcore-3.11.2-1.el8.noarch
  • qpid-proton-c-0.32.0-3.el8.x86_64
  • rubygem-foreman-tasks-4.1.2-1.fm2_5.el8.noarch
  • rubygem-foreman-tasks-core-0.3.5-1.fm2_5.el8.noarch
  • rubygem-foreman_maintain-0.8.1-1.el8.noarch
  • rubygem-foreman_remote_execution-4.5.0-1.fm2_5.el8.noarch
  • rubygem-foreman_remote_execution_core-1.4.3-1.el8.noarch
  • rubygem-hammer_cli-2.5.0-1.el8.noarch
  • rubygem-hammer_cli_foreman-2.5.0-1.el8.noarch
  • rubygem-hammer_cli_foreman_remote_execution-0.2.1-2.fm2_5.el8.noarch
  • rubygem-hammer_cli_foreman_tasks-0.0.15-2.fm2_5.el8.noarch
  • rubygem-hammer_cli_katello-1.1.2-1.el8.noarch
  • rubygem-katello-4.1.0-1.el8.noarch
  • rubygem-pulp_ansible_client-0.7.3-1.el8.noarch
  • rubygem-pulp_certguard_client-1.0.3-2.el8.noarch
  • rubygem-pulp_container_client-2.4.0-1.el8.noarch
  • rubygem-pulp_deb_client-2.11.1-1.el8.noarch
  • rubygem-pulp_file_client-1.6.1-1.el8.noarch
  • rubygem-pulp_rpm_client-3.10.0-1.el8.noarch
  • rubygem-pulpcore_client-3.11.0-1.el8.noarch
  • rubygem-qpid_proton-0.32.0-3.el8.x86_64
  • rubygem-smart_proxy_pulp-3.0.0-1.fm2_5.el8.noarch

Distribution and version:

CentOS 8.4 x64

Other relevant data:

Actions::Katello::Applicability::Repository::Regenerate
Input:
{"repo_ids"=>[12],
 "current_request_id"=>"1ab32666-9671-469d-b318-12ad83d1dc43",
 "current_timezone"=>"Europe/Bucharest",
 "current_organization_id"=>1,
 "current_location_id"=>nil,
 "current_user_id"=>4}
Output:
{}
Exception:
NoMethodError: undefined method `last_contents_changed' for #<Katello::Repository:0x000055ff523caad0>
Backtrace:
/usr/share/gems/gems/activemodel-6.0.3.7/lib/active_model/attribute_methods.rb:432:in `method_missing'
/usr/share/gems/gems/audited-4.9.0/lib/audited/auditor.rb:99:in `method_missing'
/usr/share/gems/gems/katello-4.1.0/app/lib/actions/katello/applicability/repository/regenerate.rb:12:in `block in run'
/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in `each'
/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in `each'
/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/query_methods.rb:271:in `select'
/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/query_methods.rb:271:in `select'
/usr/share/gems/gems/katello-4.1.0/app/lib/actions/katello/applicability/repository/regenerate.rb:11:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:571:in `block (3 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/execution_wrapper.rb:88:in `wrap'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action/progress.rb:17:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/load_setting_values.rb:20:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
/usr/share/gems/gems/foreman-tasks-4.1.2/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/middleware/world.rb:31:in `execute'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:570:in `block (2 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:569:in `catch'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:569:in `block in execute_run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `block in with_error_handling'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `catch'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:472:in `with_error_handling'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:564:in `execute_run'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/action.rb:285:in `execute'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/director.rb:68:in `execute'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/executors.rb:18:in `run_user_code'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
/usr/share/gems/gems/dynflow-1.4.8/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq.rb:37:in `block in <module:Sidekiq>'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in `call'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

This method is defined dynamically based on the DB schema. It seems you database hasn’t been migrate and the column was not added, therefore the method does not exist. Is there a chance you only upgraded pacakges but didn’t run the installer? What do you see when you run foreman-rake db:migrate? If the command passes without the error, just restart the foreman service and it should work. To avoid such problems, always make sure to run the installer when you do the upgrade and make sure it does not fail during the process.

2 Likes

You are right, the columns has been added after running foreman-rake db:migrate

I didn’t know that I had to run it just for updating katello, maybe I missed it in the docs.

Thanks for your help!

Migrations used to be run automatically after the rpm upgrade, but that was removed at some point. It’s always good to run the installer which will take care of all required steps.

Glad it works now.