Katello upgrade 3.3 --> 3.4 (Candlepin Suspend mode)

Support
1

Hi all, no luck with foreman upgrades.

I’m having an issue with Foreman upgrade from version 3.3 to 3.4.

yum update was successfully but the command

foreman-installer --scenario katello --upgrade

ends with:

Upgrade Step: clean_backend_objects (this may take a while) …
rake aborted!
RestClient::ServiceUnavailable: Katello::Resources::Candlepin::Consumer: 503 Service Unavailable {“displayMessage”:“Candlepin is in Suspend mode, please check /status resource to get more details”,“requestUuid”:“00ff119f-e4ed-43e7-8268-169dd5ffc9d2”} (GET /candlepin/consumers/6f1f5183-e960-4a2a-b592-6907ccb01eff)
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/abstract_response.rb:74:in return!' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:495:in process_result’
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:421:in block in transmit' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:413:in transmit’
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:176:in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:41:in execute’
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/resource.rb:51:in get' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/app/lib/katello/http_resource.rb:84:in get’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/app/lib/katello/resources/candlepin.rb:104:in get' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:23:in block in test_candlepin’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:32:in test_method' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:23:in test_candlepin’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:6:in block in cleanup_hosts' /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:51:in block (2 levels) in find_each’
/opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:51:in each' /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:51:in block in find_each’
/opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:124:in find_in_batches' /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:50:in find_each’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:5:in cleanup_hosts' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:67:in block (2 levels) in <top (required)>’
Tasks: TOP => katello:clean_backend_objects
(See full trace by running task with --trace)
foreman-rake katello:clean_backend_objects COMMIT=true failed! Check the output for error!
Upgrade step clean_backend_objects failed. Check logs for more information.
Upgrade Step: clean_backend_objects (this may take a while) …
rake aborted!
RestClient::ServiceUnavailable: Katello::Resources::Candlepin::Consumer: 503 Service Unavailable {“displayMessage”:“Candlepin is in Suspend mode, please check /status resource to get more details”,“requestUuid”:“00ff119f-e4ed-43e7-8268-169dd5ffc9d2”} (GET /candlepin/consumers/6f1f5183-e960-4a2a-b592-6907ccb01eff)
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/abstract_response.rb:74:in return!' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:495:in process_result’
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:421:in block in transmit' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:413:in transmit’
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:176:in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/request.rb:41:in execute’
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.8.0/lib/restclient/resource.rb:51:in get' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/app/lib/katello/http_resource.rb:84:in get’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/app/lib/katello/resources/candlepin.rb:104:in get' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:23:in block in test_candlepin’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:32:in test_method' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:23:in test_candlepin’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:6:in block in cleanup_hosts' /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:51:in block (2 levels) in find_each’
/opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:51:in each' /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:51:in block in find_each’
/opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:124:in find_in_batches' /opt/rh/sclo-ror42/root/usr/share/gems/gems/activerecord-4.2.5.1/lib/active_record/relation/batches.rb:50:in find_each’
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:5:in cleanup_hosts' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5/lib/katello/tasks/clean_backend_objects.rake:67:in block (2 levels) in <top (required)>’
Tasks: TOP => katello:clean_backend_objects
(See full trace by running task with --trace)
foreman-rake katello:clean_backend_objects COMMIT=true failed! Check the output for error!
Upgrade step clean_backend_objects failed. Check logs for more information.

We use Red Hat Content with version 3.3. is this a problem?

Thanks! BR,
cnu80

2

Hi, I found the problem and a solution.

After the upgrade “Node not found: pulp.agent…” messages are shown up in /var/log/messages.

I cleaned it up with the command:

for i in $(journalctl --all --this-boot | grep “error Error on attach: Node not found” | awk -F "error Error on attach: Node not found: " ‘{ print $2 }’ | cut -d. -f3 | sort -u); do qpid-config --ssl-certificate=/etc/pki/katello/qpid_client_striped.crt -b amqps://localhost:5671 add queue pulp.agent.$i --durable; done

Afterwards the katello upgrade finished successfully.

thanks to the Red Hat knowledge base.

BR, cnu80