I had to reboot my foreman/katello server as well as a separate katello content proxy on another server. However, the proxy wasn’t running when the main server started.
Now I have a task “Pulp disk space notification” hanging for hours. It is state ‘running’, result ‘running’. However, it’s not really running but only has an error for Action Dynflow::ActiveJob::QueueAdapters::JobWrapper, exception
Foreman::WrappedException: ERF50-5345 [Foreman::WrappedException]: Unable to connect ([ProxyAPI::ProxyException]: ERF12-3580 [ProxyAPI::ProxyException]: Unable to detect pulp storage ([RestClient::Exceptions::OpenTimeout]: Timed out connecting to server) for proxy https://foreman-proxy.example.com:9090/pulpnode/status/disk_usage)
I have restarted the services on the main server (foreman-maintain service restart) which usually helps with the hanging repo sync tasks, but not with this one. It still hangs there. I cannot cancel it. I don’t know how to get rid of it.
Expected outcome:
Task ended with failure/exception instead of running.
This should be fixed starting with dynflow-1.4.5. The fix will prevent this from occuring again, but the borked tasks you already have will stay this way. For getting rid of the ones you already have I’d recommend using the foreman_tasks:cleanup rake task.
# foreman-rake foreman_tasks:cleanup NOOP=true VERBOSE=true STATES='running' TASK_SEARCH='label = CreatePulpDiskSpaceNotifications'
rake aborted!
The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.3/lib/dynflow/rails.rb:75:in `world'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:80:in `initialize'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:8:in `new'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:8:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/tasks/cleanup.rake:37:in `block (3 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => foreman_tasks:cleanup => foreman_tasks:cleanup:run
(See full trace by running task with --trace)
