Smart Proxy is no longer communicating: Not all necessary pulp workers running at https://smartproxy.lab.lan/pulp/api/v3

Problem:
I detected an issue, which apparently happened a few weeks ago. After one of the later upgrades (I don’t recall which one was done when this error started appearing), the smart proxy doesn’t sync anymore. Neither full sync nor incremental works, also the sync after publishing a new content version.

When I click on the smart proxy in the GUI, I get this message:
Oops, we’re sorry but something went wrong Not all necessary pulp workers running at https://smartproxy.lab.lan/pulp/api/v3.

Expected outcome:
Smart Proxy to sync.

Foreman and Proxy versions:
Foreman 3
Katello 4.2

Foreman and Proxy plugin versions:

Distribution and version:
CentOS 7.9

Other relevant data:

foreman-rake errors:fetch_log request_id=9c96379d
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
2021-10-21T13:56:44 [I|app|9c96379d] Started GET "/smart_proxies/2-smartproxy-lab-lan/pulp_storage" for 10.10.14.55 at 2021-10-21 13:56:44 +0200
2021-10-21T13:56:44 [I|app|9c96379d] Processing by SmartProxiesController#pulp_storage as HTML
2021-10-21T13:56:44 [I|app|9c96379d]   Parameters: {"id"=>"2-smartproxy-lab-lan"}
2021-10-21T13:56:45 [W|app|9c96379d] Not all necessary pulp workers running at https://smartproxy.lab.lan/pulp/api/v3.
2021-10-21T13:56:45 [I|app|9c96379d] Backtrace for 'Not all necessary pulp workers running at https://smartproxy.lab.lan/pulp/api/v3.' error (RuntimeError): Not all necessary pulp workers running at https://smartproxy.lab.lan/pulp/api/v3.
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.1/app/models/katello/ping.rb:200:in `pulp3_without_auth'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.1/app/models/katello/concerns/smart_proxy_extensions.rb:375:in `ping_pulp3'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.1/app/models/katello/concerns/smart_proxy_extensions.rb:180:in `pulp_disk_usage'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.1/app/controllers/katello/concerns/smart_proxies_controller_extensions.rb:29:in `pulp_storage'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/base.rb:195:in `process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rendering.rb:30:in `process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
 9c96379d | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in `set_timezone'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 9c96379d | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 9c96379d | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:139:in `run_callbacks'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/callbacks.rb:41:in `process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/rescue.rb:22:in `process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in `block in instrument'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications.rb:180:in `instrument'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/abstract_controller/base.rb:136:in `process'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionview-6.0.3.7/lib/action_view/rendering.rb:39:in `process'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal.rb:190:in `dispatch'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_controller/metal.rb:254:in `dispatch'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:33:in `serve'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:49:in `block in serve'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `each'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/journey/router.rb:32:in `serve'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/routing/route_set.rb:834:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.1/lib/katello/middleware/organization_created_enforcer.rb:18:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.1/lib/katello/middleware/event_daemon.rb:10:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-dsl-2.4.0/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 9c96379d | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
 9c96379d | /usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/conditional_get.rb:27:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
 9c96379d | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/cookies.rb:648:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/callbacks.rb:101:in `run_callbacks'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/rack/logger.rb:37:in `call_app'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/rack/logger.rb:28:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
 9c96379d | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/request_id.rb:27:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.2.0.1/lib/katello/prevent_json_parsing.rb:12:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/executor.rb:14:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/static.rb:126:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.7/lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/engine.rb:527:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `public_send'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.7/lib/rails/railtie.rb:190:in `method_missing'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/configuration.rb:249:in `call'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/request.rb:77:in `block in handle_request'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/thread_pool.rb:338:in `with_force_shutdown'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/request.rb:76:in `handle_request'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/server.rb:438:in `process_client'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-5.3.2/lib/puma/thread_pool.rb:145:in `block in spawn_thread'
 9c96379d | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2021-10-21T13:56:45 [I|app|9c96379d]   Rendering common/500.html.erb
2021-10-21T13:56:45 [I|app|9c96379d]   Rendered common/500.html.erb (Duration: 13.9ms | Allocations: 5983)
2021-10-21T13:56:45 [I|app|9c96379d] Completed 500 Internal Server Error in 677ms (Views: 15.2ms | ActiveRecord: 17.6ms | Allocations: 27071)

Foreman version: 3.0.0
Plugins:
 - foreman-tasks 5.1.0
 - foreman_ansible 6.4.1
 - foreman_puppet 1.0.3
 - foreman_remote_execution 4.7.0
 - katello 4.2.0.1