Hi all,
I have a problem with the configuration of redis and dynflow, I want to use an external redis cluster deployed on 3 servers, so I modified the unit systemd /usr/lib/systemd/system/dynflow-sidekiq@.service
Environment=DYNFLOW_REDIS_URL=redis://default:mykey@srv1-tst.mydomain.com:7000/0,redis://default:mykey@srv2-tst.mydomain.com:7000/0,redis://default:mykey@srv3-alh.mydomain.com:7000/0
here’s the error
Mar 15 11:17:02 app01-alh systemd[1]: Starting Foreman jobs daemon - orchestrator on sidekiq...
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: 2024-03-15T10:17:02.220Z pid=196594 tid=46su INFO: Enabling systemd notification integration
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: 2024-03-15T10:17:02.785Z pid=196594 tid=46su INFO: Booting Sidekiq 6.5.12 with Sidekiq::RedisConnection::RedisAdapter options {:url=>"redis://default:REDACTED@srv1-tst.mydomain.com:7000/0,redis://default:REDACTED@srv2-tst.mydomain.com,redis://default:REDACTED@srv3-tst.mydomain.com
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: 2024-03-15T10:17:02.786Z pid=196594 tid=46su WARN: Redis::CommandError: MOVED 7798 10.249.16.24:7000
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: 2024-03-15T10:17:02.786Z pid=196594 tid=46su WARN: /usr/share/gems/gems/redis-4.5.1/lib/redis/client.rb:163:in `call'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/redis-4.5.1/lib/redis.rb:1678:in `block in zcard'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/redis-4.5.1/lib/redis.rb:72:in `block in synchronize'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/ruby/monitor.rb:202:in `synchronize'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/ruby/monitor.rb:202:in `mon_synchronize'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/redis-4.5.1/lib/redis.rb:72:in `synchronize'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/redis-4.5.1/lib/redis.rb:1677:in `zcard'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/api.rb:645:in `block in size'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq.rb:164:in `block in redis'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/connection_pool-2.4.1/lib/connection_pool.rb:110:in `block (2 levels) in with'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/connection_pool-2.4.1/lib/connection_pool.rb:109:in `handle_interrupt'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/connection_pool-2.4.1/lib/connection_pool.rb:109:in `block in with'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/connection_pool-2.4.1/lib/connection_pool.rb:106:in `handle_interrupt'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/connection_pool-2.4.1/lib/connection_pool.rb:106:in `with'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq.rb:161:in `redis'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/api.rb:645:in `size'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/api.rb:640:in `initialize'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/gitlab-sidekiq-fetcher-0.9.0/lib/sidekiq/interrupted_set.rb:9:in `initialize'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/gitlab-sidekiq-fetcher-0.9.0/lib/sidekiq/base_reliable_fetch.rb:119:in `new'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/gitlab-sidekiq-fetcher-0.9.0/lib/sidekiq/base_reliable_fetch.rb:119:in `initialize'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/gitlab-sidekiq-fetcher-0.9.0/lib/sidekiq/semi_reliable_fetch.rb:11:in `initialize'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/gitlab-sidekiq-fetcher-0.9.0/lib/sidekiq/base_reliable_fetch.rb:56:in `new'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/gitlab-sidekiq-fetcher-0.9.0/lib/sidekiq/base_reliable_fetch.rb:56:in `setup_reliable_fetch!'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/core.rb:13:in `block in <top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq.rb:142:in `configure_server'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/sidekiq/core.rb:9:in `<top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `block in require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:299:in `load_dependency'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/parallel.rb:8:in `<class:Parallel>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/parallel.rb:4:in `<module:Executors>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/parallel.rb:3:in `<module:Dynflow>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors/parallel.rb:2:in `<top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `block in require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:299:in `load_dependency'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors.rb:5:in `<module:Executors>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors.rb:3:in `<module:Dynflow>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow/executors.rb:2:in `<top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `block in require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:299:in `load_dependency'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow.rb:64:in `<module:Dynflow>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/dynflow-1.8.2/lib/dynflow.rb:20:in `<top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:158:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:158:in `rescue in require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:147:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `block in require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:299:in `load_dependency'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/foreman/lib/foreman/dynflow.rb:1:in `<top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `block in require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:299:in `load_dependency'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/activesupport-6.1.7.6/lib/active_support/dependencies.rb:332:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/foreman/config/application.rb:27:in `<top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/foreman/extras/dynflow-sidekiq.rb:6:in `<top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:83:in `require'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/cli.rb:290:in `boot_application'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/lib/sidekiq/cli.rb:53:in `run'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/share/gems/gems/sidekiq-6.5.12/bin/sidekiq:31:in `<top (required)>'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/bin/sidekiq:23:in `load'
Mar 15 11:17:02 app01-alh dynflow-sidekiq@orchestrator[196594]: /usr/bin/sidekiq:23:in `<main>'
Mar 15 11:17:02 app01-alh systemd[1]: dynflow-sidekiq@orchestrator.service: Main process exited, code=exited, status=1/FAILURE
Mar 15 11:17:02 app01-alh systemd[1]: dynflow-sidekiq@orchestrator.service: Failed with result 'exit-code'.
Mar 15 11:17:02 app01-alh systemd[1]: Failed to start Foreman jobs daemon - orchestrator on sidekiq.
is cluster mode managed by foreman ?