Remote execution regression in 2.5 - have to set remote_execution_ssh_user per host

Support
1

Problem:

Configuring Remote Execution in a new 2.5 installation. RE does not work with ‘SSH User’ configured in Settings -> Remote Execution. Configuring remote_execution_ssh_user per host with the same value works OK, though. System messages show login attempts as root@target.example.com instead of expected user @target.example.com.

Seems that RE module / proxy is ignoring the Settings -> Remote Execution -> ‘SSH User’ config item.

Expected outcome:

Expect RE module / proxy to utilise the Settings -> Remote Execution -> ‘SSH User’ config item.

Foreman and Proxy versions:

Foreman Version: 2.5.0
Foreman-proxy Version: 2.5.0
Dynflow version: 0.3.0

Foreman and Proxy plugin versions:

foreman-tasks: 4.1.1
foreman_bootdisk: 17.1.0
foreman_discovery: 17.0.0
foreman_expire_hosts: 7.0.4
foreman_hooks: 0.3.17
foreman_openscap: 4.3.0
foreman_remote_execution: 4.5.0
foreman_setup: 7.0.0
foreman_statistics: 1.1.1
foreman_templates: 9.0.2

Distribution and version:

RHEL 8.4

Other relevant data:

2021-06-07T08:22:29 acbb1559 [I] Started GET /dynflow/tasks/count state=running
2021-06-07T08:22:29 acbb1559 [I] Finished GET /dynflow/tasks/count with 200 (1.76 ms)
2021-06-07T08:22:30 acbb1559 [I] Started GET /version
2021-06-07T08:22:30 acbb1559 [I] Finished GET /version with 200 (0.87 ms)
2021-06-07T08:22:30 acbb1559 [I] Started POST /dynflow/tasks/launch
2021-06-07T08:22:30 acbb1559 [I] Finished POST /dynflow/tasks/launch with 200 (66.47 ms)
2021-06-07T08:22:30 [E] error while initalizing command Net::SSH::AuthenticationFailed Authentication failed for user root@foreman.example.com:
/usr/share/gems/gems/net-ssh-4.2.0/lib/net/ssh.rb:254:in `start’

2

Just a wild guess, but does the issue go away if you restart all dynflow-sidekiq services with systemctl restart dynflow-sidekiq@*?

3

Hi Adam,

Many thanks - yes, that seems to be the fix.

I’ve added it to my box of tricks.

Best regards,

Ross

4

Since 2.5 some things changed and dynflow-sidekiq services don’t notice when settings change and merrily use the old values so this is definitely something we should fix. CC @ezr-ondrej

5

If that’s the case, then I think if you have multiple Foreman application servers (like in a load balanced setup) they also won’t change the value until you restart.

6

Hi Ewoud,

nod fair enough. I had restarted the forman-proxy service to attempt to clear the issue, however that was insufficient. Restarting dynflow-sidekiq processes fixed it.

I have a parallel 2.3.2 instance which didn’t exhibit this behaviour, so I had suspected a regression of some kind…

Many thanks,

Ross

7

I’ve opened fix for this and it is tracked by Bug #32729: Dynflow doesn't reflect changes in Setting - foreman-tasks - Foreman