Dynflow-sidekick worker and orchestrator error

Support
1

Problem:

Hi,

After a successfull standard installation of Foreman i’m unable to login. I’m get invalid password/username (password/username are good, given by output terminal at the end of installation)

I have dynflow-sidekick worker and orchestrator error with heartbeat Redis server
connection.

 dynflow-sidekiq@orchestrator[14980]: 2021-02-18T12:44:55.373Z 14980 TID-gpaoy8e64 ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)
dynflow-sidekiq@worker[15666]: 2021-02-18T12:51:19.414Z 15666 TID-gmxf62fui ERROR: Heartbeat thread error: Error connecting to Redis on localhost:6379 (Errno::ECONNREFUSED)

Expected outcome:

Foreman and Proxy versions:
Foreman 2.3
Foreman and Proxy plugin versions:

Distribution and version:
Ubuntu 18.04 Bionic
Other relevant data:

2
  • foreman-proxy is running
  • foreman is running
  • redis is running and listening
  • postgresql is running and listening
  • puppet is running and listening
3

Do you know how dynflow-sidekick works with heartbeat ?

https://dynflow.github.io/documentation/

Regards

4

Hi,
It seems highly unlikely that any issue with dynflow could prevent you from logging in. Have you by any chance ran the installer a few times?

The heartbeat in question is not dynlfow’s heartbeat, it is something that sidekiq uses to know which workers are still alive. The workers are connected to redis and periodically advertise that they are still there. Does the error keep popping up or did it appear only once?

5

Hi @aruzicka ,

Thank you for these explanation.

I run installer twice because postgresql was not working properly. I removed and reinstall postgresql, foreman installer.
I removed redis-server and reinstall it.
There heartbeat error appears every second in /var/log/syslog

I can’t see login error in foreman logs…

6

In /var/log/foreman/production.log i have this :

2021-02-18T15:14:38 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2021-02-18T15:14:38 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2021-02-18T15:14:41 [I|app|06c323ca] Started GET "/node/foreman.mydomain.com?format=yml" for 127.0.0.1 at 2021-02-18 15:14:41 +0100
2021-02-18T15:14:41 [I|app|06c323ca] Processing by HostsController#externalNodes as YML
2021-02-18T15:14:41 [I|app|06c323ca]   Parameters: {"name"=>"foreman.mydomain.com"}
2021-02-18T15:14:41 [I|app|06c323ca]   Rendering text template
2021-02-18T15:14:41 [I|app|06c323ca]   Rendered text template (Duration: 0.0ms | Allocations: 3)
2021-02-18T15:14:41 [I|app|06c323ca] Completed 200 OK in 110ms (Views: 1.1ms | ActiveRecord: 31.7ms | Allocations: 18358)
2021-02-18T15:14:44 [I|app|ecde389b] Started POST "/api/hosts/facts" for 127.0.0.1 at 2021-02-18 15:14:44 +0100
2021-02-18T15:14:44 [I|app|ecde389b] Processing by Api::V2::HostsController#facts as JSON
2021-02-18T15:14:44 [I|app|ecde389b]   Parameters: {"facts"=>"[FILTERED]", "name"=>"foreman.mydomain.com", "certname"=>"foreman.mydomain.com", "apiv"=>"v2", "host"=>{"certname"=>"foreman.mydomain.com", "name"=>"foreman.mydomain.com"}}

Processing by UsersController#login as HTML
2021-02-18T15:18:31 [I|app|52aec7e7]   Parameters: {"login"=>{"login"=>"admin", "password"=>"[FILTERED]"}, "authenticity_token"=>"hYCphAoa8bWHnpVKs8stiGNI521VExjbPdG1FQqDkZVifLlGiR/kBDE1mP1UB0XSoa/3+Iw=="}
2021-02-18T15:18:31 [I|app|52aec7e7] invalid user
2021-02-18T15:18:31 [W|app|52aec7e7] Failed login attempt from 192.168.XX.XX with username 'admin'
2021-02-18T15:18:31 [I|app|52aec7e7] Redirected to https://foreman.mydomain.com/users/login

Is there a problem with hostname or /etc/hosts resolving ?

7

So weird after a sudo foreman-rake permissions:reset login is OK

9

Hi,

localhost was missing in /etc/hosts on 127.0.0.1 line. So redis was listening on another hostname.

After add localhost and restart redis service, worker and orchestrator are connecting on redis successfully :smiley:

10

It sounds like it was a combination of several smaller issues, leading to quite interesting behavior. Glad to hear you managed to resolve it

1 Like
11

Thank you for your help ! :+1: