Remote execution dynflow not initialized yet

Problem:
I want try using remote execution but when I try submit job then I got this error:
The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer
Expected outcome:
succesfully execute Job
Foreman and Proxy versions:
Foreman 3.7.0
Foreman and Proxy plugin versions:
foreman-tasks 8.0.2
foreman_puppet 5.1.2
foreman_remote_execution 10.0.1

Distribution and version:
Debian 11.7

How exactly did you install Foreman?

I just tried deploying a fresh instance and it seems to work out of the box.

I run foreman-installer
and after that:
foreman-installer --enable-foreman-plugin-proxmox --enable-foreman-plugin-remote-execution --enable-foreman-proxy-plugin-remote-execution-script

nothing else

That should be be fine. All this dynflow stuff should be set up when puma workers boot so that’s most likely the place to look at. What does systemctl status foreman and journalctl -u foreman give you?

systemctl status foreman

     Loaded: loaded (/lib/systemd/system/foreman.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/foreman.service.d
             └─installer.conf
     Active: active (running) since Wed 2023-07-26 15:48:15 CEST; 1h 12min ago
TriggeredBy: ● foreman.socket
       Docs: https://theforeman.org
   Main PID: 3603 (foreman-ruby)
     Status: "Puma 6.3.0: worker: { 5/5 threads, 5 available, 0 backlog }"
      Tasks: 16 (limit: 2318)
     Memory: 289.9M
        CPU: 26.642s
     CGroup: /system.slice/foreman.service
             └─3603 puma 6.3.0 (unix:///run/foreman.sock) [foreman]

Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: * Puma version: 6.3.0 (ruby 2.7.4-p191) ("Mugi No Toki Itaru")
Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: *  Min threads: 5
Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: *  Max threads: 5
Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: *  Environment: production
Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: *          PID: 3603
Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: * Activated unix:///run/foreman.sock
Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: * Starting control server on unix:///usr/share/foreman/tmp/sockets/pumactl.sock
Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: Sending status to systemd every 1.0 sec
Jul 26 15:48:15 foreman.feel4.cz foreman[3603]: Use Ctrl-C to stop
Jul 26 15:48:15 foreman.feel4.cz systemd[1]: Started Foreman.

Jul 26 13:04:04 foreman.feel4.cz systemd[1]: foreman.service: Succeeded.
Jul 26 13:04:04 foreman.feel4.cz systemd[1]: Stopped Foreman.
Jul 26 13:04:04 foreman.feel4.cz systemd[1]: foreman.service: Consumed 13.064s CPU time.
Jul 26 13:04:04 foreman.feel4.cz systemd[1]: Starting Foreman...
Jul 26 13:04:08 foreman.feel4.cz foreman[34279]: => Booting Puma
Jul 26 13:04:08 foreman.feel4.cz foreman[34279]: => Rails 6.1.7.3 application starting in production
Jul 26 13:04:08 foreman.feel4.cz foreman[34279]: => Run `bin/rails server --help` for more startup options
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: * Enabling systemd notification integration
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: Puma starting in single mode...
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: * Puma version: 6.3.0 (ruby 2.7.4-p191) ("Mugi No Toki Itaru")
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: *  Min threads: 5
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: *  Max threads: 5
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: *  Environment: production
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: *          PID: 34279
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: * Activated unix:///run/foreman.sock
Jul 26 13:04:16 foreman.feel4.cz foreman[34279]: Sending status to systemd every 1.0 sec

That somewhat differs from what I see on my box.

# systemctl status foreman
● foreman.service - Foreman
     Loaded: loaded (/lib/systemd/system/foreman.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/foreman.service.d
             └─installer.conf
     Active: active (running) since Wed 2023-07-26 14:55:49 UTC; 9min ago
TriggeredBy: ● foreman.socket
       Docs: https://theforeman.org
   Main PID: 26796 (foreman-ruby)
     Status: "Puma 6.3.0: cluster: 3/3, worker_status: [{ 5/5 threads, 5 available, 0 backlog },{ 5/5 threads, 5 available, 0 backlog },{ 5/5 threads, 5 available, 0 backlog }]"
      Tasks: 62 (limit: 14315)
     Memory: 704.5M
        CPU: 19.815s
     CGroup: /system.slice/foreman.service
             ├─26796 puma 6.3.0 (unix:///run/foreman.sock) [foreman]
             ├─27012 puma: cluster worker 0: 26796 [foreman]
             ├─27013 puma: cluster worker 1: 26796 [foreman]
             └─27014 puma: cluster worker 2: 26796 [foreman]

Jul 26 14:55:49 special-reptile foreman[26796]: [26796] *     Restarts: (✔) hot (✖) phased
Jul 26 14:55:49 special-reptile foreman[26796]: [26796] * Preloading application
Jul 26 14:55:49 special-reptile foreman[26796]: [26796] * Activated unix:///run/foreman.sock
Jul 26 14:55:49 special-reptile foreman[26796]: [26796] Use Ctrl-C to stop
Jul 26 14:55:49 special-reptile foreman[26796]: [26796] * Starting control server on unix:///usr/share/foreman/tmp/sockets/pumactl.sock
Jul 26 14:55:49 special-reptile foreman[26796]: [26796] Sending status to systemd every 1.0 sec
Jul 26 14:55:49 special-reptile foreman[26796]: [26796] - Worker 0 (PID: 27012) booted in 0.11s, phase: 0
Jul 26 14:55:49 special-reptile foreman[26796]: [26796] - Worker 2 (PID: 27014) booted in 0.11s, phase: 0
Jul 26 14:55:49 special-reptile foreman[26796]: [26796] - Worker 1 (PID: 27013) booted in 0.12s, phase: 0
Jul 26 14:55:49 special-reptile systemd[1]: Started Foreman.

What do you have in /etc/systemd/system/foreman.service.d/installer.conf?

[Service]
User=foreman
Environment=FOREMAN_ENV=production
Environment=FOREMAN_HOME=/usr/share/foreman
Environment=FOREMAN_PUMA_THREADS_MIN=5
Environment=FOREMAN_PUMA_THREADS_MAX=5
Environment=FOREMAN_PUMA_WORKERS=-1

The -1 for FOREMAN_PUMA_WORKERS is odd. Looking at where it comes from[1] I’d say your machine is a bit underpowered. You could probably try forcing it to something else by adding --foreman-foreman-service-puma-workers=1 to the installer, but ymmv. Getting a beefier machine would be better approach in the long run.

[1] - https://github.com/theforeman/puppet-foreman/blob/a4157ab77ed8589c3d312c13890a57b7b62d3599/manifests/config.pp#L78-L86C4

1 Like

For manual input we do verify the data type with as Integer[0], which sets 0 as a lower bound. However, we don’t do so for the formula because we didn’t anticipate it.

But even then, 0 seems wrong, shouldn’t it be Integer[1]?

So I have 6 CPU for virtual and 4GB of ram, after I run installer with puma workers=1 it started working. Im not sure if it right to let it run that way

with 4GB memory it should have created 2 workers. weird.
what’s the output of facter memory.system.total_bytes on your system?

Output is: 4108808192

Hmm, that’s 3.8GiB, minus 1.5 makes 2.3, floored 2. Yepp, still should have done 2 workers. weird.

if I will increase memory it will automaticaly increase workers after rerun foreman-insteller?

Now that you passed --foreman-foreman-service-puma-workers it will not, no.
But you can reset the value with --reset-foreman-foreman-service-puma-workers, and then it will become dynamic again.

0 workers means single (non-clustered) mode:

So thats mean in single mode you cant use foreman remote execute commnads and some more features? Until I force some workers?

after a while I figured i needed to not set FOREMAN_PUMA_WORKERS = >1 or foreman-installer --foreman-puma-workders with a value over 1, or not set it at all.
and boom, fresh deployment works with every tested version (did each from 3.2 to 3.7…)