Hi, thank you for bringing this up.
In this context “X workers in the pool” means, “the dynflow executor process has X threads which function as workers”. Because of how ruby handles threads (GIL and all that), you will start to see diminishing returns if you ramp it up too high. From my observation, 5 is a good default, with 15 being the value above which it just doesn’t really help any more. I’ve seen people trying to set this into the hundreds range and that’s a no-no.
Increasing the number of executors should be a safe thing to do, but it has its own drawbacks, which are described in the upcoming Dynflow changes post[2].
Also please note that this changes the size of the pool for the default queue. Some actions use different queues, so changing this shouldn’t have any impact of processing them. As a relevant example I’d like to mention the hosts_queue, tracked by BZ1721679[1].
May I propose one more test? Maybe you have seen my proposal/report about upcoming changes to Dynflow[2]. I’d be more than happy to work with you to set it up on the testing machine, run the tests there and explore which scaling options it gives us.
In general I don’t have anything against moving the host actions from Dynflow as described in BZ 1758285, but maybe it isn’t/won’t be necessary.
[1] - https://bugzilla.redhat.com/show_bug.cgi?id=1721679
[2] - Upcoming changes to Dynflow - #21 by aruzicka