Recurring Logics don't recur

Problem:
I configured two recurring tasks, however, it seems like only the last configured task is actually recurring. The first one doesn’t even bother to run.

Expected outcome:
All configured tasks to run

Foreman and Proxy versions:
Foreman: 1.22
Proxy: 1.22

I’ve also had this in the 1.21 version unfortunatly.
ALso, I’ve checked the logs but don’t seem to find any relevant parts?

Hi, I’m afraid you’ll have to elaborate a bit more on the issue. How did you configure the recurring tasks?

Hi @aruzicka,

Sorry, you’re right, I was a bit short on information.
The overview screen already shows me some “interesting” information:

As you can see, the next occurence is set to happen on the 29th of July; which is passed already but it never really runs.

Please, let me know if there is more information I can provide. Looking at the overview screen, it seems it provides the same information as the detail screen so I didn’t really screenshot those.

Kind regards,
Arend

If you could go to the status tab of dynflow console (/foreman_tasks/dynflow/worlds iirc), is there an executor which has delayed_executor => true in its metadata?

Hi,

This is the only entry I see for executors. Below is a table with a few clients as well (but I’m not familiar with dynflow…).

Executors

Id: ce159c71-75fd-4377-9d4c-c8641c80a42b

Metadata: {“hostname”=>“redacted-foreman-server.com”, “pid”=>5602, “queues”=>{“default”=>{“pool_size”=>5}, “remote_execution”=>{“pool_size”=>5}}, “delayed_executor”=>true}

Last seen: 2019-08-14 13:30:41.182

Kind regards,
Arend

That looks good, what if you go to the execution plans tab in the dynflow console and select only “scheduled” plans to be shown. Are there any which would match the tasks that should be triggered by the recurring logics?

Hi,

I did as you asked:

However, only the first entry actually seems to list a scheduled task (the other task is not mentioned):

The other two basically have the same info:

Kr,
Arend

Last thing, just out of curiosity, could you paste contents of dynflow_delayed_records table?

It seems that we have lost track of tasks backing that recurring logic and sadly there is currently no way how to force it to start again.

I’m afraid the only possibility right now is to cancel the stuck one and schedule it again.

Hi,

Euhm, I’m unsure where I can fetch the dynflow_delayed_records table?

Also, I’ve recreated these tasks a few times already, seems to be fairly reproducible on my systems; as soon as I create the new one, the older recurring tasks doesn’t work anymore :frowning:

Kind regards,
Arend

FYI - I ran into the exact same issue on a DEV instance that I updated to 1.22 recently. I only had a few jobs so I simply rescheduled them.

Okay, will try rescheduling, could be that this occurred after the upgrade

Hi,

So, I did recreate the scheduled tasks, but it seems to only run once. After which the tasks is never executed again. Configuration wise both scheduled tasks are the same (except for the job they execute ofcourse).

I also checked dynflow and it confirms there 1 only one of my recurring jobs scheduled => the one that I haven’t recreated. (I didn’t expect this)

Kind regards,
Arend

Could you bump log level to debug, try creating a new one, waiting for it to trigger (or not?) and then posting the logs somewhere?