Dynflow database or not?

Problem: Smart-Proxy dynflow is issuing a warning “Could not open DB for dynflow at ‘’, will keep data in memory. Restart will drop dynflow data.” but when looking at the puppet module there is a comment “use in-memory sqlite by default for performance reasons”, so it seems intentional. But if it is better to run without, then why a warning? And as we had problems in the environment with tasks during offline backup, I am not sure which option would be better.

Expected outcome: Best practice without a warning.

Foreman and Proxy versions: 3.2

Foreman and Proxy plugin versions: Katello 4.4

Distribution and version: CentOS 7 / CentOS Stream 8

@aruzicka is this a question you can answer?

Up until this month[1] or so, smart_proxy_dynflow itself defaulted to a on-disk database. The implications of flipping it over to using an in-memory one may not be immediately obvious, hence the warning about it.

For the use in remote execution (dynflow on the smart proxy is not really used for anything else), we do not require data to be persisted on the smart proxy across process restarts and on-disk sqlite is not really good with concurrent access. We took this into consideration and opted to get more performance at the cost of occasionally losing historical data. Here we made the decision for the users, but the users can still override that if they determine they need higher data persistence.

I would be surprised if you would experience something different when switching to an on-disk database for dynflow on the smart proxy. With that being said, which tasks do you have in mind?

[1] - Fixes #34654 - Default to in-memory database by ekohl · Pull Request #108 · theforeman/smart_proxy_dynflow · GitHub

Thanks Adam for clarification! So if this is best practice, I am not sure if the smart proxy should create a warning here. How about changing the wording and log level, so it is green in the UI and says something like “Using in-memory database to provide better performance. Restart will drop dynflow data.”? I can create a pull request if you like it.

This was the thread one internal colleague opened about this: Recurring logics Fails with Smart Proxy Errors - #7 by aruzicka
I also changed the backup to online, but as I missed the change to dynflow I thought this could also be an additional problem.

I’m not convinced that a message about data not being persisted is less important than a message about using default logging settings:

2022-04-08T11:01:02  [W] Couldn't find settings file /etc/foreman-proxy/settings.d/logs.yml. Using default settings.

In this case we can’t really distinguish between it being a default, us (the installer) setting it or the user setting it explicitly by hand. We can’t really say who and why configured it this way, so the claim about the reasoning behind it might be misleading.

I would say this is something the users should be aware of and I’m afraid noone reads the logs if they’re all green.

With that being said, if you do open a PR, I won’t object.

I totally agree, but on the other hand I then try to avoid warnings that are misleading to believe something has to be changed as I do like to teach customers and colleagues to not ignore warnings. This is why I am asking. But I also do not have such a strong opinion that I see it as a must have. Lets see if we can get some other opinions here, I will also ask the customer and then decide on the pull request.