Moving foreman_ansible_core to be a SPDC-only plugin

Hi all!

I am in the process of moving away from foreman_ansible_core. Its initial purpose was to be able to reuse the same code in the proxy and in Foreman itself. However this led to inconsistencies - some features would work only through the proxy or through Foreman, so eventually we dropped the ability of running roles from Foreman, then importing roles, and now the plugin is exclusively used through the proxy.

The proxy plugin contains a copy of foreman_ansible_core. https://github.com/theforeman/smart_proxy_ansible/tree/master/lib . The reason for this is simple: the gem was only built to run in the SCL environment. Since the smart-proxy runs on the non-SCL environment, I needed some way of getting the code to run there, so it was easy to just move it there.

This works well in Debian, as far as I know. For RHEL environments, I had to make a weird loading script (https://github.com/theforeman/smart_proxy_ansible/blob/master/lib/foreman_ansible_core.rb) which only loads certain parts of the gem in the SPDC (smart proxy dynflow core) environment and another part in smart-proxy.

Initially I thought I could nuke it completely https://github.com/theforeman/foreman_ansible/pull/213 and just move everything to the proxy. However, there are pieces of foreman_ansible_core that will need to keep on existing. Everything that modifies REX_core and tasks_core should be a gem in the SCL, I don’t see any other way.

So the final result should be something like:

  1. smart_proxy_ansible implements functions to read Ansible roles, read Ansible variables. These classes (RolesReader, VariablesReader) are removed from foreman_ansible_core and moved to smart_proxy_ansible.

  2. foreman_ansible_core implements functions to run Ansible. All of that functionality is related with REX_core and tasks_core. For example, overriding the REX_core settings to use the Ansible PlaybookRunner when a job contains an input ‘ansible_inventory’.

I have successfully tested point 1 in both dev and prod, however I have been struggling at testing point 2., mainly as I cannot get SPDC to run in a dev environment standalone. It seems to always run ‘attached’ to the proxy, which is useless for me, as that would not be the case in production. It has to run ‘standalone’ to mimic the behavior I see in production, where SPDC- runs in the SCL alongside REX_core, Tasks_core, Ansible_core, and the smart-proxy just connects to it.

Is it possible to fully remove foreman_ansible_core in this scenario I described? Am I misunderstanding anything?

cc @tasks @aruzicka @iNecas

1 Like

For the record, I have been able to test ‘2’ just fine now. If anyone wants to take a stab at it, to run SPDC as a standalone process, you need to make sure that the smart-proxy does not have 'gem ‘smart_proxy_dynflow_core’ on its bundle. Otherwise dynflow_core will run directly from the smart-proxy and the standalone process will be ignored.

1 Like

:+1: for this move: keeping 2 separate execution paths proved to be hard to maintain, and eventually lead to the all-in-one process to be broken for some time. Focusing on 1 path and doing it right seems like better investment of our time.

Seems like the question you had was already answered outside of this thread.

For the logging issue, you can apply this patch to get it working:

This will run, but you won’t see exceptions thrown from the wrapped loggers until another patch is merged into SPD:

https://projects.theforeman.org/issues/25239