Dropping "Run puppet" button with a replacement using remote execution plugin

Yes if you used the regular template that we ship. See description of concurrency level and time span. If you build a template that run mco, then the execution gets out of our hands. But if mco supports that as an argument, it would be possible to pass this value or define template input where user could set the batch size.

There is at least one scenario (which we are using right now) where the remote execution plug-in might not be able to replace the “Run Puppet” button. We are using the Puppet run provider “puppet_proxy_customrun” to execute a web hook, which connects to a API on a remote server, which then triggers a Puppet run on the remote foreman node. Our Foreman Servers/Proxies don’t have the ability to connect to the all our Puppet nodes directly via ssh or mco right now out of compliance reasons …

I think the only solution, to do the same thing with remote execution, might be a new (custom?) provider type which emulates the buttons custom run function.

I think that’s in fact very similar to the mco case I described above. You could write a job template and run that against the host, where you have smart proxy today. The content of this job template would be the same thing, you today call via customrun, the webhook. The only difference is the HTTP vs SSH for this communication, would that be a concern?

Note custom job template can be easily linked to the new “Run puppet” button. In this case it would require target host as user input, so one would need to go through the job invocation form. This would probably be less convenient.

Also could you please describe a bit more how does the web hook triggers the puppet run? Why is that considered safer than SSH/mco? I can understand sometimes the active connection is impossible since the host does not have accessible IP, but that shouldn’t be a problem with mco.

Thanks for your response. Using a job template and run it against one node e.g. the smart proxy to trigger the web hook is indeed less convenient, especially when you want to execute a puppet run on multiple hosts (which can be currently done easily by simply selecting the puppet nodes on the hosts page)

We are not using mco currently. Our puppet nodes are only accessible via a ssh bastion host, which I can not ssh into from our Foreman servers (security policies). However I can use a Web API to trigger certain jobs on that bastion host (like the execution of a puppet run on a remote server). I would not say that this is certainly more secure than using mco or ssh directly, but we are a big organization and sometimes we have to make a compromise to getting things done.

I think it would be possible for us, to get the permission to log-in to the bastion host directly from our Foreman servers to create a ssh tunnel via the bastion host to the puppet clients. This would enable us to use the remote exec plugin …

Cheers,
Martin

At the moment we are using ‘customrun’ I also think that is the only way you need to support it in Foreman as the DevOpps users can write their own script to run ‘puppet agent -t $OPTION’ on the supplied host name. In my case I am using bolt to run the remote command or just nohup if it is to run on the local host. No need to worry so much about mcollective as it has been deprecated. But for anyone still using mcollective your example should work fine wrapped up into a customrun script.

I see no major issues if you can give us developers / users some examples on how we can port our code over to the ‘new way’ of doing Run puppet.

mcollective has/will become choria (https://choria.io) still very much supported although separate from puppetlabs now.
I certainly don’t intend to stop using mcollective but as you say if we can write our own execution templates it should be fine.

Hi Marek,

I’m an newbie to foreman and I would like to know how it is possible to extend the “Schedule remote Job” drop-down list by the entry “Run Puppet Once” ?
Can you give me a hint ?
Best regards
Martin

Hello,

it’s work in progress right now. This is the PR [1] that first needs to get in, then new remote execution plugin needs to be released. I’ll need to find some time to finish it.

[1] https://github.com/theforeman/foreman_remote_execution/pull/407

Hi Marek,

thank you for your answer. I will wait for the release then.

Best regards

Martin

A quick update on this, I have some WIP code in my git but I was unable to finish the removal. I think in order to keep the user experience, we’ll need to improve some REX capabilities first, such as have an input type where users can easily specify other hosts. Then it should be possible to replace the feature without dropping the user experience. We now work on improving template inputs for other types, such as date time or autocomplete search fields, so once this is done, we can continue.

1 Like

Glad to hear that work is in good progress. I’m nosey :slight_smile:

OK, good that the button is still here but what is the way to make this work now ? I have seen too much noise around it the last couple of years and I wonder if mcollective/choria is really into that much of need for something simple.

Can you please explain @Marek_Hulan ?

It didn’t change at all. You need to configure your smart proxy to allow puppet run feature on it. See puppet run providers section in our manual. If you don’t need to enforce puppet run on demand from Foreman, don’t worry too much. It’s useful e.g. if you know you made some changes and you need to deploy them right-away. If you haven’t started configuring mcollective/choria for this, I’d recommend start using REX for this. There’s a WIP PR that adds “Run Puppet” to schedule remote job dropdown on host detail which will make it easier. In the same way, this can be triggered on multiple hosts.

Since the foreman integrates with several different configuration management tools, is there any WIP to trigger arbitrary (of the big four) configuration management manifest application? For example schedule remote job category for “run (ansible|salt|chef)”?

TL;DR: There are REX jobs to run puppet, chef and ansible.

In more details - we’re not using native tooling (mcollective/choria, chef push jobs), but connects to the VM over SSH and triggers one run. That means foreman-proxy needs to have connectivity to the VM. For an example, look at the puppet implementation. Chef plugin provides jobs on it’s own if it sees REX present. Ansible capabilities are provided by foreman_ansible plugin, it is a standalone REX provider and gives user ability to even define the inventory. For Salt I think there’s a PoC of a REX provider being worked on.

oh I see that makes more sense. I didn’t see the other jobs because I don’t
have the plugin installed, but the run puppet button is in the interface
regardless. Thanks for the clarification!

… and there are REX jobs to run Salt.

So this is now implemented, user input value type can be of value type search, users can specify target hosts in a search syntax and use load_hosts macro (probably only available in reports scope now). That would address the need for users who would still use mco/choria due to push base is not a possibility in their infra. Sadly, I have no longer time to finish this effort, if someone is interested in taking over the PR, here it is. I’m closing it for now.

Pending tasks:

  • rebase the dropping PR
  • make load_hosts macro available in job templates
  • optionally provide a job template with input of hosts that would trigger mco/choria at least as an example

I will bring this old topic to live. I’ve starting moving Puppetrun to REX plugin. So that also means remove puppetrun (https://github.com/theforeman/foreman/pull/7719) code from core. PR is inspired from previous Marek’s closed PR.

3 Likes

@dmatoulek have these two tasks been completed already?