would anyone be against if we drop the “Run puppet” button from host detail page and bulk action menu? I know it still works if correctly configured, but given how complicated it is, I’d propose to support just one way of running puppet on demand, which would be through remote execution plugin. It would look like
this. As long as you have Foreman’s SSH key deployed to hosts, it will run out of the box. People using mcollective could easily write a job template that would trigger something like mco puppet runonce and link it to “run puppet” feature.
It also means we’d need to drop some API endpoints, which is not backwards compatible change.
I started working on it but before I get too far, I’d like to know if there is some showstopper.
I would appreciate this.
Getting the “run puppet” button to work correctly is quite a headache and my users here usually are quite confused about the differences and why those two things work differently.
I think we can safely remove this. My only concern is that we should provide documentation on how to set up similar permissions, e.g. how can you allow users to only run puppet on hosts and not arbitrary commands.
Yeah same permission granularity would be a must for us and as long as it supports mcollective/choria we’d be fine.
I’m not familiar with the remote execution plugin would it add the ability to force batching and batch interval etc on puppet runs executed via bulk actions?
Yes if you used the regular template that we ship. See description of concurrency level and time span. If you build a template that run mco, then the execution gets out of our hands. But if mco supports that as an argument, it would be possible to pass this value or define template input where user could set the batch size.
There is at least one scenario (which we are using right now) where the remote execution plug-in might not be able to replace the “Run Puppet” button. We are using the Puppet run provider “puppet_proxy_customrun” to execute a web hook, which connects to a API on a remote server, which then triggers a Puppet run on the remote foreman node. Our Foreman Servers/Proxies don’t have the ability to connect to the all our Puppet nodes directly via ssh or mco right now out of compliance reasons …
I think the only solution, to do the same thing with remote execution, might be a new (custom?) provider type which emulates the buttons custom run function.
I think that’s in fact very similar to the mco case I described above. You could write a job template and run that against the host, where you have smart proxy today. The content of this job template would be the same thing, you today call via customrun, the webhook. The only difference is the HTTP vs SSH for this communication, would that be a concern?
Note custom job template can be easily linked to the new “Run puppet” button. In this case it would require target host as user input, so one would need to go through the job invocation form. This would probably be less convenient.
Also could you please describe a bit more how does the web hook triggers the puppet run? Why is that considered safer than SSH/mco? I can understand sometimes the active connection is impossible since the host does not have accessible IP, but that shouldn’t be a problem with mco.
Thanks for your response. Using a job template and run it against one node e.g. the smart proxy to trigger the web hook is indeed less convenient, especially when you want to execute a puppet run on multiple hosts (which can be currently done easily by simply selecting the puppet nodes on the hosts page)
We are not using mco currently. Our puppet nodes are only accessible via a ssh bastion host, which I can not ssh into from our Foreman servers (security policies). However I can use a Web API to trigger certain jobs on that bastion host (like the execution of a puppet run on a remote server). I would not say that this is certainly more secure than using mco or ssh directly, but we are a big organization and sometimes we have to make a compromise to getting things done.
I think it would be possible for us, to get the permission to log-in to the bastion host directly from our Foreman servers to create a ssh tunnel via the bastion host to the puppet clients. This would enable us to use the remote exec plugin …
At the moment we are using ‘customrun’ I also think that is the only way you need to support it in Foreman as the DevOpps users can write their own script to run ‘puppet agent -t $OPTION’ on the supplied host name. In my case I am using bolt to run the remote command or just nohup if it is to run on the local host. No need to worry so much about mcollective as it has been deprecated. But for anyone still using mcollective your example should work fine wrapped up into a customrun script.
I see no major issues if you can give us developers / users some examples on how we can port our code over to the ‘new way’ of doing Run puppet.
mcollective has/will become choria (https://choria.io) still very much supported although separate from puppetlabs now.
I certainly don’t intend to stop using mcollective but as you say if we can write our own execution templates it should be fine.
I’m an newbie to foreman and I would like to know how it is possible to extend the “Schedule remote Job” drop-down list by the entry “Run Puppet Once” ?
Can you give me a hint ?
Best regards
Martin
it’s work in progress right now. This is the PR [1] that first needs to get in, then new remote execution plugin needs to be released. I’ll need to find some time to finish it.
A quick update on this, I have some WIP code in my git but I was unable to finish the removal. I think in order to keep the user experience, we’ll need to improve some REX capabilities first, such as have an input type where users can easily specify other hosts. Then it should be possible to replace the feature without dropping the user experience. We now work on improving template inputs for other types, such as date time or autocomplete search fields, so once this is done, we can continue.
OK, good that the button is still here but what is the way to make this work now ? I have seen too much noise around it the last couple of years and I wonder if mcollective/choria is really into that much of need for something simple.
It didn’t change at all. You need to configure your smart proxy to allow puppet run feature on it. See puppet run providers section in our manual. If you don’t need to enforce puppet run on demand from Foreman, don’t worry too much. It’s useful e.g. if you know you made some changes and you need to deploy them right-away. If you haven’t started configuring mcollective/choria for this, I’d recommend start using REX for this. There’s a WIP PR that adds “Run Puppet” to schedule remote job dropdown on host detail which will make it easier. In the same way, this can be triggered on multiple hosts.
Since the foreman integrates with several different configuration management tools, is there any WIP to trigger arbitrary (of the big four) configuration management manifest application? For example schedule remote job category for “run (ansible|salt|chef)”?
TL;DR: There are REX jobs to run puppet, chef and ansible.
In more details - we’re not using native tooling (mcollective/choria, chef push jobs), but connects to the VM over SSH and triggers one run. That means foreman-proxy needs to have connectivity to the VM. For an example, look at the puppet implementation. Chef plugin provides jobs on it’s own if it sees REX present. Ansible capabilities are provided by foreman_ansible plugin, it is a standalone REX provider and gives user ability to even define the inventory. For Salt I think there’s a PoC of a REX provider being worked on.
