General Questions about Foreman usage for VM Provisionning


We are using Foreman for a little time now and I wanted to share my experience on the product and see if we are wrong in our way we use the product…

I first looked on the web for a tool :
1/ To provision in a simple way Virtual Machines on our VMWare infrastructure
2/ To provision automatically, along with the VM, our different infrastructure tools (DNS, IPAM, Monitoring, Backup server…)
3/ To be able to push applications & configuration to those virtual machines
4/ To be able to execute, on demand, some commands & scripts on several VM at the same time
5/ To be able to monitor & deploy security patches on our VM
6/ To be able to execute, on demand, some commands & scripts on several network devices at the same time
7/ To be stable and simple !

Then I found Foreman who seems to perfectly fit our needs.
However, after some time using it, here is some problems or feature requests I’d like to share for each listed items.

1/ We are using VM Provisionning with User Data template (because boot PXE is not an option for us : too complex on the network side, longer than template provisionning). It works fine, however I find it a little too complex to deploy a new VM. Let’s take an exemple :

  • You click on “Create Host”
  • On the “Host” tab, you choose VM name and Host Group. OK
  • On the “Virtual Machine” tab you must choose CPU, RAM, Disk… OK. But you must also choose some irrelevant information for a template deployment : Firmware, Guest OS, Virtual HW version… all these are included in the VMWare template and therefore ignored.
    One feature also would be to be able to choose the specific ESX Host on which we want the VM to be deployed. Currently, it is always deployed on the Host where the template is stored, which force us to manually move the VM afterwards.
  • On the “Operating System” tab, you must again choose Architecture, Operating System, the image name…
  • Then finally you must configure IP information on the “Interfaces” tab. We are using an external IPAM tool, called Netbox. It would be really great that Foreman could work with external IPAM so that it only proposes free IPs.
    To sum up, it would be great to have a “Create Host from Template” button, asking only for the needed informations : destination ESX, Image Name, CPU/RAM/Disk, Interface.
    Also, a VMWare Template comes with a default disk size. When we choose a different disk size, we must manually resize it on the OS level. So maybe it would be great to gather information on the template, display it, and let the user decide whether he wants to add a new disk or change the template disk size

2/ After VM deployment, we would like to execute some commands on the Host and provision several different tools. To do so, we are using Hooks. First feature request : be able to configure Hooks through the WebUI, because actually it’s just a script on the foreman server… not very user friendly.
We use a “create” hook that just does a “touch file”. Then we use a “after_commit” hook to execute the real script because during the create or post create we don’t have access to all the Host information (see “Problem with Hooks events” support entry).
The problem of doing so is that there is no graphical information, during Host Creation, whether the process was ok or not.
Our after_commit hooks are currently :

  • Creating all VM and IP information on our Netbox IPAM through webservices
  • Deploying the Foreman Proxy SSH Key so that Foreman Remote Execution can work
    We would like also to be able to, in the future :
  • Automatically provision our Network Monitoring tool (Centreon)
  • Automatically provision our Backup tool
  • Automatically update our DNS server (Windows AD)
    Last thing, when we delete a Host we must be able to automatically “unprovision” all those tools.
    To do so, we are currently using a “destroy” hook, but again without any information on the WebUI if the process was ok or not.
    To sum up : are hooks the right way to achieve all this ? If yes, it would be great that hooks can be completely integrated in the WebUI instead of having to change script files.

3/ To push application & configuration to deployed VM, we are mainly using Puppet (because it is the most integrated into Foreman). However, Puppet requires an Agent on the VM and some network port to be opened (which is sometime forbidden by our customers security policy).
Puppet Agent is deployed, after Host Creation, through a Remote Execution job template.
For our “complicated” hosts, we are using Ansible. Far more simple as it does not require an agent and just SSH connection. However, Ansible is today not very integrated in the WebUI.
To sum up : it would be great to have a real Ansible integration on the WebUI.

4/ Remote Execution is a great tool, however when I run it on several Host at a time, it often fails for some hosts, without good reason. For instance, I just tried to execute “uname -a” on 12 Linux Hosts. 58% Failed with “Net::SSH::AuthenticationFailed”. However, if I rerun only failed Hosts several times, it ends up to work on all of them. This Module does not seem very reliable…

5/ We are using CentOS and Ubuntu Linux Distribution. As far as I understand, Katello plugin does only deal with rpm packages. Is there a solution for monitoring & deploying security fixes regardless of the OS distribution ?

6/ Here, it is the only thing were Foreman clearly does not do the Job. First, Host Creation seems to be always tied to a deployment (through PXE or VM Template). Not possible to simply create just an empty Host representing a Network Device (or even a remote Standard Host not deployed by Foreman). I had to do so through a job template and a hammer command.
Then, for Network Devices, Remote Execution does not work because it is build in such a way that it needs a linux shell on the remote device. I tried using Ansible, but did not manage to make it work either…

7/ Stability seems OK, but regarding simplicity, Foreman is not the best product ever… :wink:
Installation is quite complex (even with the foreman setup wizard), logs are complicated to find and analyse, monitoring is not very obvious (for instance, I see a red cross on a Host. What happens ? I click on it, it says “Status : Error” and “Execution : last execution failed”. OK, then how do I go from there to the explanation of those errors ?), many things are not integrated in the WebUI (see my previous remarks).

Anyway, Foreman stays a great product, but I think it would gain in being more “user friendly” and simple for dumb users as me :wink:

Thank you for your feedback on this message !





first of all, thanks for your thorough feedback :slight_smile:
While I’m not familiar with all the things you commented on, I want to at least adress the things I am aware of.

Integration with external IPAM tools is already possible, but the list is somewhat small.
Regarding the “out-of-the-box” supported DNS/DHCP services, you can integrate existing ones via the smart-proxy-dhcp and smart-proxy-dns. All other IPAM services need aditional smart-proxy plugins to work, since Foreman does not (and can not) know how to talk to all that are out there. The list of existing plugins can be found here. If your IPAM service is not in the list, you are very welcome to write one yourself :wink:

While I agree that the hooks plugin is currently not integrated very well with the UI, you should recieve feedback on them. Each hook should have an entry in the “box” that appears for orchestration tasks when you create/update a host. Not shure though wether this is limited to certain entry points.

For IPAM/DNS/DHCP, as I suggested above, using smart-proxys with plugins would probably be the best way to go.
For deploying SSH keys, installing a backup tool and maybe also the monitoring tool (I don’t know how that one works, wether this is a client installation you are doing or something else), I would suggest taking a look at finish templates (I think those are correct for VMWare image provisioning). Those are there to execute tasks after the main provisioning finished.

Ansible integration is currently worked on quite a lot afaik, so you should see improvements in the near future :slight_smile:

This is odd. Remote execution has been quite reliable for years now here at my site. You might want to take a deeper look into this to figure out what is going wrong exactly. We will be here to help you out with that, but I remote execution itself has been working well for quite some time now.

Monitoring security fixes in Katello relies on errata information published alongside the repositories. Not all distributions provide those. To my knowledge, Debian based distos only provide information about the content of an update in the changelog. There are tools out there that can do that, but it is currently not implmented in Katello. I guess one could develop a plugin for that, but I’m not aware of any that’s currently around.

I never tried to integrate network devices, but I would guess this is because Foreman is primarily designed to deploy and manage Linux (and Windows or BSD to some extent) servers, not for managing your whole datacenter infrastructure.

Stability is (to my experience) getting better with about every release :wink:
Regarding userfriendlyness and ease of setup, these are problems the team is aware of and currently trying to improve on in several areas :slight_smile: Feedback like this is what helps with this :wink:


Whaou, thanx for this quick feedback !

Regarding IPAM integration, I will check with DHCP Provider but it does not seem to fit our needs as we don’t use DHCP. Maybe I can configure “Internal DB” on each subnet, and then populate this Foreman Internal DB with all my IPAM entries… Do you think it would be possible ?
It would resolve the “what IP can I use ?” question during Host Creation, but not the IPAM update with the choosen IP. Actually, I don’t really understand how DHCP provider works and could deal with this…

Regarding DNS Integration, it seems that Foreman can natively integrate with Microsoft Active Directory DNS. I will check that…

Regarding Hook feedback, only “create” or “postcreate” hooks gives a feedback, not “after_commit”…

Regarding Finish templates, I believe it cannot be used with user-data template. And we need user-data template to pass to VMWare information on network configuration…



I can not tell you whether or how it could work with your IPAM solution, but we use the dhcp_infoblox provider to actually manage IPAM in our Infoblox. Since I’m not a dev, I cannot tell you how those work internally, but it boils down to each one implementing defined API endpoints (like, create, query next free IP, delete, etc). A call from Foreman to those API endpoints then triggers the code defined in your plugin to do “the right thing” with your external system via calling your IPAM/DHCP’s APIs.
Regarding populating Foreman with all the infos from your IPAM, I doubt that would be a great solution. It would probably result in some sort off mess.

You might be right there. I have never used either since we only do PXE at the moment and I will probably never understand the differences between those two headscratch

@tcastelle Did you ever finish linking Foreman to Netbox? My organization is looking to do the sasme. What was your after_commit hook to create the IP information?