The goal of this is to gather knowledge about how Foreman users use Puppet. In particular around the ENC.
This area hasn’t seen many updates in a long time and especially the UI needs some love. It was originally designed way back in the Puppet 0.25 days with improvements along the way. In particular around Puppet 2.6 when it became possible to set class parameters via the ENC. In Puppet land things also evolved and Puppet 3 included Hiera but Foreman stopped evolving its integration. Puppet has reached version 6 so it’s time to see if we need to rethink things.
The questions are strongly focused around the ENC because that’s the area I’m considering. Managing the content can be relevant but not my current focus.
Assigning classes to nodes
How do you assign classes to your nodes?
Within Foreman hosts can end up with classes from multiple sources (directly, via host groups, config groups). All of these count as the ENC.
Another common way is to set a classes array in Hiera and include those via (for example) site.pp.
It’s also possible to hardcode node class assignment in site.pp.
Some may have a hybrid approach so it’s multiple choice.
Foreman as ENC
Hiera
Puppet manifests
Other
0voters
Setting class parameters
Do you set class parameters via Foreman?
Yes
No, Foreman isn’t assigning classes
No, Foreman assigns classes but parameters are set via Hiera
0voters
Disclaimers
A lot of these situations require nuance and a simple multiple choice question isn’t enough. Since this is a forum, it’s highly encouraged to describe how you currently work and how you’d like to work.
Note that this is in the community section because I’d like to avoid turning this into a developer discussion talking about specific implementations. Implementations come at a later stage if we decide changes are needed.
It’s also encouraged to mention how other tools solve a problem better than Foreman. The goal is to find out what users think, like and dislike. The why question is the most important because it helps to understand users, their use cases and the problems they want to solve.
I would say from the environments I see about 80% use Puppet only with a mixture of assigning classes via hiera or site.pp because they want to have everything in git.
The majority of the others use foreman to assign classes but parameters stay in hiera for the same reason.
And a small number uses Foreman only with Config groups being used similar to Profiles and Hostgroups being Roles in the Roles-Profiles-Pattern of Puppet and Parameters set in Foreman.
Another small number is mixing approaches with having some classes in Foreman and others in Puppet which is perhaps the option resulting in most debugging if something goes wrong.
I’d like to use Foreman for parameters, but we need some options where we can put in Change ticket numbers and such. So we use Hiera for all the parameters…in git…with PR’s that have change tickets in them
We are exclusively using Foreman ENC for everything.
Since we have a largely heterogenous environment with several hundred different applications, each running (on avarage) on around 2-3 systems only (plus usually one DB per instance) we needed some highly flexible approach. Hacking together roles classes for every 2-3 servers did not seem desirable, too.
This way, we ended up with the following setup:
Hostgroups act as a replacement for some baseline puppet roles functionality, assigning profiles and setting some default SC params. This was a horrible idea in retrospective and will go overboard as soon as we find time for a major refactoring. In the future, I would like to directly assign roles on a hostgroup level.
Additional requirements on top of those baseline roles are set via SC params. We have some wrapper classes assigned to all of our servers that can handle the most usual requirements like additional package installs, creation of users/groups, systemd units, etc.
Only requirements that can not be handled by those means and/or are to complex get an additional Puppet class that is then set on a per-server level through Foreman ENC again.
Very big application environments (which we have very few of) receive the treatment of a dedicated hostgroup with it’s own default parameters.
For all “complex” data structures (OS users, systemd services, etc) we have a dedicated “puppet data module” containing only virtual resources that can be realized by the wrapper classes via SC params from the ENC.
Everything is deployed via r10k from repos in our on-prem GitLab instance, with tags for staging through our Puppet environments.
This is by far not a perfect design and has quite some downsides, but we did solve some very fundamental problems with this approach:
Writing a dedicated role for every application is only going to bring you that far. Once you surpass a certain number of applications it get’s out of hand really fast. We had a similar approach before making the switch to Foreman and it was a mess.
Writing YAML code by hand is pain, especially when you have to do it a lot. Having a nice git trail for all the changes is great, but that is about the only advantage. One user messed up the syntax in a way your tests did not catch (if you have any)? Production is probably broken now. This is what kicked hiera out for us.
Since we often need the same data objects (like one systemd service) across multiple servers, we needed a way to ensure they are consistent across systems and puppet stages. This, combined with the fact how painful YAML and JSON are when done by hand, is what eliminated the idea of storing the data directly in the Foreman ENC (plus, version control is still nice).
Despite offering this flexibility, we at least have highly standardized base platforms. Integration with some obscure requirements by occult third-party software can be somewhat tough, but at least it’s working.
Having most of the data in Foreman also helps with some other needs:
You can have all the information about the systems using certain puppet resources in a nice, searchable webUI. This also makes bulk editing smoother.
Since there is a REST API for it, it’s automation and integration friendly. This is a big negative for hiera based solutions in my opinion.
All the information about a system is in one place. You do not have to query several data sources for information but instead have a “single source of truth”.
Input validation can prevent some mistakes from even being saved. Nice little safety net we have learned to love.
Some other things we found to not be useful/practical:
Puppet Content via Katello: This is way to complicated and unflexible, causes way more overhead than necessary and is generally impractical. Should not be of much interest in the future, though, since Pulp3 does not have a Puppet plugin anymore (at least not yet).
When deploying new/changed classes, importing them into Foreman is “not so great”. Depending on the number of environments and classes you have, this can easily result in a HTTP timeout. I found this to be better in recent versions, but it’s still tedious.
Smart Variables: TBH, I never found out what these are meant for. In another thread, I read something about them getting deprecated because they are legacy related to old Puppet versions, so nothing to bother here.
A way to hide override values from the “edit host”/“edit hostgroup” forms would be really awesome. We have some values that only need to be set on OS/top-level-hostgroup level and nowhere else or are rarely used. Those are cluttering up the parameters tab quite a lot while it would be enough if they were overridable via the SC param page directly.
An integration with r10k (or maybe some other Puppet deployment tool) would be really nice. Currently we deploy our environments via rex job that calls r10k on all our Puppet proxies. Works, but it’s not really elegant.
I have little to no insight about how other companies are handling things, so I assume a lot of these problems and solutions are somewhat special to our situation.
In another thread (I think it was the reddit related one) I read about other ENC tools being way better at the job nowadays than Foreman. Yet, no one gave any examples for which ones or why that is. I only had my hands on Cobbler and Puppet Enterprise in that regard and found both to be “not so great” at the job. If there really are tools that are better at this, some examples would probably benefit this topic a lot.
Thanks @areyus for the great insight. That’s very much what I was looking for.
Have you considered using Puppet Data Types in your modules? Ideally Foreman would actually use those as well to hint its own validations. Should be quite easy to add and I’ll happily point anyone to where it should be implemented but I never had time for it myself.
However, if we know that the subclasses are not intended for direct consumption, but are only really part of the internal structure of the module, then we would want to exclude those from the import mechanism, so that Foreman only offers to import git . We can achieve this via the file /usr/share/foreman/config/ignored_environments.yml .
This is a workaround of Puppetserver not being able to hide things. You can also use it when you have the convention of only exposing the roles and/or profiles module(s) and filtering everything else.
This is also something @TimoGoebel suggested. I have some ideas as well so perhaps the three of us should sit down and see what we can come up with.
We use Foreman for provisioning and node classification but not to assign classes. Our site.pp uses the hostgroup from Foreman to dynamically include the role:: class. From there, it’s a rather conventional role/profile/hiera pattern.
At one point we tried the full ENC to assign classes, but found it difficult to track down what puppet would do on a given server or group of servers (classes that include other classes really muddy the waters). It also proved difficult to track down historical changes while trying to troubleshoot.
We have started using data types for our Puppet modules, but since they were all written for Puppet 4 we have not gotten around to update all of them. Having the validation done in Foreman still has the benefit of getting them caught before wrong data can be saved, which is a huge plus. If Foreman would pick the validations from the manifest, this would in fact be the optimal solution (at least for us).
I read about those some time, but have not had time to look into them. I will add that to the todo backlog right now
Sign me up for that While I will not be able to help with implementation, I would be glad to give some input on possible workflows.
We are currently in the process of migrating to the implementation of the Roles and Profiles Pattern using Foreman as ENC. Using Hiera for parameters. Stand alone puppet master(s).
In our old setup we used Katello and had issues with it. So we decided to split the package managent
from the “puppet management” on our new iteration. Using Foreman for parameters works, but we don’t like not having that data into source control. There is also the handling of sensitive parameters. Hiera-Eyaml and the new integration with Vault seem a lot better.
We like and find useful the parameters sent by Foreman. Like location, subnets, etc.
We use foreman with puppet as ENC and with smart_class_parameters too for all our servers. The missing point here, for smart_class_parameters is the lack of easy rollback and the ability to link changes to issues like some of you said in this thread. To make this more git/issue tracker friendly I’m currently working on foreman_smart_class_parameter ansible module. Not ready yet but I hope to make the PR before FOSDEM . In our usecase, it’s mostly the only missing piece to store all foreman config into git and applying it via a gitlab-ci pipeline (as we already have hostgroup and config group implemented in FAM). I hope it will help some of you that want to store foreman config and puppet related foreman config in git.
I’m a bit late to this, hope the info is still of use.
Our Puppet and Foreman journeys started together and are definitely ongoing, some of the things we’re doing may have better solutions so if anyone spots anything feel free to chime in. Some of the ways we learnt to do things early on have complicated our workflow a little and things are still evolving.
We use Puppet extensively and the Foreman ENC is our primary way of assigning classes to nodes, both Linux (RHEL/CentOS) and Windows. Puppet is deployed on separate hosts (2 x masters and 1 x CA) with the appropriate Foreman Smart Proxies installed. Puppet is not installed on the Foreman/Katello host. We are in the final stages of deploying the foreman-puppet modules to manage our Foreman infrastructure, these are managed in the same manner as all our other Puppet modules.
Our Puppet deployment is pretty standard with Foreman assuming the ENC role. We use a mix of internally written and external forge modules with a control-repo managing the module versions, heira structure and data. All of our parameters are in heira data. Puppet code is version controlled by an on prem GitLab instance, all modules are version tagged and versions managed in the control-repo puppetfile. We have some custom scripts triggered on each merge by GitLab-CI that run Puppet lint and unit tests, deploy code to the puppetmasters via r10k and , for the control-repo, hit the Foreman API to import new classes.
We don’t use Katello for Puppet classes at all.
We have built a Foreman Host Group structure based on node tier (dev, test, prod), OS and role. As much as possible Puppet classes are assigned to Foreman Host Groups and inherited by the contained nodes. We do have some snowflakes getting their classes from site.pp but this is under review with the preference being to assign classes to nodes in Foreman.
We don’t use Foreman for parameter assignment, with one exception. The main reason for this is that we were learning Puppet at the same time we were learning Foreman and we have made extensive use of includes in our Puppet code. Foreman can’t see the heira data for included classes so the parameters don’t show up. We are reviewing this to see if using Foreman for parameters provides enough value to warrant refactoring our Puppet code. The exception is that we use some Foreman parameters (location, app_role, server_role, tier) with templated Smart Class Parameter values and a custom Puppet module to set Puppet Facts dynamically. These facts are then used to determine our Puppet hiera structure and in our Puppet modules.
At the moment some of the work on utilising Foreman more for parameters etc is on the backburner until we see how the decoupled setup will work.
