Info to move foreman to hiera - want to build a guide

I’m starting to expand my foreman experience into other backends, and I’d like to move foreman to completely use Hiera as the backend for puppet class parameters.

I appreciate there are questions which will be environment specific, but I’d like to try to write a guide for the use of alternative backends, the first one being Hiera as I can’t find any guide or real discussion on it beyond a few early comments in 2013 on earlier editions of Foreman when smart class parameters where first really coming into use.

What do I have to change on the puppet master, on the foreman node, to tell foreman to use Heira for the backend data - I’d like to support different environments if possible, if I can set this up and test it I can then write a guide and how to and give some deployment options around things such as structure and pattern matching.

If you’re using Hiera with foreman, I’d love to hear about what you did / how you did it / problems you faced so I can try to put together a solid guide with good background.

thanks

There is no particular setup needed. Since Puppet 4 it will always use Hiera. Even if you use the Foreman ENC, Hiera will be used as well. This can be used when migrating.

https://puppet.com/docs/puppet/latest/hiera_intro.html is a good start. Personally store my environments in a git repo (each branch an env). Every env contains a modules directory, but also hiera.yaml and data.

I’ve actually moved back to using a site.pp and assign classes there. Parameters are set via Hiera. My environment is small (~10 machines) so I don’t use host groups but if you’re using Foreman as the ENC then you still have those variables available. I also know that some people set classes via Hiera and then use lookup('classes', {merge => unique}).include.

1 Like

this is a really good response and quite surprising, if foreman is offering params as smart class parameters is it not going to conflict with hiera, wouldn’t foreman need to stop offering them or the puppet master be told to use hiera over smart class parameters ?

In Puppet the ENC is higher priority than Hiera so anything coming from the ENC (foreman in this case) will override data from hiera. So there are no conflicts so to speak just plain old hierarchy.

Where you will get interesting behavior is with including classes from both places and “when” variables and parameters get set. The order is like this ENC > site.pp > node inside the node scope you have ENC data too and that overrides data from Hiera https://puppet.com/docs/puppet/latest/lang_scope.html.

Hiera merge strategies do not include ENC node/class data so you cannot merge ENC data inside hiera but you can use the scope function https://puppet.com/docs/puppet/latest/hiera_merging.html#interpolation_functions
to look up ENC data from the top scope inside hiera. You can of course merge data from both places in puppet code later if you want.

We use a number of strategies to get a best of both worlds situation.

  • We only include a single module from foreman we called foreman_enc, this module has a main class with one line lookup('classes', Array[String], 'unique').include. This line would normally be in site.pp for an ENC-less hiera set up but putting it in a class included from foreman means that all classes are declared in the expected order avoiding the weird ordering you get if they are not daisy chained. All subsequent class includes are done via hiera and puppet roles and profiles.
  • All hostgroups in foreman are are nested inside a single hostgroup which includes the above class. All hosts are automatically added to a hostgroup.
  • The other sub classes in our foreman_enc module contain no code but just have class parameters. These classes are included in foreman hostgroups where appropriate to make only the parameters we need available in foreman available as smart class parameters in the UI. We have 10 smart class params, 4 of which are global and 9000 nodes. Everything else is in hiera.
  • We filter all non necessary puppet classes from foreman to avoid them being added accidentally to nodes and for performance reasons. However due to fixing the ordering above it does not matter where classes are added so you could allow this if you don’t mind everything not being in version control or having to look in two places. You can also use a naming convention or trusted facts or whatever to assign roles directly from hiera, it all works.

The last one is a biggie.

What you loose from the above set up is the ability to move a host from one hostgroup to another to change its config (apart from the 10 smart params of course) because doing so will not change what classes are included or parameters are applied. So how do we fix this to get back the best of both worlds?

Well the first part is we added a hierarchy level to hiera that is based on the ENC parameter$hostgroup which looks like this Global/Darwin/Workstation . Like so:

  - name: "Foreman Hostgroup specific data"
    path: "21_hostgroups/%{hostgroup}.yaml"

Our hiera yaml would then be at hiera/21_hostgroups/Global/Darwin/Workstation.yaml

That was great but in foreman hostgroups inherit data from their parent so to get a similar behavior we would have to duplicate data in each yaml file at that hierarchy level and things quickly got very confusing and difficult to refactor.

What we decided to do was add a template based global smart variable that produced an array of hostgroup paths that would look something like this:

  parent_hostgroup_paths:
  - Global/Darwin/Workstation
  - Global/Darwin
  - Global

We could then use hiera’s mapped paths method to create a mini hierarchy inside our main hierarchy with the path array like this (replacing the previous entry)

  - name: "Foreman Hostgroup specific data"
    mapped_paths: [parent_hostgroup_paths, hostgroup_path, "22_hostgroups/%{hostgroup_path}.yaml"]

That way hiera would look in all hostgroup yaml files and the behaviour is exactly the same as foreman.

The smart variable template is like this

<% parent_hostgroups=@host.hostgroup.to_s.split('/') %><% counter=0 %><%= parent_hostgroups.map! { |g|
   counter+=1
   if counter == 1
      g
   else
      parent_hostgroups[counter-2] + "/" + g
   end
}.reverse %>

That’s the only bit that feels hacky so i suppose the biggest win for us would be trying to get that variable array added as a default ENC variable in foreman.

Anyway I let me know what you think or if you have any qusetions or if you think I am a crazy person.

4 Likes