Defaulting Puppet to off in the Katello Scenario

In a lot of scenarios a Puppetserver is not desired by the user. If it is, it may make a lot more sense to deploy it on a separate machine via the foreman-proxy-content scenario. This is easier in terms of upgrades, scaling and backups.

Current Situation

To get a default installation, the following command is executed:

foreman-installer --scenario [katello|foreman-proxy-content]

Both include a Puppetserver, even though it’s optional. To install without Puppetserver, the following command will achieve that.

foreman-installer --scenario [katello|foreman-proxy-content] --no-enable-puppet --foreman-proxy-puppet false --foreman-proxy-puppetca false --foreman-proxy-content-puppet false

It is also possible to only build a Puppet compile server (i.e., disable the CA):

# Rely on defaults
foreman-installer --scenario [katello|foreman-proxy-content] --puppet-server-ca false --foreman-proxy-puppetca false

# Explicit
foreman-installer --scenario [katello|foreman-proxy-content] --puppet-server true --foreman-proxy-puppet true --puppet-server-ca false --foreman-proxy-puppetca false --foreman-proxy-content-puppet true

Proposed Situation

In the first phase we focus on removing the Puppetserver installation from the default, making it optional. This means the argumentless installer will achieve the same result as the previous long command.

To enable the Puppetserver, the goal is that the following command works:

foreman-installer --scenario [katello|foreman-proxy-content] --puppet-server true

By default this includes the CA server. To only build a compile server, the CA server can be disabled as follows:

foreman-installer --scenario [katello|foreman-proxy-content] --puppet-server true --puppet-server-ca false

To do this, we need changes in Kafo to store more information in the answers file.

This assumes it’s possible to leverage Hiera to automatically couple answers, but there has been no proof of concept to show this works in practice. It may be possible through hooks. Worst case we will need the user to be explicit by passing in more parameters:

foreman-installer --scenario [katello|foreman-proxy-content] --puppet-server true --foreman-proxy-puppet true --puppet-server-ca true --foreman-proxy-puppetca true

Proposed Implementation

In building this, the initial focus will be the Kafo work to introduce an answers file version 2. This means we’ll need to store the file format somewhere. A logical place to store this is the scenario file. A helper to migrate the content should be provided by Kafo to easily upgrade.

To explain why a new file is needed, we’ll first need to look at answer file version 1. Generally is as follows:

classname_a: false # don't include this class
classname_b: true # include and use the defaults
classname_c:
  param: value # include and override the default(s)

A side effect of this is that it’s impossible to disable a class and store answers at the same time. Kafo will also store all defaults in their fully expanded form. This means we can’t use Hiera features to do lookups and it’s impossible to hide parameters even if the user shouldn’t touch them.

A draft proposal for the new format would be:

classname:
  enabled: true # Other values: false, always
  parameters:
    param: value
  blacklist:
    - param
  whitelist:
    - param

enabled now also accepts the option always which means there’s no way to disable a class. This makes sense in the katello scenario where certain classes shouldn’t be disabled because it breaks the installer. Generating the --[no-]enable-foreman-plugin-PLUGIN gives the illusion of control and only clutters the UI at best, breaking it at worst.

parameters is optional.

The whitelist supersedes the blacklist. Both are optional. It is possible to store an answer for a blacklisted or non-whitelisted parameter and it is passed to application. It just doesn’t show up in the UI. This allows setting a lookup answer. While technically not needed, it would mean that we’d need to detect the interpolation and disable kafo’s validation. This while most users wouldn’t even want to change it because it breaks their deployment.

The proposed default for Puppet would be:

puppet:
  enabled: false
  parameters:
    server_environments_owner: apache
    server_foreman_ssl_ca: "%{lookup('certs::puppet::ssl_ca_cert')}"
    server_foreman_ssl_cert: "%{lookup('certs::puppet::client_cert')}"
    server_foreman_ssl_key: "%{lookup('certs::puppet::client_key')}"
  blacklist:
    - server_environments_owner
    - server_foreman_ssl_ca
    - server_foreman_ssl_cert
    - server_foreman_ssl_key

There may be other options we wish to hide, but it’s wise to limit the scope for this project.

We should evaluate how many users do wish to change these values and if we’re not breaking some deployment. To allow users to unblacklist options it’s wise to keep generating the parser cache for these parameters. Then it’s as simple as removing them from the blacklist in the YAML file. This has a neglible overhead and saves developer time by not having to optimize an existing process.

Introducing answer coupling is more complicated because the answer is composed. It’s whether the puppet class is enabled and the answer to server (or server_ca). We also wish to support remote Puppetservers reached over HTTP in which case the puppet module is disabled, but the answer is still true. The user does need to be able to answer. At this point I have no concrete proposal on how to solve this which is why the worst case is requiring additional parameters.

Lastly we should get rid of the foreman_proxy_content::puppet answer. The best implementation depends on the feasibility of a few things, but can be done.

Other Related Tasks

We have many help texts and other documentation that needs to be evaluated.

One example is the command output to install a proxy after generating certificates for it. That includes a line to set the Foreman URL for the Puppetserver integration. That effectively enables the Puppet installation again, defeating the purpose. One solution is only implementing this change for the katello scenario and not foreman-proxy-content, but it’s likely not what we want.

The manual is another important task. We need a chapter about how to install Puppetserver for users who do want it.

Another is the Puppet environments integration in Katello/Pulp. Currently Katello can create environments for content hosts but this relies on Pulp writing out files for Puppetserver to use. If there’s no Puppetserver, it may fail. I haven’t done any research into this.

This will also need testing. Our current pipelines assume Puppet is present and passes in options which can also enable it.

Conclusion

There’s a few steps we need to take. First of all, we need to agree this is a good goal. Then my good approach needs to be agreed on. Of course it also needs to be implemented and tested. Once it’s merged, documentation needs to happen. While ambitious, I do want to target all of this to 1.24. That’s why I would like feedback, preferably soon :slight_smile: . If people are interested in helping with this, even better.

2 Likes

I really like the idea of the Puppetserver being optional (also for Foreman not only Katello, but this will need a solution for the certficates). :+1:
How difficult is it to get this as additional scenarios like katello-with-puppet and katello-with-ansible?

Currently we need to duplicate all migrations for each scenario. That means it might be easy to add one, but easy to mess up its maintenance. I am thinking about a common.migrations and how that could look like to simplify shared migrations that only reflect changes in Puppet modules rather than the actual scenario. That would make it much easier to add additional scenarios.

The question then becomes: what are valid scenarios? Do you also add a katello-with-puppet-and-ansible? The idea of the installer is that you can compose your own installation.

To demonstrate I wrote a scenario that split a vanilla Foreman into 3 servers:

  • puppet - a puppetserver that is both a compile server and a CA
  • server - a vanilla foreman that only runs foreman and dynflow
  • proxy - a vanilla foreman proxy that only runs foreman-proxy and talks to the others

It does register itself to puppet and runs the agent, but strictly speaking you could just configure puppet.conf and run puppet ssl bootstrap to get the certificates. It’s not fully automated (see the TODOs), but see this pipeline:

Thanks @ekohl for nice writeup.

Have you considered not touching the existing answer files but introducing other file that would hold parameters for disabled classes? Or external file to keep the track of enabled/disabled classes. That would be perhaps easier than taking care of a migration. I like the new format structure though. If it lands in 1.24, then I’m happy with nicer and more descriptive answerfiles :slight_smile:

I really like this idea, we currently have

  • a foreman server to deploy baremetal / virtual images.
  • a foreman/katello server and capsules to support repositories in content-views for CentOS
  • Red Hat Satellite and capsules for RedHat content (which uses foreman/katello)
  • Puppet Enterprise server.

We would like to use the Puppet Enterprise server to manage all the servers (including the above), but need to work around the puppet-servers running on all the above.

Ideally Foreman / Katello installs/manages itself without Puppet ór the modules / hiera can be transferred to the Puppet Enterprise server, greatly improving the way we deploy foreman as well as we can then fully puppetize its installation / upgrades as well.

Looking forward to release 1.24!

Regards, Eize Speerstra

Using an external Puppet Enterprise server is on my agenda. Note that is not running the installer, but using it at runtime. Ideally getting reports out of it. I’ve done some research and have some ideas, but this is the first step to making that possible. Foreman Proxy together with Puppet in docker container has a slight preview in that direction.

We have users who do this manually. For Katello it’s currently not really supported for upgrades because a lot of logic is in the hooks, but I’m hoping we can make this easier in the future by removing those hooks. There’s also migrations and I don’t know how to deal with that. Currently it’s unsupported, but users who do this manually know how to read the CHANGELOGs and do it themselves.

Following up on this idea, I like the feature of including/excluding parameters but disagree with the answers file as the landing spot for it.

The included/excluded parameters for a module should be in the scenario configuration rather than the answers file. My reasoning is that hiding a parameter / forcing module default is more a feature of the scenario itself, that all installations of that scenario would have in common regardless of any choices the user makes in installation. The answers file should contain only that which might be reasonably configured by the end user / that which might vary between two installations of the same scenario.

I discussed this with @ehelms a bit because it came up in the context of Introduce a class to represent the answer file by ehelms · Pull Request #337 · theforeman/kafo · GitHub . I’ll summarize a few more thoughts on the topic:

  1. We want to move away from treating the answers file (and scenario config) as something that the user might hand edit to specify the configuration. My understanding is that this can introduce subtle skips and breakages to installer hooks due to the order in which various hook stages are run relative to loading saved answers and loading module defaults.

  2. There are use cases where hand edits may still be necessary such as cloning to DR, test, or reproducer environments. But we do not want the design to be beholden to these concerns.

  3. My key concern is separating the concepts of configuration that is static to the scenario (hiding parameters, enabling/disabling modules) vs. configuration that may vary across installations of the same scenario (parameter values). In the newer view of the answers file as a system of record, this remains a valid concern.

For example, one may need to read an answers file that is provided to them in an sosreport from an end user. Even for one’s one machine, reading params from an answers file (“how did I or my colleagues configure that again?”) is usually preferable to foreman-installer --help as it doesn’t have an annoying load time, doesn’t print extraneous information such as param docs and other help text, and is rather specifically formatted for param values.

Another idea that was discussed is making the distinction between default and overridden param values, i.e. module: { default_params: { ... }, overridden_params: { ... } } where the defaults would always be populated and the overridden params would contain only the subset with custom values.

This is not only useful for human readability as I described above, but could also be applied in writing hooks and/or migrations.

1 Like

For Satellite-Server, RedHat has stated that they want to get rid of Puppet (with 7.0 according to the roadmap shared on the last Birthday party, whatever Foreman version that corresponds to).

The “problem” I have with Foreman currently is that we don’t want to have puppet in the first place. We use ansible and ansible only.

As mentioned above, it’s not clear where the certificates will be coming from once puppet is completely gone.
The “natural” choice would be DogTag CA (as included in IPA), or use an external IPA instance right away, because that’s more likely to exist anyway.

Hi @rainer_d,
Thanks for the input!

Starting with Foreman 3.0, while the puppet plugin will be installed by default in the Foreman scenario, you will be able to remove it if you do not need it. (See The Road to Making Puppet Optional and Puppet in 3.0 release for more details).

You are correct that certificate handling is one of the issues that need to be resolved in order to allow a fully puppet-free installation. It is already possible to provide your own certificates as options to the installer, but the default Foreman scenario uses the puppet CA for those. There has been some work on enabling the installer to generate certificates without relying on Puppet CA (see RFC: Redesign Certificate Handling within Foreman Deployments for details) but I don’t expect a solution to be ready prior to Foreman 3.1.
We are now revisiting this thread since in the Katello scenario we don’t rely on Puppet CA (rather on Candlepin) for certificate generation, so we could default to not installing Puppet at all in this scenario and only install it for those who wish to use it.

@ekohl Do you think we should split out the RFC around installer answer file V2 to a separate RFC or rename/refocus this RFC on that broader outcome?