RFC: Foreman "Blueprints" as abstracted deploy model

Development RFCs
1

The reason I love Foreman, is because it loves all my stuff. There are not many things it does not support in one or the other way; even more, all these things get modelled nice and neat.
It gives me a good and clear and accurate view of my entire resource pool.

Now, as was discussed this week during CfgMgtmCamp, it was presented that 71 variables need to be provided on a vanilla installation of foreman (so without any additional plugins), a high number! Even worse, it might make user adoption a bit slower as well.

With all this in mind, I started dreaming and I believe we have a very big opportunity available at our feet:
a concept most easily described as “Blueprints” (I’m not a MaaS user, but from reading and seeing demos, maybe I could also call it “JuJu on steroids”?)

In my dreams, a Foreman Blueprint would probably look something like this (still rough around the edges perhaps):

name: kubernetes-cluster
description: this blueprint will create a high-available K8S cluster, with 3 masters and 5 nodes
foundation:
  operating_systems:
    - name: WindowsServer2019Standard
  cfgmgmt:
      ansible_playbooks:
	    - xabre.kubernetes_master
	    - xabre.kubernetes_slave
		- xabre.some_k8s_deployment_role
  resources:
    - cpu: 4
	  ram: 16000
	  hdd: 32000
	  count: 8
      name: standard_computer
stages:
  stage_0:
	- provision: standard_computer
	  count: 8
	  os: Windows Server 2019
  stage_1:
	- commission: standard_computer
	  ansible_playbooks:
		- xabre.kubernetes_master
	  count: 3
	- commission: standard_computer
	  ansible_playbooks:
	    - xabre.kubernetes_slave
  stage_2:
	- commission: standard_computer
	  has_ansible_playbooks:
		- xabre.some_k8s_deployment_role
	  select: any

the “foundation” bit is where validation is done for availability of the set of resources, operatings systems and configuration scripts, basically our tools available in foreman right now. In the beginning, a user might just see a descriptive error message stating that these will have to be made available before it can proceed, but in a later stage this might try to overcome issues by creating an operating system if it’s not available (e.g. if it were an open-source OS like ubuntu, this would probably be fairly trivial to create an operating system in foreman using some safe defaults?).
An interesting addition here, would be the resources, in which the user would be asked to give a list of compute resources and/or bare metal systems (adhering to the requirement of 4 CPU threads, 16GB RAM, etc…) that can be used for this deployment.

After this, the “stages” can be constructed. To go to the next stage, all sub-steps would need to be ready before it can continue to the next stage. In this example, the first stage would provision 8 systems with “windows server 2019”, the next stage would install kubernetes_master on 3 of them and kubernetes_slave on the remainder. Finally, something could actually be deployed to this cluster.

To make another interesting case, IMO, it should also be possible to create, for instance, an oVirt cluster and deploy a set of VMs as well.

This could all go to an “architect”-service (kinda what galaxy is for roles) where all these blueprints could be made freely available for everyone to use and share!

I think the benefits are:

  • Simple! no programming skills required, it’s YAML!
  • builds on the abstraction foreman provides
  • could probably already be build as an external tool which consumes the API
  • concise package, one file to rule them all, one file to bind them all :slight_smile:
  • could enhance user experience by a great deal (could greatly reduce 71 parameters)
  • easy user adaption
  • an answer to Juju? (or just a stepping stone?)

Okay, I know I’m probably pushing this list a bit, but I hope I could make a case for this one. Really, really, really looking forward to some feedback on this topic and curious what your views are on such a topic? Are there perhaps already efforts or brainstorms being done on this subject?

Enjoy reading my midnight “snack”

2

Hi @UXabre,

If I understand you correctly, it sounds like you’re describing an orchestration tool like Hashicorp’s Terraform, OpenStack’s Heat or AWS’ Cloudformation and to a lesser degree tools like Ansible or Salt. Basically, you want a higher level tool that can describe an application stack and then have it talk to tools like Foreman to have them implement the full application. Is that correct? If so, I think you’ll find that a couple of these tools already have Foreman “providers”. Therefore, the next step in your vision, would be to use one of those tools to create blueprints for common stacks, such as your K8s example.

Now, I’m sorry if I’m hijacking the original intent of your thread, but your comment below really grabbed my attention and since you mentioned it here, it seems fair game to bring this up.

I’m very interested in this comment, because I, and at least a couple of other people, are currently trying to find our way through the installation and configuration of Foreman and Katello. As you say, we’re finding user adoption to be a slow process. As mentioned in another thread, I have been scripting the install, but I’m finding that there is no clearly defined path toward getting a properly functioning Foreman/Katello system up and running. I seem to be repeatedly taking two steps forward and one back. Even the “Success Story” that is buried in the documentation isn’t terribly useful, because there are big holes in it.

I’d really appreciate it if a more seasoned Foreman/Katello user could look over my current install script and help me understand why I’m still not able to successfully provision a new host from Foreman. As you can see from my README, I don’t think I’m being overly ambitious in what I’m trying to accomplish.

3

Hi everyone,

first, my two cents on the main idea of the thread:
In my opinion, Foreman itself is mainly designed and meant to deploy, manage and lifecycle standardized types of hosts. With the concept of hostgroups, we have some quite mighty tool already there define these standardized types.
I do like the idea of deploying a whole set of hosts based on some parameters, but I do dislike the approach you took there (at least from a Foreman perspective).
a) This should really use the abstractions Foreman already provides us with, like hostgroups, compute resources and compute profiles, etc.
b) I can not really see where this adresses the underlying issue of having to set up a myriad of things before beeing able to deploy the first host.
c) Like @mason already wrote, this could (probably, I’ve never tried it) be done quite well by other tools that are specialized on orchestrationg whole infrastructures instead of single hosts. I could imagine this to be a somewhat decent idea for a plugin though, since not everybode will want to set up and maintain another tool for that since if they do not need it for anything else. I just cannot see this as a feature in vanilla Foreman because it aims somewhere else (or rather, beyond) than Foreman.

To adress the underlying issue:
The main problem adressed here is that installing Foreman and getting it up and running is quite some task to handle and can easily becomm a lot of pain and slamming your head agains walls/monitors. Lately I have seen a lot of people bringing that up, and the first solutions currently beeing worked on (e.g. here, another thread about that topic that just poped up today. Sadly had no time to give it a complete read by now). I have also thought about building a script to set up quick test-and-throw-away instances, but never found the time to do so. Once I actually find the time, I will be sure to give your script a look, but without trying it out I can’t help you in getting it to work. On a quick scan, it looks like all the basics are there though. But actually, I don’t think that should be discussed here any further since it is at least somewhat off topic. You can still open a thread of your own though :slight_smile: