Deployment objects in Foreman

Development
1

Hi all,

(TL;DR - multi-host orchestration is coming, help us make it awesome!)

For a while we've been discussing the concept of multi-host
orchestration in Foreman, and how to achieve it. We've formed a
feature team to start looking into implementing it, and I'm writing
this email to give the community some insight into our planning so
far, and to invite input at this early to make sure we haven't missed
anything. (There's also quite a lot of lessons from Staypuft in this
document as well!)

Some terms we're using:

  • Resource: a Foreman object (Host, Subnet, Puppetclass etc)
  • Stack: the instructions needed to assemble a set of resources for a
    specific purpose (e.g. a database stack)
  • Stacks can be nested (e.g. an app stack combines a db stack with a
    webserver stack, etc)
  • Deployment: the result of performing the instructions in a stack,
    taking in any necessasry input parameters (MyTwitter is a deployment
    of the app stack) [note: I actually favour the term Scenario here, but
    I got outvoted - anyone else prefer this? :P]

The current thinking is that we need some new first-class models in
Foreman for the Stack and Deployment objects, so that we can store
them. The Stack object is intended to be an abstract representation of
the thing to be built, the "receipe" if you will. This contains things
like the number of Hosts to build, the PuppetClasses to execute on
those hosts, and so on. Stacks can be nested, so that one can write a
simple stack for, say, a db cluster, and then use that stack for
defining a more complex object. The Stack also contains ordering,
where it makes sense, so that you can ensure a db host is built before
an app host tries to use it.

A Deployment is an instance of a Stack, where we can record the exact
hosts used, the parameters passed to the puppet classes (so we can
correctly share things like DB passwords to the right hosts, etc), and
where we can at a later time update the deployment.

For provisioning, the eventual plan is to either have the user provide
a set of prebuilt hosts (configured with appropriate authentication so
that Foreman can log in and configure them), or to have the user
specify one (or more) compute resources (or bare metal, via discovery
as staypuft does) and allow Foreman to build the necessary hosts
before proceeding to the provisioning step. Clearly this is only
needed when creating a Deployment, since a Stack is an abstract
representation.

There will of course be a UI eventually, but initial efforts will be
focussed on creating these stacks/deployments via the API, and on
defining an appropriate schema for the JSON used to create them. Any
UI efforts will likely be just for displaying what has already been
created.

To get started on this, we've broken it down into some user stories.
This is by no means a final list, but in rough order, we have
(indents are comments on the story):

  • A data design for stack and deployments

  • A JSON schema for creating these objects via the API

  • Ability to deploy a single-machine 'stack'

    • Basically what Foreman already does, but allows us to test stack
      nesting, etc
    • Ensures we can deploy independant services
    • Ensures we can deploy intra-dependant services on a single box
      (ordering / puppetclasses)

  • Ability to define network configuration for a stack (single network
    only initially)

  • Ability to detect / act upon when a host has completed it's configuration

    • Needed to know when to start the deployment of the next host in sequence
    • Staypuft uses the incoming Puppet reports, but we can probably
      improve on this

  • Ability to deploy a 3 machine stack on the single network

    • 3 is arbitrary
    • Introduces inter-depdant services between machines

  • Ability to deploy multi-network stack

    • Permits concepts such as compute/management networks, provisioning
      networks etc

  • Ability to consume existing hosts within a deployment

  • Ability to spin up hosts on a single compute resource for a deployment

  • Ability to spin up hosts on multiple compute resources for a deployment

  • Ability to consume discovered hosts via the Discovery plugin for a deployment

The ultimate goal is to make these Stack objects shareable, so that we
can all contribute to having well designed stacks running within our
infrastructures. We have a long way to go though.

Some of you will want to know what research we've done, and our
thoughts on the tools already out there - I'll let Daniel reply with
that info. Petr is already working on the data designs, and will reply
shortly with his work so far. For myself, I'll be picking up the task
of extending how we know when a host is "finished" which, since it
involves things like running Puppet after reboot, is more nuanced than
simply receiving the /unattended/built call from the installer.

Thanks for reading, and we want to hear your thoughts on the design!

Greg

2

> Hi all,
>
> (TL;DR - multi-host orchestration is coming, help us make it awesome!)
>
> For a while we've been discussing the concept of multi-host
> orchestration in Foreman, and how to achieve it. We've formed a
> feature team to start looking into implementing it, and I'm writing
> this email to give the community some insight into our planning so
> far, and to invite input at this early to make sure we haven't missed
> anything. (There's also quite a lot of lessons from Staypuft in this
> document as well!)
>
> Some terms we're using:
> - Resource: a Foreman object (Host, Subnet, Puppetclass etc)
> - Stack: the instructions needed to assemble a set of resources for a
> specific purpose (e.g. a database stack)
> - Stacks can be nested (e.g. an app stack combines a db stack with a
> webserver stack, etc)
> - Deployment: the result of performing the instructions in a stack,
> taking in any necessasry input parameters (MyTwitter is a deployment
> of the app stack) [note: I actually favour the term Scenario here, but
> I got outvoted - anyone else prefer this? :P]
>
> The current thinking is that we need some new first-class models in
> Foreman for the Stack and Deployment objects, so that we can store
> them. The Stack object is intended to be an abstract representation of
> the thing to be built, the "receipe" if you will. This contains things
> like the number of Hosts to build, the PuppetClasses to execute on
> those hosts, and so on. Stacks can be nested, so that one can write a
> simple stack for, say, a db cluster, and then use that stack for
> defining a more complex object. The Stack also contains ordering,
> where it makes sense, so that you can ensure a db host is built before
> an app host tries to use it.
>
> A Deployment is an instance of a Stack, where we can record the exact
> hosts used, the parameters passed to the puppet classes (so we can
> correctly share things like DB passwords to the right hosts, etc), and
> where we can at a later time update the deployment.
>
> For provisioning, the eventual plan is to either have the user provide
> a set of prebuilt hosts (configured with appropriate authentication so
> that Foreman can log in and configure them), or to have the user
> specify one (or more) compute resources (or bare metal, via discovery
> as staypuft does) and allow Foreman to build the necessary hosts
> before proceeding to the provisioning step. Clearly this is only
> needed when creating a Deployment, since a Stack is an abstract
> representation.
>
> There will of course be a UI eventually, but initial efforts will be
> focussed on creating these stacks/deployments via the API, and on
> defining an appropriate schema for the JSON used to create them. Any
> UI efforts will likely be just for displaying what has already been
> created.
>
> To get started on this, we've broken it down into some user stories.
> This is by no means a final list, but in rough order, we have
> (indents are comments on the story):
>
> * A data design for stack and deployments
> * A JSON schema for creating these objects via the API

How does this format relate to Heat and other such descriptors?

> * Ability to deploy a single-machine 'stack'
> * Basically what Foreman already does, but allows us to test stack
> nesting, etc
> * Ensures we can deploy independant services
> * Ensures we can deploy intra-dependant services on a single box
> (ordering / puppetclasses)

How does a stack relate to a HostGroup?

– bk

··· On 11/21/2014 09:39 AM, Greg Sutcliffe wrote:
3

> Thanks for reading, and we want to hear your thoughts on the design!

Nice writeup, thanks. If you can put it on the wiki page giving it a
Features parent page that would be fantastic. Something like:

http://projects.theforeman.org/projects/foreman/wiki/PXE_Booting_UEFI

··· -- Later, Lukas #lzap Zapletal
4

Sounds very cool!

How do Stacks integrate with Host groups - and more importantly, how
will Stacks play with extensions to Hosts/Host groups?

My main concern is just being able to get the extensions to Hosts/Host
groups that plugins add for "free" (e.g. my hosts in a Stack deployment
would get Katello content, Salt states, etc). If that doesn't
fit with the design and it'll be something entirely new, it would be
nice to have some kind of specific (and stable) extension points for the
Stack, instead of relying on deface/alias_method_chain.

··· On Fri, Nov 21, 2014 at 02:39:09PM +0000, Greg Sutcliffe wrote: > Hi all, > > (TL;DR - multi-host orchestration is coming, help us make it awesome!) > > For a while we've been discussing the concept of multi-host > orchestration in Foreman, and how to achieve it. We've formed a > feature team to start looking into implementing it, and I'm writing > this email to give the community some insight into our planning so > far, and to invite input at this early to make sure we haven't missed > anything. (There's also quite a lot of lessons from Staypuft in this > document as well!) > > Some terms we're using: > - Resource: a Foreman object (Host, Subnet, Puppetclass etc) > - Stack: the instructions needed to assemble a set of resources for a > specific purpose (e.g. a database stack) > - Stacks can be nested (e.g. an app stack combines a db stack with a > webserver stack, etc) > - Deployment: the result of performing the instructions in a stack, > taking in any necessasry input parameters (MyTwitter is a deployment > of the app stack) [note: I actually favour the term Scenario here, but > I got outvoted - anyone else prefer this? :P] > > The current thinking is that we need some new first-class models in > Foreman for the Stack and Deployment objects, so that we can store > them. The Stack object is intended to be an abstract representation of > the thing to be built, the "receipe" if you will. This contains things > like the number of Hosts to build, the PuppetClasses to execute on > those hosts, and so on. Stacks can be nested, so that one can write a > simple stack for, say, a db cluster, and then use that stack for > defining a more complex object. The Stack also contains ordering, > where it makes sense, so that you can ensure a db host is built before > an app host tries to use it. > > A Deployment is an instance of a Stack, where we can record the exact > hosts used, the parameters passed to the puppet classes (so we can > correctly share things like DB passwords to the right hosts, etc), and > where we can at a later time update the deployment. > > For provisioning, the eventual plan is to either have the user provide > a set of prebuilt hosts (configured with appropriate authentication so > that Foreman can log in and configure them), or to have the user > specify one (or more) compute resources (or bare metal, via discovery > as staypuft does) and allow Foreman to build the necessary hosts > before proceeding to the provisioning step. Clearly this is only > needed when creating a Deployment, since a Stack is an abstract > representation. > > There will of course be a UI eventually, but initial efforts will be > focussed on creating these stacks/deployments via the API, and on > defining an appropriate schema for the JSON used to create them. Any > UI efforts will likely be just for displaying what has already been > created. > > To get started on this, we've broken it down into some user stories. > This is by no means a final list, but in *rough* order, we have > (indents are comments on the story): > > * A data design for stack and deployments > * A JSON schema for creating these objects via the API > * Ability to deploy a single-machine 'stack' > * Basically what Foreman already does, but allows us to test stack > nesting, etc > * Ensures we can deploy independant services > * Ensures we can deploy intra-dependant services on a single box > (ordering / puppetclasses) > * Ability to define network configuration for a stack (single network > only initially) > * Ability to detect / act upon when a host has completed it's configuration > * Needed to know when to start the deployment of the next host in sequence > * Staypuft uses the incoming Puppet reports, but we can probably > improve on this > * Ability to deploy a 3 machine stack on the single network > * 3 is arbitrary > * Introduces inter-depdant services between machines > * Ability to deploy multi-network stack > * Permits concepts such as compute/management networks, provisioning > networks etc > > * Ability to consume existing hosts within a deployment > * Ability to spin up hosts on a single compute resource for a deployment > * Ability to spin up hosts on multiple compute resources for a deployment > * Ability to consume discovered hosts via the Discovery plugin for a deployment > > The ultimate goal is to make these Stack objects shareable, so that we > can all contribute to having well designed stacks running within our > infrastructures. We have a long way to go though. > > Some of you will want to know what research we've done, and our > thoughts on the tools already out there - I'll let Daniel reply with > that info. Petr is already working on the data designs, and will reply > shortly with his work so far. For myself, I'll be picking up the task > of extending how we know when a host is "finished" which, since it > involves things like running Puppet after reboot, is more nuanced than > simply receiving the /unattended/built call from the installer. > > Thanks for reading, and we want to hear your thoughts on the design!

–
Stephen Benjamin

Red Hat GmbH | http://de.redhat.com/ | Sitz: Grasbrunn
Handelsregister: Amtsgericht München, HRB 153243
Geschäftsführer: Charles Cachera, Michael Cunningham,
Michael O’Neill, Charles Peters

5

Thanks Greg for the nice writeup. As you've mentioned I am working on
data model. The first part representing the Stacks (template of a
Deployment) is attached.

I'll continue working on Deployment instance representation tomorrow.
Then I'll send some examples of actual stack instances describing simple
apps/products.

I've used PlanUML [1] and Idea plugin [2].

Legend

Entities:
(A) = Abstract class
(C) = Class
(M) = Module

Arrows:
A --|> B (blank closed arrow) = represents inheritance B.is_a?(A)
A --o B (blank diamond) = Aggregation - B is composed of A

Petr

[1] Class Diagram syntax and features
[2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm

··· On 21.11.14 15:39, Greg Sutcliffe wrote: > Hi all, > > (TL;DR - multi-host orchestration is coming, help us make it awesome!) > > For a while we've been discussing the concept of multi-host > orchestration in Foreman, and how to achieve it. We've formed a > feature team to start looking into implementing it, and I'm writing > this email to give the community some insight into our planning so > far, and to invite input at this early to make sure we haven't missed > anything. (There's also quite a lot of lessons from Staypuft in this > document as well!) > > Some terms we're using: > - Resource: a Foreman object (Host, Subnet, Puppetclass etc) > - Stack: the instructions needed to assemble a set of resources for a > specific purpose (e.g. a database stack) > - Stacks can be nested (e.g. an app stack combines a db stack with a > webserver stack, etc) > - Deployment: the result of performing the instructions in a stack, > taking in any necessasry input parameters (MyTwitter is a deployment > of the app stack) [note: I actually favour the term Scenario here, but > I got outvoted - anyone else prefer this? :P] > > The current thinking is that we need some new first-class models in > Foreman for the Stack and Deployment objects, so that we can store > them. The Stack object is intended to be an abstract representation of > the thing to be built, the "receipe" if you will. This contains things > like the number of Hosts to build, the PuppetClasses to execute on > those hosts, and so on. Stacks can be nested, so that one can write a > simple stack for, say, a db cluster, and then use that stack for > defining a more complex object. The Stack also contains ordering, > where it makes sense, so that you can ensure a db host is built before > an app host tries to use it. > > A Deployment is an instance of a Stack, where we can record the exact > hosts used, the parameters passed to the puppet classes (so we can > correctly share things like DB passwords to the right hosts, etc), and > where we can at a later time update the deployment. > > For provisioning, the eventual plan is to either have the user provide > a set of prebuilt hosts (configured with appropriate authentication so > that Foreman can log in and configure them), or to have the user > specify one (or more) compute resources (or bare metal, via discovery > as staypuft does) and allow Foreman to build the necessary hosts > before proceeding to the provisioning step. Clearly this is only > needed when creating a Deployment, since a Stack is an abstract > representation. > > There will of course be a UI eventually, but initial efforts will be > focussed on creating these stacks/deployments via the API, and on > defining an appropriate schema for the JSON used to create them. Any > UI efforts will likely be just for displaying what has already been > created. > > To get started on this, we've broken it down into some user stories. > This is by no means a final list, but in *rough* order, we have > (indents are comments on the story): > > * A data design for stack and deployments > * A JSON schema for creating these objects via the API > * Ability to deploy a single-machine 'stack' > * Basically what Foreman already does, but allows us to test stack > nesting, etc > * Ensures we can deploy independant services > * Ensures we can deploy intra-dependant services on a single box > (ordering / puppetclasses) > * Ability to define network configuration for a stack (single network > only initially) > * Ability to detect / act upon when a host has completed it's configuration > * Needed to know when to start the deployment of the next host in sequence > * Staypuft uses the incoming Puppet reports, but we can probably > improve on this > * Ability to deploy a 3 machine stack on the single network > * 3 is arbitrary > * Introduces inter-depdant services between machines > * Ability to deploy multi-network stack > * Permits concepts such as compute/management networks, provisioning > networks etc > > * Ability to consume existing hosts within a deployment > * Ability to spin up hosts on a single compute resource for a deployment > * Ability to spin up hosts on multiple compute resources for a deployment > * Ability to consume discovered hosts via the Discovery plugin for a deployment > > The ultimate goal is to make these Stack objects shareable, so that we > can all contribute to having well designed stacks running within our > infrastructures. We have a long way to go though. > > Some of you will want to know what research we've done, and our > thoughts on the tools already out there - I'll let Daniel reply with > that info. Petr is already working on the data designs, and will reply > shortly with his work so far. For myself, I'll be picking up the task > of extending how we know when a host is "finished" which, since it > involves things like running Puppet after reboot, is more nuanced than > simply receiving the /unattended/built call from the installer. > > Thanks for reading, and we want to hear your thoughts on the design! > > Greg >
6

Petr's hosting a deep dive session on Monday at 2pm UTC on the subject
of the design.

https://plus.google.com/u/0/events/cqnl4cngn7gaqh6bfhc22bjvfs0

Anybody with an interest or questions about the specifics of the design
he's posted in this thread should come and ask them then. If you can
let him know what to focus the session on beforehand, he'd appreciate it.

··· On 21/11/14 14:39, Greg Sutcliffe wrote: > The current thinking is that we need some new first-class models in > Foreman for the Stack and Deployment objects, so that we can store > them. The Stack object is intended to be an abstract representation of > the thing to be built, the "receipe" if you will. This contains things > like the number of Hosts to build, the PuppetClasses to execute on > those hosts, and so on. Stacks can be nested, so that one can write a > simple stack for, say, a db cluster, and then use that stack for > defining a more complex object. The Stack also contains ordering, > where it makes sense, so that you can ensure a db host is built before > an app host tries to use it. > > A Deployment is an instance of a Stack, where we can record the exact > hosts used, the parameters passed to the puppet classes (so we can > correctly share things like DB passwords to the right hosts, etc), and > where we can at a later time update the deployment.

–
Dominic Cleal
Red Hat Engineering

7

Howdy, a quick overview of tools out there that aim for similar objectives:

Oasis TOSCA
- I could find an example of this "standard" in Cloudify Cosmo.
GitHub - cloudify-cosmo/cloudify-nodecellar-example: A sample Cloudify 3 application consisted of a nodejs server and mongodb database.. It's an XML
language with the basic information for any kind of host (interfaces,
purpose, etc…). I don't think we can bake in any kind of configuration
management here, but there's a completely open field 'properties' where we
can sort of specify attributes to be used in Foreman.

Terraform

  • Template and application to define state and resources. It works for
    hybrid cloud deployments, bare metal, everything, so in that regard it
    could be usable. It's major con in my opinion is that it doesn't really
    leverage Foreman at all, and resources have to be very specifically
    defined. For instance, you cannot say "I want an apache load balancer" but
    instead you have to say "I want an apache load balancer in AWS, which will
    connect to bare metal nodes B and C". We can probably do better and have
    Foreman look where stuff could be deployed by examining compute resources.
    The DSL is very nice.

Openstack Heat & AWS Cloudformation are worth mentioning and looking at
their DSLs but they're overly specific for the Foreman use case and syntax
is limited so we can hardly add Foreman-specific attributes.

Foregit??
- This gives us a description of Foreman in terms of JSONs that
represent the objects in the application. Potentially the objects needed
for an orchestration could be put into a git repository then tell foregit
which order to follow to make the appropriate API calls to generate the
objects in Foreman.

··· ------

After reading Petr’s model, it looks like for the stack we will have to
sort of define an ‘XXXResource’ for each object we want to use. That
sounds good I guess although ‘Block’ might be more appropriate as we
already use Resource for other things (ComputeResources…). It sounds like
that would clutter Foreman core a bit too, if we follow this approach I’d
say it can be fairly easily done via a plugin that we can maintain on its
own, relatively decoupled from core.

On Mon, Nov 24, 2014 at 4:15 PM, Petr Chalupa pchalupa@redhat.com wrote:

Thanks Greg for the nice writeup. As you’ve mentioned I am working on data
model. The first part representing the Stacks (template of a Deployment) is
attached.

I’ll continue working on Deployment instance representation tomorrow. Then
I’ll send some examples of actual stack instances describing simple
apps/products.

I’ve used PlanUML [1] and Idea plugin [2].

Legend

Entities:
(A) = Abstract class
(C) = Class
(M) = Module

Arrows:
A --|> B (blank closed arrow) = represents inheritance B.is_a?(A)
A --o B (blank diamond) = Aggregation - B is composed of A

Petr

[1] http://plantuml.sourceforge.net/classes.html
[2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm

On 21.11.14 15:39, Greg Sutcliffe wrote:

Hi all,

(TL;DR - multi-host orchestration is coming, help us make it awesome!)

For a while we’ve been discussing the concept of multi-host
orchestration in Foreman, and how to achieve it. We’ve formed a
feature team to start looking into implementing it, and I’m writing
this email to give the community some insight into our planning so
far, and to invite input at this early to make sure we haven’t missed
anything. (There’s also quite a lot of lessons from Staypuft in this
document as well!)

Some terms we’re using:

  • Resource: a Foreman object (Host, Subnet, Puppetclass etc)
  • Stack: the instructions needed to assemble a set of resources for a
    specific purpose (e.g. a database stack)
  • Stacks can be nested (e.g. an app stack combines a db stack with a
    webserver stack, etc)
  • Deployment: the result of performing the instructions in a stack,
    taking in any necessasry input parameters (MyTwitter is a deployment
    of the app stack) [note: I actually favour the term Scenario here, but
    I got outvoted - anyone else prefer this? :P]

The current thinking is that we need some new first-class models in
Foreman for the Stack and Deployment objects, so that we can store
them. The Stack object is intended to be an abstract representation of
the thing to be built, the “receipe” if you will. This contains things
like the number of Hosts to build, the PuppetClasses to execute on
those hosts, and so on. Stacks can be nested, so that one can write a
simple stack for, say, a db cluster, and then use that stack for
defining a more complex object. The Stack also contains ordering,
where it makes sense, so that you can ensure a db host is built before
an app host tries to use it.

A Deployment is an instance of a Stack, where we can record the exact
hosts used, the parameters passed to the puppet classes (so we can
correctly share things like DB passwords to the right hosts, etc), and
where we can at a later time update the deployment.

For provisioning, the eventual plan is to either have the user provide
a set of prebuilt hosts (configured with appropriate authentication so
that Foreman can log in and configure them), or to have the user
specify one (or more) compute resources (or bare metal, via discovery
as staypuft does) and allow Foreman to build the necessary hosts
before proceeding to the provisioning step. Clearly this is only
needed when creating a Deployment, since a Stack is an abstract
representation.

There will of course be a UI eventually, but initial efforts will be
focussed on creating these stacks/deployments via the API, and on
defining an appropriate schema for the JSON used to create them. Any
UI efforts will likely be just for displaying what has already been
created.

To get started on this, we’ve broken it down into some user stories.
This is by no means a final list, but in rough order, we have
(indents are comments on the story):

  • A data design for stack and deployments

  • A JSON schema for creating these objects via the API

  • Ability to deploy a single-machine ‘stack’

    • Basically what Foreman already does, but allows us to test stack
      nesting, etc
    • Ensures we can deploy independant services
    • Ensures we can deploy intra-dependant services on a single box
      (ordering / puppetclasses)

  • Ability to define network configuration for a stack (single network
    only initially)

  • Ability to detect / act upon when a host has completed it’s
    configuration

    • Needed to know when to start the deployment of the next host in
      sequence
    • Staypuft uses the incoming Puppet reports, but we can probably
      improve on this

  • Ability to deploy a 3 machine stack on the single network

    • 3 is arbitrary
    • Introduces inter-depdant services between machines

  • Ability to deploy multi-network stack

    • Permits concepts such as compute/management networks, provisioning
      networks etc

  • Ability to consume existing hosts within a deployment

  • Ability to spin up hosts on a single compute resource for a deployment

  • Ability to spin up hosts on multiple compute resources for a deployment

  • Ability to consume discovered hosts via the Discovery plugin for a
    deployment

The ultimate goal is to make these Stack objects shareable, so that we
can all contribute to having well designed stacks running within our
infrastructures. We have a long way to go though.

Some of you will want to know what research we’ve done, and our
thoughts on the tools already out there - I’ll let Daniel reply with
that info. Petr is already working on the data designs, and will reply
shortly with his work so far. For myself, I’ll be picking up the task
of extending how we know when a host is “finished” which, since it
involves things like running Puppet after reboot, is more nuanced than
simply receiving the /unattended/built call from the installer.

Thanks for reading, and we want to hear your thoughts on the design!

Greg

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–
Daniel Lobato

@elobatoss

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30

8

Hi

Updating the model for stacks and adding data model of a deployment.

Petr

deployment.puml (1.99 KB)

··· On 24.11.14 16:15, Petr Chalupa wrote: > Thanks Greg for the nice writeup. As you've mentioned I am working on > data model. The first part representing the Stacks (template of a > Deployment) is attached. > > I'll continue working on Deployment instance representation tomorrow. > Then I'll send some examples of actual stack instances describing simple > apps/products. > > I've used PlanUML [1] and Idea plugin [2]. > > ## Legend > > Entities: > (A) = Abstract class > (C) = Class > (M) = Module > > Arrows: > A --|> B (blank closed arrow) = represents inheritance `B.is_a?(A)` > A --o B (blank diamond) = Aggregation - B is composed of A > > Petr > > [1] http://plantuml.sourceforge.net/classes.html > [2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm > > On 21.11.14 15:39, Greg Sutcliffe wrote: >> Hi all, >> >> (TL;DR - multi-host orchestration is coming, help us make it awesome!) >> >> For a while we've been discussing the concept of multi-host >> orchestration in Foreman, and how to achieve it. We've formed a >> feature team to start looking into implementing it, and I'm writing >> this email to give the community some insight into our planning so >> far, and to invite input at this early to make sure we haven't missed >> anything. (There's also quite a lot of lessons from Staypuft in this >> document as well!) >> >> Some terms we're using: >> - Resource: a Foreman object (Host, Subnet, Puppetclass etc) >> - Stack: the instructions needed to assemble a set of resources for a >> specific purpose (e.g. a database stack) >> - Stacks can be nested (e.g. an app stack combines a db stack with a >> webserver stack, etc) >> - Deployment: the result of performing the instructions in a stack, >> taking in any necessasry input parameters (MyTwitter is a deployment >> of the app stack) [note: I actually favour the term Scenario here, but >> I got outvoted - anyone else prefer this? :P] >> >> The current thinking is that we need some new first-class models in >> Foreman for the Stack and Deployment objects, so that we can store >> them. The Stack object is intended to be an abstract representation of >> the thing to be built, the "receipe" if you will. This contains things >> like the number of Hosts to build, the PuppetClasses to execute on >> those hosts, and so on. Stacks can be nested, so that one can write a >> simple stack for, say, a db cluster, and then use that stack for >> defining a more complex object. The Stack also contains ordering, >> where it makes sense, so that you can ensure a db host is built before >> an app host tries to use it. >> >> A Deployment is an instance of a Stack, where we can record the exact >> hosts used, the parameters passed to the puppet classes (so we can >> correctly share things like DB passwords to the right hosts, etc), and >> where we can at a later time update the deployment. >> >> For provisioning, the eventual plan is to either have the user provide >> a set of prebuilt hosts (configured with appropriate authentication so >> that Foreman can log in and configure them), or to have the user >> specify one (or more) compute resources (or bare metal, via discovery >> as staypuft does) and allow Foreman to build the necessary hosts >> before proceeding to the provisioning step. Clearly this is only >> needed when creating a Deployment, since a Stack is an abstract >> representation. >> >> There will of course be a UI eventually, but initial efforts will be >> focussed on creating these stacks/deployments via the API, and on >> defining an appropriate schema for the JSON used to create them. Any >> UI efforts will likely be just for displaying what has already been >> created. >> >> To get started on this, we've broken it down into some user stories. >> This is by no means a final list, but in *rough* order, we have >> (indents are comments on the story): >> >> * A data design for stack and deployments >> * A JSON schema for creating these objects via the API >> * Ability to deploy a single-machine 'stack' >> * Basically what Foreman already does, but allows us to test stack >> nesting, etc >> * Ensures we can deploy independant services >> * Ensures we can deploy intra-dependant services on a single box >> (ordering / puppetclasses) >> * Ability to define network configuration for a stack (single network >> only initially) >> * Ability to detect / act upon when a host has completed it's >> configuration >> * Needed to know when to start the deployment of the next host in >> sequence >> * Staypuft uses the incoming Puppet reports, but we can probably >> improve on this >> * Ability to deploy a 3 machine stack on the single network >> * 3 is arbitrary >> * Introduces inter-depdant services between machines >> * Ability to deploy multi-network stack >> * Permits concepts such as compute/management networks, provisioning >> networks etc >> >> * Ability to consume existing hosts within a deployment >> * Ability to spin up hosts on a single compute resource for a deployment >> * Ability to spin up hosts on multiple compute resources for a deployment >> * Ability to consume discovered hosts via the Discovery plugin for a >> deployment >> >> The ultimate goal is to make these Stack objects shareable, so that we >> can all contribute to having well designed stacks running within our >> infrastructures. We have a long way to go though. >> >> Some of you will want to know what research we've done, and our >> thoughts on the tools already out there - I'll let Daniel reply with >> that info. Petr is already working on the data designs, and will reply >> shortly with his work so far. For myself, I'll be picking up the task >> of extending how we know when a host is "finished" which, since it >> involves things like running Puppet after reboot, is more nuanced than >> simply receiving the /unattended/built call from the installer. >> >> Thanks for reading, and we want to hear your thoughts on the design! >> >> Greg >> >
9

> How do Stacks integrate with Host groups - and more importantly, how
> will Stacks play with extensions to Hosts/Host groups?

Good question (and BK too).

As you'll see from Petr's notes, we're leaning heavily on Hostgroups
to provide the defaults for building hosts - when your stack says "a
host" there's a lot of baggage that comes along with that - you need
to decide an OS, templates, etc. So, at least initially, we're using
the hostgroup to supply that info, and when defining a stack, you'll
specify the hostgroup it goes with. There's some discussion as to
whether than should be associated with the deploymentt instead of the
stack, but we'll see how we go. Likewise we'll probably lean on
config_groups at some point for the puppet configuration of the hosts,
and compute_profiles for the compute_resources.

In summary, we use them the same way Foreman does today, as a
collection of defaults to save specifying every little thing.

> My main concern is just being able to get the extensions to Hosts/Host
> groups that plugins add for "free" (e.g. my hosts in a Stack deployment
> would get Katello content, Salt states, etc). If that doesn't
> fit with the design and it'll be something entirely new, it would be
> nice to have some kind of specific (and stable) extension points for the
> Stack, instead of relying on deface/alias_method_chain.

I don't think we'll be extending those objects directly, there'll be
new tables/models added for storing stacks and deployments (stack
has_many deployments, for the record). Use of other config management
systems is something I'll add to the story list - given that the idea
is to orchestrate lots of objects that Foreman already understands how
to build, then I think that's fair.

As for Katello, I'd expect the relevant extras would already be
present via the hostgroup association no? All Foreman is really going
to do is spin up X hosts in group Y (potentially on compute resource
Z, and in the right order), so that should already have the right
content?

Greg

··· On 24 November 2014 at 14:20, Stephen Benjamin wrote:
10

I think I told him already, but it'd be nice to explain how the
orchestration pieces can / cannot work regardless of the configuration
management system behind. Foreman's design still drags a Puppet-centric
vision and new features should be more agnostic in my opinion.

··· On Wed, Dec 10, 2014 at 1:52 PM, Dominic Cleal wrote:

On 21/11/14 14:39, Greg Sutcliffe wrote:

The current thinking is that we need some new first-class models in
Foreman for the Stack and Deployment objects, so that we can store
them. The Stack object is intended to be an abstract representation of
the thing to be built, the “receipe” if you will. This contains things
like the number of Hosts to build, the PuppetClasses to execute on
those hosts, and so on. Stacks can be nested, so that one can write a
simple stack for, say, a db cluster, and then use that stack for
defining a more complex object. The Stack also contains ordering,
where it makes sense, so that you can ensure a db host is built before
an app host tries to use it.

A Deployment is an instance of a Stack, where we can record the exact
hosts used, the parameters passed to the puppet classes (so we can
correctly share things like DB passwords to the right hosts, etc), and
where we can at a later time update the deployment.

Petr’s hosting a deep dive session on Monday at 2pm UTC on the subject
of the design.

Google Workspace Updates: New community features for Google Chat and an update on Currents

Anybody with an interest or questions about the specifics of the design
he’s posted in this thread should come and ask them then. If you can
let him know what to focus the session on beforehand, he’d appreciate it.

–
Dominic Cleal
Red Hat Engineering

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–
Daniel Lobato

@elobatoss

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30

11

Does this mean that cloud resources such as block storage and load
balancers will finally become part of Foreman?

Say I want to create an ELK cluster in a cloud environment and as part of
that I need to attach 1TB block storage to each of my Elasticsearch nodes,
as well as cluster Kibana running on some nginx hosts behind a load
balancer. Is this finally going to be possible from within Foreman?

Jim

··· On 26 November 2014 at 08:27, Petr Chalupa wrote:

Hi

Updating the model for stacks and adding data model of a deployment.

Petr

On 24.11.14 16:15, Petr Chalupa wrote:

Thanks Greg for the nice writeup. As you’ve mentioned I am working on
data model. The first part representing the Stacks (template of a
Deployment) is attached.

I’ll continue working on Deployment instance representation tomorrow.
Then I’ll send some examples of actual stack instances describing simple
apps/products.

I’ve used PlanUML [1] and Idea plugin [2].

Legend

Entities:
(A) = Abstract class
(C) = Class
(M) = Module

Arrows:
A --|> B (blank closed arrow) = represents inheritance B.is_a?(A)
A --o B (blank diamond) = Aggregation - B is composed of A

Petr

[1] http://plantuml.sourceforge.net/classes.html
[2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm

On 21.11.14 15:39, Greg Sutcliffe wrote:

Hi all,

(TL;DR - multi-host orchestration is coming, help us make it awesome!)

For a while we’ve been discussing the concept of multi-host
orchestration in Foreman, and how to achieve it. We’ve formed a
feature team to start looking into implementing it, and I’m writing
this email to give the community some insight into our planning so
far, and to invite input at this early to make sure we haven’t missed
anything. (There’s also quite a lot of lessons from Staypuft in this
document as well!)

Some terms we’re using:

  • Resource: a Foreman object (Host, Subnet, Puppetclass etc)
  • Stack: the instructions needed to assemble a set of resources for a
    specific purpose (e.g. a database stack)
  • Stacks can be nested (e.g. an app stack combines a db stack with a
    webserver stack, etc)
  • Deployment: the result of performing the instructions in a stack,
    taking in any necessasry input parameters (MyTwitter is a deployment
    of the app stack) [note: I actually favour the term Scenario here, but
    I got outvoted - anyone else prefer this? :P]

The current thinking is that we need some new first-class models in
Foreman for the Stack and Deployment objects, so that we can store
them. The Stack object is intended to be an abstract representation of
the thing to be built, the “receipe” if you will. This contains things
like the number of Hosts to build, the PuppetClasses to execute on
those hosts, and so on. Stacks can be nested, so that one can write a
simple stack for, say, a db cluster, and then use that stack for
defining a more complex object. The Stack also contains ordering,
where it makes sense, so that you can ensure a db host is built before
an app host tries to use it.

A Deployment is an instance of a Stack, where we can record the exact
hosts used, the parameters passed to the puppet classes (so we can
correctly share things like DB passwords to the right hosts, etc), and
where we can at a later time update the deployment.

For provisioning, the eventual plan is to either have the user provide
a set of prebuilt hosts (configured with appropriate authentication so
that Foreman can log in and configure them), or to have the user
specify one (or more) compute resources (or bare metal, via discovery
as staypuft does) and allow Foreman to build the necessary hosts
before proceeding to the provisioning step. Clearly this is only
needed when creating a Deployment, since a Stack is an abstract
representation.

There will of course be a UI eventually, but initial efforts will be
focussed on creating these stacks/deployments via the API, and on
defining an appropriate schema for the JSON used to create them. Any
UI efforts will likely be just for displaying what has already been
created.

To get started on this, we’ve broken it down into some user stories.
This is by no means a final list, but in rough order, we have
(indents are comments on the story):

  • A data design for stack and deployments

  • A JSON schema for creating these objects via the API

  • Ability to deploy a single-machine ‘stack’

    • Basically what Foreman already does, but allows us to test stack
      nesting, etc
    • Ensures we can deploy independant services
    • Ensures we can deploy intra-dependant services on a single box
      (ordering / puppetclasses)

  • Ability to define network configuration for a stack (single network
    only initially)

  • Ability to detect / act upon when a host has completed it’s
    configuration

    • Needed to know when to start the deployment of the next host in
      sequence
    • Staypuft uses the incoming Puppet reports, but we can probably
      improve on this

  • Ability to deploy a 3 machine stack on the single network

    • 3 is arbitrary
    • Introduces inter-depdant services between machines

  • Ability to deploy multi-network stack

    • Permits concepts such as compute/management networks, provisioning
      networks etc

  • Ability to consume existing hosts within a deployment

  • Ability to spin up hosts on a single compute resource for a deployment

  • Ability to spin up hosts on multiple compute resources for a deployment

  • Ability to consume discovered hosts via the Discovery plugin for a
    deployment

The ultimate goal is to make these Stack objects shareable, so that we
can all contribute to having well designed stacks running within our
infrastructures. We have a long way to go though.

Some of you will want to know what research we’ve done, and our
thoughts on the tools already out there - I’ll let Daniel reply with
that info. Petr is already working on the data designs, and will reply
shortly with his work so far. For myself, I’ll be picking up the task
of extending how we know when a host is “finished” which, since it
involves things like running Puppet after reboot, is more nuanced than
simply receiving the /unattended/built call from the installer.

Thanks for reading, and we want to hear your thoughts on the design!

Greg

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

12

Hi Pert,all

I'm pretty new in foreman, so please bear with me. Deployment feature
sounds exiting.
modeling looks good, few questions than remarks:

-doesn't the HostgroupResource already include subnet management? and if
not what is the redundancy with the subnetTypeResource
-what exactly does the ComputeResource/Profile is in-charge of?

i understand puppet is used and continue to be used for basic config
management, never the less i would prefer to see more secure/robust way to
configure a deployment or use it as another deployment resource.
for example: when i add a DB as a deployment stand alone object, given the
option to foreman to use it as another deployment resource.
and when i add my web app:
would be nice to have the options to ->
web_app.createDb("dbtype=postgres","dbname=my_db","description"…) .
would be great if this api to be normalized so if DB is postgres/mysql…
all should be called with the same functions and parameters for basic
operations
same would go with openstack/ovirt deployment when we want to use it as a
foreman resource and my_cloud.createVm("mem=8G",cores="2",HD="80GB"…)

so to summarize
-each stack should have basic api (of the deployment)
-api needed to configure the stack to create deployment - in complex
deployment like private cloud: hypervisors,storage…
-api that stacks can use it as a resource
(my_web_app.createDb("my_app_db"))

-stack should be derived from a class representing the stack function or
what services does it provides (db,cloud, web app…)
-a basic abstract functions should be define to each of the abstract
classes
-would be nice to have a set of tools to do some of the sysadmin work as
well (my_db.backup(…),my_cloude.backup()). giving the option to foreman to
take care of the deployment lifecycle - later on…

hope i made myself clear.
Thanks,
Moran.

··· On Wednesday, November 26, 2014 10:27:54 AM UTC+2, Petr Chalupa wrote: > > Hi > > Updating the model for stacks and adding data model of a deployment. > > Petr > > On 24.11.14 16:15, Petr Chalupa wrote: > > Thanks Greg for the nice writeup. As you've mentioned I am working on > > data model. The first part representing the Stacks (template of a > > Deployment) is attached. > > > > I'll continue working on Deployment instance representation tomorrow. > > Then I'll send some examples of actual stack instances describing simple > > apps/products. > > > > I've used PlanUML [1] and Idea plugin [2]. > > > > ## Legend > > > > Entities: > > (A) = Abstract class > > (C) = Class > > (M) = Module > > > > Arrows: > > A --|> B (blank closed arrow) = represents inheritance `B.is_a?(A)` > > A --o B (blank diamond) = Aggregation - B is composed of A > > > > Petr > > > > [1] http://plantuml.sourceforge.net/classes.html > > [2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm > > > > On 21.11.14 15:39, Greg Sutcliffe wrote: > >> Hi all, > >> > >> (TL;DR - multi-host orchestration is coming, help us make it awesome!) > >> > >> For a while we've been discussing the concept of multi-host > >> orchestration in Foreman, and how to achieve it. We've formed a > >> feature team to start looking into implementing it, and I'm writing > >> this email to give the community some insight into our planning so > >> far, and to invite input at this early to make sure we haven't missed > >> anything. (There's also quite a lot of lessons from Staypuft in this > >> document as well!) > >> > >> Some terms we're using: > >> - Resource: a Foreman object (Host, Subnet, Puppetclass etc) > >> - Stack: the instructions needed to assemble a set of resources for a > >> specific purpose (e.g. a database stack) > >> - Stacks can be nested (e.g. an app stack combines a db stack with a > >> webserver stack, etc) > >> - Deployment: the result of performing the instructions in a stack, > >> taking in any necessasry input parameters (MyTwitter is a deployment > >> of the app stack) [note: I actually favour the term Scenario here, but > >> I got outvoted - anyone else prefer this? :P] > >> > >> The current thinking is that we need some new first-class models in > >> Foreman for the Stack and Deployment objects, so that we can store > >> them. The Stack object is intended to be an abstract representation of > >> the thing to be built, the "receipe" if you will. This contains things > >> like the number of Hosts to build, the PuppetClasses to execute on > >> those hosts, and so on. Stacks can be nested, so that one can write a > >> simple stack for, say, a db cluster, and then use that stack for > >> defining a more complex object. The Stack also contains ordering, > >> where it makes sense, so that you can ensure a db host is built before > >> an app host tries to use it. > >> > >> A Deployment is an instance of a Stack, where we can record the exact > >> hosts used, the parameters passed to the puppet classes (so we can > >> correctly share things like DB passwords to the right hosts, etc), and > >> where we can at a later time update the deployment. > >> > >> For provisioning, the eventual plan is to either have the user provide > >> a set of prebuilt hosts (configured with appropriate authentication so > >> that Foreman can log in and configure them), or to have the user > >> specify one (or more) compute resources (or bare metal, via discovery > >> as staypuft does) and allow Foreman to build the necessary hosts > >> before proceeding to the provisioning step. Clearly this is only > >> needed when creating a Deployment, since a Stack is an abstract > >> representation. > >> > >> There will of course be a UI eventually, but initial efforts will be > >> focussed on creating these stacks/deployments via the API, and on > >> defining an appropriate schema for the JSON used to create them. Any > >> UI efforts will likely be just for displaying what has already been > >> created. > >> > >> To get started on this, we've broken it down into some user stories. > >> This is by no means a final list, but in *rough* order, we have > >> (indents are comments on the story): > >> > >> * A data design for stack and deployments > >> * A JSON schema for creating these objects via the API > >> * Ability to deploy a single-machine 'stack' > >> * Basically what Foreman already does, but allows us to test stack > >> nesting, etc > >> * Ensures we can deploy independant services > >> * Ensures we can deploy intra-dependant services on a single box > >> (ordering / puppetclasses) > >> * Ability to define network configuration for a stack (single network > >> only initially) > >> * Ability to detect / act upon when a host has completed it's > >> configuration > >> * Needed to know when to start the deployment of the next host in > >> sequence > >> * Staypuft uses the incoming Puppet reports, but we can probably > >> improve on this > >> * Ability to deploy a 3 machine stack on the single network > >> * 3 is arbitrary > >> * Introduces inter-depdant services between machines > >> * Ability to deploy multi-network stack > >> * Permits concepts such as compute/management networks, provisioning > >> networks etc > >> > >> * Ability to consume existing hosts within a deployment > >> * Ability to spin up hosts on a single compute resource for a > deployment > >> * Ability to spin up hosts on multiple compute resources for a > deployment > >> * Ability to consume discovered hosts via the Discovery plugin for a > >> deployment > >> > >> The ultimate goal is to make these Stack objects shareable, so that we > >> can all contribute to having well designed stacks running within our > >> infrastructures. We have a long way to go though. > >> > >> Some of you will want to know what research we've done, and our > >> thoughts on the tools already out there - I'll let Daniel reply with > >> that info. Petr is already working on the data designs, and will reply > >> shortly with his work so far. For myself, I'll be picking up the task > >> of extending how we know when a host is "finished" which, since it > >> involves things like running Puppet after reboot, is more nuanced than > >> simply receiving the /unattended/built call from the installer. > >> > >> Thanks for reading, and we want to hear your thoughts on the design! > >> > >> Greg > >> > > >
13

Hi Petr,all

I'm pretty new to foreman, so please bear with me.
after reviewing the suggested models i'm mostly missing the child -> parent
stack/deployment connection possibility.

for example:
-i have deployed a DB - most basic deployment child in the chain (i think)
-now i would like to use it as a central DB for some of my other
deployments, can you explain how it is being done in the current model?

I think that this result might be achieved by exposing the some of the api
of the installed product.
and from where i see it, it's actually been done in foreman before -
compute resource is good example of it.
so, the stack would be openstack/oVirt/VMware… i would imagine the stack
class is inherited from compute abstract class. and this stack should tell
the story on how foreman can install it, the story should include a way to
configure their sub services like nova, glance, storage…
now, the outcome of such implantation of this stack is a compute resource,
which have a common set of functions - run vm /stop vm … no matter what
provider/stack is underneath.

So going back to the DB example, when openstack deployment is taking place,
and a DB is needed for on of the components i can use the central DB which
is now available as a DB_resource. with the api to create DB, managing
users etc. and i don't really care if it's postgres/mysql… i have basic
functions to do it without caring too much on the layer underneath (if of
course the stack allows it)

so everything we deploy using foreman can actually become a resource that
can be used for other deployments in foreman.

Thanks,
Moran.

··· On Wednesday, November 26, 2014 10:27:54 AM UTC+2, Petr Chalupa wrote: > > Hi > > Updating the model for stacks and adding data model of a deployment. > > Petr > > On 24.11.14 16:15, Petr Chalupa wrote: > > Thanks Greg for the nice writeup. As you've mentioned I am working on > > data model. The first part representing the Stacks (template of a > > Deployment) is attached. > > > > I'll continue working on Deployment instance representation tomorrow. > > Then I'll send some examples of actual stack instances describing simple > > apps/products. > > > > I've used PlanUML [1] and Idea plugin [2]. > > > > ## Legend > > > > Entities: > > (A) = Abstract class > > (C) = Class > > (M) = Module > > > > Arrows: > > A --|> B (blank closed arrow) = represents inheritance `B.is_a?(A)` > > A --o B (blank diamond) = Aggregation - B is composed of A > > > > Petr > > > > [1] http://plantuml.sourceforge.net/classes.html > > [2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm > > > > On 21.11.14 15:39, Greg Sutcliffe wrote: > >> Hi all, > >> > >> (TL;DR - multi-host orchestration is coming, help us make it awesome!) > >> > >> For a while we've been discussing the concept of multi-host > >> orchestration in Foreman, and how to achieve it. We've formed a > >> feature team to start looking into implementing it, and I'm writing > >> this email to give the community some insight into our planning so > >> far, and to invite input at this early to make sure we haven't missed > >> anything. (There's also quite a lot of lessons from Staypuft in this > >> document as well!) > >> > >> Some terms we're using: > >> - Resource: a Foreman object (Host, Subnet, Puppetclass etc) > >> - Stack: the instructions needed to assemble a set of resources for a > >> specific purpose (e.g. a database stack) > >> - Stacks can be nested (e.g. an app stack combines a db stack with a > >> webserver stack, etc) > >> - Deployment: the result of performing the instructions in a stack, > >> taking in any necessasry input parameters (MyTwitter is a deployment > >> of the app stack) [note: I actually favour the term Scenario here, but > >> I got outvoted - anyone else prefer this? :P] > >> > >> The current thinking is that we need some new first-class models in > >> Foreman for the Stack and Deployment objects, so that we can store > >> them. The Stack object is intended to be an abstract representation of > >> the thing to be built, the "receipe" if you will. This contains things > >> like the number of Hosts to build, the PuppetClasses to execute on > >> those hosts, and so on. Stacks can be nested, so that one can write a > >> simple stack for, say, a db cluster, and then use that stack for > >> defining a more complex object. The Stack also contains ordering, > >> where it makes sense, so that you can ensure a db host is built before > >> an app host tries to use it. > >> > >> A Deployment is an instance of a Stack, where we can record the exact > >> hosts used, the parameters passed to the puppet classes (so we can > >> correctly share things like DB passwords to the right hosts, etc), and > >> where we can at a later time update the deployment. > >> > >> For provisioning, the eventual plan is to either have the user provide > >> a set of prebuilt hosts (configured with appropriate authentication so > >> that Foreman can log in and configure them), or to have the user > >> specify one (or more) compute resources (or bare metal, via discovery > >> as staypuft does) and allow Foreman to build the necessary hosts > >> before proceeding to the provisioning step. Clearly this is only > >> needed when creating a Deployment, since a Stack is an abstract > >> representation. > >> > >> There will of course be a UI eventually, but initial efforts will be > >> focussed on creating these stacks/deployments via the API, and on > >> defining an appropriate schema for the JSON used to create them. Any > >> UI efforts will likely be just for displaying what has already been > >> created. > >> > >> To get started on this, we've broken it down into some user stories. > >> This is by no means a final list, but in *rough* order, we have > >> (indents are comments on the story): > >> > >> * A data design for stack and deployments > >> * A JSON schema for creating these objects via the API > >> * Ability to deploy a single-machine 'stack' > >> * Basically what Foreman already does, but allows us to test stack > >> nesting, etc > >> * Ensures we can deploy independant services > >> * Ensures we can deploy intra-dependant services on a single box > >> (ordering / puppetclasses) > >> * Ability to define network configuration for a stack (single network > >> only initially) > >> * Ability to detect / act upon when a host has completed it's > >> configuration > >> * Needed to know when to start the deployment of the next host in > >> sequence > >> * Staypuft uses the incoming Puppet reports, but we can probably > >> improve on this > >> * Ability to deploy a 3 machine stack on the single network > >> * 3 is arbitrary > >> * Introduces inter-depdant services between machines > >> * Ability to deploy multi-network stack > >> * Permits concepts such as compute/management networks, provisioning > >> networks etc > >> > >> * Ability to consume existing hosts within a deployment > >> * Ability to spin up hosts on a single compute resource for a > deployment > >> * Ability to spin up hosts on multiple compute resources for a > deployment > >> * Ability to consume discovered hosts via the Discovery plugin for a > >> deployment > >> > >> The ultimate goal is to make these Stack objects shareable, so that we > >> can all contribute to having well designed stacks running within our > >> infrastructures. We have a long way to go though. > >> > >> Some of you will want to know what research we've done, and our > >> thoughts on the tools already out there - I'll let Daniel reply with > >> that info. Petr is already working on the data designs, and will reply > >> shortly with his work so far. For myself, I'll be picking up the task > >> of extending how we know when a host is "finished" which, since it > >> involves things like running Puppet after reboot, is more nuanced than > >> simply receiving the /unattended/built call from the installer. > >> > >> Thanks for reading, and we want to hear your thoughts on the design! > >> > >> Greg > >> > > >
14

Removing any associations to existing Foreman model from stacks to allow
stack sharing without problems between users.

deployment.puml (1.85 KB)

··· On 26.11.14 9:27, Petr Chalupa wrote: > Hi > > Updating the model for stacks and adding data model of a deployment. > > Petr > > On 24.11.14 16:15, Petr Chalupa wrote: >> Thanks Greg for the nice writeup. As you've mentioned I am working on >> data model. The first part representing the Stacks (template of a >> Deployment) is attached. >> >> I'll continue working on Deployment instance representation tomorrow. >> Then I'll send some examples of actual stack instances describing simple >> apps/products. >> >> I've used PlanUML [1] and Idea plugin [2]. >> >> ## Legend >> >> Entities: >> (A) = Abstract class >> (C) = Class >> (M) = Module >> >> Arrows: >> A --|> B (blank closed arrow) = represents inheritance `B.is_a?(A)` >> A --o B (blank diamond) = Aggregation - B is composed of A >> >> Petr >> >> [1] http://plantuml.sourceforge.net/classes.html >> [2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm >> >> On 21.11.14 15:39, Greg Sutcliffe wrote: >>> Hi all, >>> >>> (TL;DR - multi-host orchestration is coming, help us make it awesome!) >>> >>> For a while we've been discussing the concept of multi-host >>> orchestration in Foreman, and how to achieve it. We've formed a >>> feature team to start looking into implementing it, and I'm writing >>> this email to give the community some insight into our planning so >>> far, and to invite input at this early to make sure we haven't missed >>> anything. (There's also quite a lot of lessons from Staypuft in this >>> document as well!) >>> >>> Some terms we're using: >>> - Resource: a Foreman object (Host, Subnet, Puppetclass etc) >>> - Stack: the instructions needed to assemble a set of resources for a >>> specific purpose (e.g. a database stack) >>> - Stacks can be nested (e.g. an app stack combines a db stack with a >>> webserver stack, etc) >>> - Deployment: the result of performing the instructions in a stack, >>> taking in any necessasry input parameters (MyTwitter is a deployment >>> of the app stack) [note: I actually favour the term Scenario here, but >>> I got outvoted - anyone else prefer this? :P] >>> >>> The current thinking is that we need some new first-class models in >>> Foreman for the Stack and Deployment objects, so that we can store >>> them. The Stack object is intended to be an abstract representation of >>> the thing to be built, the "receipe" if you will. This contains things >>> like the number of Hosts to build, the PuppetClasses to execute on >>> those hosts, and so on. Stacks can be nested, so that one can write a >>> simple stack for, say, a db cluster, and then use that stack for >>> defining a more complex object. The Stack also contains ordering, >>> where it makes sense, so that you can ensure a db host is built before >>> an app host tries to use it. >>> >>> A Deployment is an instance of a Stack, where we can record the exact >>> hosts used, the parameters passed to the puppet classes (so we can >>> correctly share things like DB passwords to the right hosts, etc), and >>> where we can at a later time update the deployment. >>> >>> For provisioning, the eventual plan is to either have the user provide >>> a set of prebuilt hosts (configured with appropriate authentication so >>> that Foreman can log in and configure them), or to have the user >>> specify one (or more) compute resources (or bare metal, via discovery >>> as staypuft does) and allow Foreman to build the necessary hosts >>> before proceeding to the provisioning step. Clearly this is only >>> needed when creating a Deployment, since a Stack is an abstract >>> representation. >>> >>> There will of course be a UI eventually, but initial efforts will be >>> focussed on creating these stacks/deployments via the API, and on >>> defining an appropriate schema for the JSON used to create them. Any >>> UI efforts will likely be just for displaying what has already been >>> created. >>> >>> To get started on this, we've broken it down into some user stories. >>> This is by no means a final list, but in *rough* order, we have >>> (indents are comments on the story): >>> >>> * A data design for stack and deployments >>> * A JSON schema for creating these objects via the API >>> * Ability to deploy a single-machine 'stack' >>> * Basically what Foreman already does, but allows us to test stack >>> nesting, etc >>> * Ensures we can deploy independant services >>> * Ensures we can deploy intra-dependant services on a single box >>> (ordering / puppetclasses) >>> * Ability to define network configuration for a stack (single network >>> only initially) >>> * Ability to detect / act upon when a host has completed it's >>> configuration >>> * Needed to know when to start the deployment of the next host in >>> sequence >>> * Staypuft uses the incoming Puppet reports, but we can probably >>> improve on this >>> * Ability to deploy a 3 machine stack on the single network >>> * 3 is arbitrary >>> * Introduces inter-depdant services between machines >>> * Ability to deploy multi-network stack >>> * Permits concepts such as compute/management networks, provisioning >>> networks etc >>> >>> * Ability to consume existing hosts within a deployment >>> * Ability to spin up hosts on a single compute resource for a deployment >>> * Ability to spin up hosts on multiple compute resources for a >>> deployment >>> * Ability to consume discovered hosts via the Discovery plugin for a >>> deployment >>> >>> The ultimate goal is to make these Stack objects shareable, so that we >>> can all contribute to having well designed stacks running within our >>> infrastructures. We have a long way to go though. >>> >>> Some of you will want to know what research we've done, and our >>> thoughts on the tools already out there - I'll let Daniel reply with >>> that info. Petr is already working on the data designs, and will reply >>> shortly with his work so far. For myself, I'll be picking up the task >>> of extending how we know when a host is "finished" which, since it >>> involves things like running Puppet after reboot, is more nuanced than >>> simply receiving the /unattended/built call from the installer. >>> >>> Thanks for reading, and we want to hear your thoughts on the design! >>> >>> Greg >>> >> >
15

Potentially, but (initially) only if you could already deploy that
today, one object at a time. This is focussed on automating many
things Foreman already knows how to deploy in one step - when I say a
loadbalancer in this context, I'm really talking about a host deployed
with (say) EL7 and Nginx, as configured by puppet, rather than
anything more speicifc. The app stack then becomes about consistently
deploying the same configuration of Nginx, with the same configuration
of the app node, with the same configuration of the db cluster, all
from one place in Foreman.

That said, once if Foreman is able to deploy other types of things
(for example, there's a lot of hype for deploying Docker containers
instead of hosts), then these will be usable in the deployments
schema. The point of the deployments is to deploy many Foreman things
at once, in a repeatable, sharable way. It doesn't really mandate
what you deploy, so long as Foreman knows how.

Greg

··· On 26 November 2014 at 11:34, James Bailey wrote: > Does this mean that cloud resources such as block storage and load balancers > will finally become part of Foreman? > > Say I want to create an ELK cluster in a cloud environment and as part of > that I need to attach 1TB block storage to each of my Elasticsearch nodes, > as well as cluster Kibana running on some nginx hosts behind a load > balancer. Is this finally going to be possible from within Foreman?
16

please discard this post, it was sent by mistake.
made myself more clear on the following one.
thanks.

··· On Thursday, November 27, 2014 3:27:01 PM UTC+2, mgol...@redhat.com wrote: > > Hi Pert,all > > I'm pretty new in foreman, so please bear with me. Deployment feature > sounds exiting. > modeling looks good, few questions than remarks: > > -doesn't the HostgroupResource already include subnet management? and if > not what is the redundancy with the subnetTypeResource > -what exactly does the ComputeResource/Profile is in-charge of? > > i understand puppet is used and continue to be used for basic config > management, never the less i would prefer to see more secure/robust way to > configure a deployment or use it as another deployment resource. > for example: when i add a DB as a deployment stand alone object, given the > option to foreman to use it as another deployment resource. > and when i add my web app: > would be nice to have the options to -> > web_app.createDb("dbtype=postgres","dbname=my_db","description"...) . > would be great if this api to be normalized so if DB is postgres/mysql... > all should be called with the same functions and parameters for basic > operations > same would go with openstack/ovirt deployment when we want to use it as a > foreman resource and my_cloud.createVm("mem=8G",cores="2",HD="80GB"...) > > so to summarize > -each stack should have basic api (of the deployment) > -api needed to configure the stack to create deployment - in complex > deployment like private cloud: hypervisors,storage.. > -api that stacks can use it as a resource > (my_web_app.createDb("my_app_db")) > > -stack should be derived from a class representing the stack function or > what services does it provides (db,cloud, web app...) > -a basic abstract functions should be define to each of the abstract > classes > -would be nice to have a set of tools to do some of the sysadmin work as > well (my_db.backup(..),my_cloude.backup()). giving the option to foreman to > take care of the deployment lifecycle - later on.. > > hope i made myself clear. > Thanks, > Moran. > > > > > > > On Wednesday, November 26, 2014 10:27:54 AM UTC+2, Petr Chalupa wrote: >> >> Hi >> >> Updating the model for stacks and adding data model of a deployment. >> >> Petr >> >> On 24.11.14 16:15, Petr Chalupa wrote: >> > Thanks Greg for the nice writeup. As you've mentioned I am working on >> > data model. The first part representing the Stacks (template of a >> > Deployment) is attached. >> > >> > I'll continue working on Deployment instance representation tomorrow. >> > Then I'll send some examples of actual stack instances describing >> simple >> > apps/products. >> > >> > I've used PlanUML [1] and Idea plugin [2]. >> > >> > ## Legend >> > >> > Entities: >> > (A) = Abstract class >> > (C) = Class >> > (M) = Module >> > >> > Arrows: >> > A --|> B (blank closed arrow) = represents inheritance `B.is_a?(A)` >> > A --o B (blank diamond) = Aggregation - B is composed of A >> > >> > Petr >> > >> > [1] http://plantuml.sourceforge.net/classes.html >> > [2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm >> > >> > On 21.11.14 15:39, Greg Sutcliffe wrote: >> >> Hi all, >> >> >> >> (TL;DR - multi-host orchestration is coming, help us make it awesome!) >> >> >> >> For a while we've been discussing the concept of multi-host >> >> orchestration in Foreman, and how to achieve it. We've formed a >> >> feature team to start looking into implementing it, and I'm writing >> >> this email to give the community some insight into our planning so >> >> far, and to invite input at this early to make sure we haven't missed >> >> anything. (There's also quite a lot of lessons from Staypuft in this >> >> document as well!) >> >> >> >> Some terms we're using: >> >> - Resource: a Foreman object (Host, Subnet, Puppetclass etc) >> >> - Stack: the instructions needed to assemble a set of resources for a >> >> specific purpose (e.g. a database stack) >> >> - Stacks can be nested (e.g. an app stack combines a db stack with a >> >> webserver stack, etc) >> >> - Deployment: the result of performing the instructions in a stack, >> >> taking in any necessasry input parameters (MyTwitter is a deployment >> >> of the app stack) [note: I actually favour the term Scenario here, but >> >> I got outvoted - anyone else prefer this? :P] >> >> >> >> The current thinking is that we need some new first-class models in >> >> Foreman for the Stack and Deployment objects, so that we can store >> >> them. The Stack object is intended to be an abstract representation of >> >> the thing to be built, the "receipe" if you will. This contains things >> >> like the number of Hosts to build, the PuppetClasses to execute on >> >> those hosts, and so on. Stacks can be nested, so that one can write a >> >> simple stack for, say, a db cluster, and then use that stack for >> >> defining a more complex object. The Stack also contains ordering, >> >> where it makes sense, so that you can ensure a db host is built before >> >> an app host tries to use it. >> >> >> >> A Deployment is an instance of a Stack, where we can record the exact >> >> hosts used, the parameters passed to the puppet classes (so we can >> >> correctly share things like DB passwords to the right hosts, etc), and >> >> where we can at a later time update the deployment. >> >> >> >> For provisioning, the eventual plan is to either have the user provide >> >> a set of prebuilt hosts (configured with appropriate authentication so >> >> that Foreman can log in and configure them), or to have the user >> >> specify one (or more) compute resources (or bare metal, via discovery >> >> as staypuft does) and allow Foreman to build the necessary hosts >> >> before proceeding to the provisioning step. Clearly this is only >> >> needed when creating a Deployment, since a Stack is an abstract >> >> representation. >> >> >> >> There will of course be a UI eventually, but initial efforts will be >> >> focussed on creating these stacks/deployments via the API, and on >> >> defining an appropriate schema for the JSON used to create them. Any >> >> UI efforts will likely be just for displaying what has already been >> >> created. >> >> >> >> To get started on this, we've broken it down into some user stories. >> >> This is by no means a final list, but in *rough* order, we have >> >> (indents are comments on the story): >> >> >> >> * A data design for stack and deployments >> >> * A JSON schema for creating these objects via the API >> >> * Ability to deploy a single-machine 'stack' >> >> * Basically what Foreman already does, but allows us to test stack >> >> nesting, etc >> >> * Ensures we can deploy independant services >> >> * Ensures we can deploy intra-dependant services on a single box >> >> (ordering / puppetclasses) >> >> * Ability to define network configuration for a stack (single network >> >> only initially) >> >> * Ability to detect / act upon when a host has completed it's >> >> configuration >> >> * Needed to know when to start the deployment of the next host in >> >> sequence >> >> * Staypuft uses the incoming Puppet reports, but we can probably >> >> improve on this >> >> * Ability to deploy a 3 machine stack on the single network >> >> * 3 is arbitrary >> >> * Introduces inter-depdant services between machines >> >> * Ability to deploy multi-network stack >> >> * Permits concepts such as compute/management networks, >> provisioning >> >> networks etc >> >> >> >> * Ability to consume existing hosts within a deployment >> >> * Ability to spin up hosts on a single compute resource for a >> deployment >> >> * Ability to spin up hosts on multiple compute resources for a >> deployment >> >> * Ability to consume discovered hosts via the Discovery plugin for a >> >> deployment >> >> >> >> The ultimate goal is to make these Stack objects shareable, so that we >> >> can all contribute to having well designed stacks running within our >> >> infrastructures. We have a long way to go though. >> >> >> >> Some of you will want to know what research we've done, and our >> >> thoughts on the tools already out there - I'll let Daniel reply with >> >> that info. Petr is already working on the data designs, and will reply >> >> shortly with his work so far. For myself, I'll be picking up the task >> >> of extending how we know when a host is "finished" which, since it >> >> involves things like running Puppet after reboot, is more nuanced than >> >> simply receiving the /unattended/built call from the installer. >> >> >> >> Thanks for reading, and we want to hear your thoughts on the design! >> >> >> >> Greg >> >> >> > >> >
17

Updating ComputeResourceResource to allow image based provisioning.
Adding flow.md describing the deployment life-cycle, also copied below.

Deployment flow

If user is not mentioned then the step is done automatically.

  • Show Deployment form to User.
    • User inputs name and description.
    • Form is submitted.
  • Deployment is created.
  • User creates required resources one by one.
  • User assigns existing foreman Subnets with SubnetTypesResources.
  • User Starts by creating all the Hostgroups for each
    HostgroupResource.
    (Custom form allowing to select just items bellow.)
    • If parent is not HostgroupResource, it needs to provided by user.
    • Hostgroup inherits configuration from parent, remaining options
      are provided by user.
    • Hostgroup is created.
    • ParamResources adds the parameters to the hostgroup.
    • Puppetclasses are looked up by PuppetClassResources.
    • ParamOverideResources overrides defined puppet class parameters.
  • User continues by creating all the Hosts for each HostResouce.
    (Custom form allowing to select just items bellow.)
    • User assigns Subnets to subnet types,
    • picks ComputeResource based on allow-list defined by
      ComputeResourceResources.
    • User selects provisioning (true is default) (Some managed
      hosts may be
      just reused without provisioning).
    • User defines number of the Hosts required within min, max
      defined on HostResource.
      • Bare metal hosts are assigned if applicable.
    • Hosts are created or if already existing just assigned to proper
      Hostgroup.
    • Provisioning may already start if it cannot be delayed. But the
      puppetrun has to be always delayed, it's orchestrated later.
  • All hosts are just created waiting for provisioning or only provisioned
    without configuration triggered.
  • User triggers deployment.
    • All hosts are provisioned if required.
    • Waits until all hosts are provisioned.
    • Orchestration of the Ordered resources. e.g.:
      • Puppetrun an all controller nodes in parallel,
      • then puppetrun on all compute nodes in parallel.

Petr

deployment.puml (1.85 KB)

··· On 28.11.14 9:13, Petr Chalupa wrote: > Removing any associations to existing Foreman model from stacks to allow > stack sharing without problems between users. > > On 26.11.14 9:27, Petr Chalupa wrote: >> Hi >> >> Updating the model for stacks and adding data model of a deployment. >> >> Petr >> >> On 24.11.14 16:15, Petr Chalupa wrote: >>> Thanks Greg for the nice writeup. As you've mentioned I am working on >>> data model. The first part representing the Stacks (template of a >>> Deployment) is attached. >>> >>> I'll continue working on Deployment instance representation tomorrow. >>> Then I'll send some examples of actual stack instances describing simple >>> apps/products. >>> >>> I've used PlanUML [1] and Idea plugin [2]. >>> >>> ## Legend >>> >>> Entities: >>> (A) = Abstract class >>> (C) = Class >>> (M) = Module >>> >>> Arrows: >>> A --|> B (blank closed arrow) = represents inheritance `B.is_a?(A)` >>> A --o B (blank diamond) = Aggregation - B is composed of A >>> >>> Petr >>> >>> [1] http://plantuml.sourceforge.net/classes.html >>> [2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm >>> >>> On 21.11.14 15:39, Greg Sutcliffe wrote: >>>> Hi all, >>>> >>>> (TL;DR - multi-host orchestration is coming, help us make it awesome!) >>>> >>>> For a while we've been discussing the concept of multi-host >>>> orchestration in Foreman, and how to achieve it. We've formed a >>>> feature team to start looking into implementing it, and I'm writing >>>> this email to give the community some insight into our planning so >>>> far, and to invite input at this early to make sure we haven't missed >>>> anything. (There's also quite a lot of lessons from Staypuft in this >>>> document as well!) >>>> >>>> Some terms we're using: >>>> - Resource: a Foreman object (Host, Subnet, Puppetclass etc) >>>> - Stack: the instructions needed to assemble a set of resources for a >>>> specific purpose (e.g. a database stack) >>>> - Stacks can be nested (e.g. an app stack combines a db stack with a >>>> webserver stack, etc) >>>> - Deployment: the result of performing the instructions in a stack, >>>> taking in any necessasry input parameters (MyTwitter is a deployment >>>> of the app stack) [note: I actually favour the term Scenario here, but >>>> I got outvoted - anyone else prefer this? :P] >>>> >>>> The current thinking is that we need some new first-class models in >>>> Foreman for the Stack and Deployment objects, so that we can store >>>> them. The Stack object is intended to be an abstract representation of >>>> the thing to be built, the "receipe" if you will. This contains things >>>> like the number of Hosts to build, the PuppetClasses to execute on >>>> those hosts, and so on. Stacks can be nested, so that one can write a >>>> simple stack for, say, a db cluster, and then use that stack for >>>> defining a more complex object. The Stack also contains ordering, >>>> where it makes sense, so that you can ensure a db host is built before >>>> an app host tries to use it. >>>> >>>> A Deployment is an instance of a Stack, where we can record the exact >>>> hosts used, the parameters passed to the puppet classes (so we can >>>> correctly share things like DB passwords to the right hosts, etc), and >>>> where we can at a later time update the deployment. >>>> >>>> For provisioning, the eventual plan is to either have the user provide >>>> a set of prebuilt hosts (configured with appropriate authentication so >>>> that Foreman can log in and configure them), or to have the user >>>> specify one (or more) compute resources (or bare metal, via discovery >>>> as staypuft does) and allow Foreman to build the necessary hosts >>>> before proceeding to the provisioning step. Clearly this is only >>>> needed when creating a Deployment, since a Stack is an abstract >>>> representation. >>>> >>>> There will of course be a UI eventually, but initial efforts will be >>>> focussed on creating these stacks/deployments via the API, and on >>>> defining an appropriate schema for the JSON used to create them. Any >>>> UI efforts will likely be just for displaying what has already been >>>> created. >>>> >>>> To get started on this, we've broken it down into some user stories. >>>> This is by no means a final list, but in *rough* order, we have >>>> (indents are comments on the story): >>>> >>>> * A data design for stack and deployments >>>> * A JSON schema for creating these objects via the API >>>> * Ability to deploy a single-machine 'stack' >>>> * Basically what Foreman already does, but allows us to test stack >>>> nesting, etc >>>> * Ensures we can deploy independant services >>>> * Ensures we can deploy intra-dependant services on a single box >>>> (ordering / puppetclasses) >>>> * Ability to define network configuration for a stack (single network >>>> only initially) >>>> * Ability to detect / act upon when a host has completed it's >>>> configuration >>>> * Needed to know when to start the deployment of the next host in >>>> sequence >>>> * Staypuft uses the incoming Puppet reports, but we can probably >>>> improve on this >>>> * Ability to deploy a 3 machine stack on the single network >>>> * 3 is arbitrary >>>> * Introduces inter-depdant services between machines >>>> * Ability to deploy multi-network stack >>>> * Permits concepts such as compute/management networks, provisioning >>>> networks etc >>>> >>>> * Ability to consume existing hosts within a deployment >>>> * Ability to spin up hosts on a single compute resource for a >>>> deployment >>>> * Ability to spin up hosts on multiple compute resources for a >>>> deployment >>>> * Ability to consume discovered hosts via the Discovery plugin for a >>>> deployment >>>> >>>> The ultimate goal is to make these Stack objects shareable, so that we >>>> can all contribute to having well designed stacks running within our >>>> infrastructures. We have a long way to go though. >>>> >>>> Some of you will want to know what research we've done, and our >>>> thoughts on the tools already out there - I'll let Daniel reply with >>>> that info. Petr is already working on the data designs, and will reply >>>> shortly with his work so far. For myself, I'll be picking up the task >>>> of extending how we know when a host is "finished" which, since it >>>> involves things like running Puppet after reboot, is more nuanced than >>>> simply receiving the /unattended/built call from the installer. >>>> >>>> Thanks for reading, and we want to hear your thoughts on the design! >>>> >>>> Greg >>>> >>> >> >
18

Hello Moran,

I've recognized 3 stories, I'll try to rephrase them. Please correct me
if I got it wrong.

  1. Sharing a stack which defines DB with multiple deployments allowing
    to get the DB API URL for sharing with other stacks in the deployment.

  2. Adding of an object representing the DB instance which would allow
    simple operations like adding DB.

  3. Produce a configured ComputeResource for RHEV which was just
    deployed. Allow to use the RHEV's ComputeResource to provision new hosts.

Let me address them:

  1. This should be supported out of the box with this model. There can be
    a ParamResource defined in the stack which will represent a API URL.
    It's value can be computed automatically with ERB snippet. The value can
    be then used in other stacks in the same deployment.

  2. That could be achieved through plugin, which would provide this
    object and possibly other ordered-resources (like create-db) which could
    be then used in other stacks. Although I think this is nice-to-have
    feature for future. Also we should contemplate if it is a good practice,
    it may be preferable to use 1. and the actions (like db creation)
    implement with puppet classes (applied in the stack using the DB).

  3. I was already thinking about that, we will need to do this in fusor.
    Rhev stack needs to produce a configured ComputeResource to be able to
    deploy CloudForms on it. It can be achieved by adding custom
    ordered-resource which will create and configure the ComputeResource as
    a last step within the RHEV's stack.

Petr

··· On 27.11.14 17:11, mgoldboi@redhat.com wrote: > Hi Petr,all > > I'm pretty new to foreman, so please bear with me. > after reviewing the suggested models i'm mostly missing the child -> > parent stack/deployment connection possibility. > > for example: > -i have deployed a DB - most basic deployment child in the chain (i think) > -now i would like to use it as a central DB for some of my other > deployments, can you explain how it is being done in the current model? > > I think that this result might be achieved by exposing the some of the > api of the installed product. > and from where i see it, it's actually been done in foreman before - > compute resource is good example of it. > so, the stack would be openstack/oVirt/VMware... i would imagine the > stack class is inherited from compute abstract class. and this stack > should tell the story on how foreman can install it, the story should > include a way to configure their sub services like nova, glance, storage.... > now, the outcome of such implantation of this stack is a compute > resource, which have a common set of functions - run vm /stop vm ... no > matter what provider/stack is underneath. > So going back to the DB example, when openstack deployment is taking > place, and a DB is needed for on of the components i can use the > central DB which is now available as a DB_resource. with the api to > create DB, managing users etc. and i don't really care if it's > postgres/mysql.. i have basic functions to do it without caring too much > on the layer underneath (if of course the stack allows it) > > so everything we deploy using foreman can actually become a resource > that can be used for other deployments in foreman. > > Thanks, > Moran. > > On Wednesday, November 26, 2014 10:27:54 AM UTC+2, Petr Chalupa wrote: > > Hi > > Updating the model for stacks and adding data model of a deployment. > > Petr > > On 24.11.14 16:15, Petr Chalupa wrote: > > Thanks Greg for the nice writeup. As you've mentioned I am > working on > > data model. The first part representing the Stacks (template of a > > Deployment) is attached. > > > > I'll continue working on Deployment instance representation > tomorrow. > > Then I'll send some examples of actual stack instances describing > simple > > apps/products. > > > > I've used PlanUML [1] and Idea plugin [2]. > > > > ## Legend > > > > Entities: > > (A) = Abstract class > > (C) = Class > > (M) = Module > > > > Arrows: > > A --|> B (blank closed arrow) = represents inheritance `B.is_a?(A)` > > A --o B (blank diamond) = Aggregation - B is composed of A > > > > Petr > > > > [1] http://plantuml.sourceforge.net/classes.html > > > [2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm > > > > > On 21.11.14 15:39, Greg Sutcliffe wrote: > >> Hi all, > >> > >> (TL;DR - multi-host orchestration is coming, help us make it > awesome!) > >> > >> For a while we've been discussing the concept of multi-host > >> orchestration in Foreman, and how to achieve it. We've formed a > >> feature team to start looking into implementing it, and I'm writing > >> this email to give the community some insight into our planning so > >> far, and to invite input at this early to make sure we haven't > missed > >> anything. (There's also quite a lot of lessons from Staypuft in > this > >> document as well!) > >> > >> Some terms we're using: > >> - Resource: a Foreman object (Host, Subnet, Puppetclass etc) > >> - Stack: the instructions needed to assemble a set of resources > for a > >> specific purpose (e.g. a database stack) > >> - Stacks can be nested (e.g. an app stack combines a db stack > with a > >> webserver stack, etc) > >> - Deployment: the result of performing the instructions in a stack, > >> taking in any necessasry input parameters (MyTwitter is a > deployment > >> of the app stack) [note: I actually favour the term Scenario > here, but > >> I got outvoted - anyone else prefer this? :P] > >> > >> The current thinking is that we need some new first-class models in > >> Foreman for the Stack and Deployment objects, so that we can store > >> them. The Stack object is intended to be an abstract > representation of > >> the thing to be built, the "receipe" if you will. This contains > things > >> like the number of Hosts to build, the PuppetClasses to execute on > >> those hosts, and so on. Stacks can be nested, so that one can > write a > >> simple stack for, say, a db cluster, and then use that stack for > >> defining a more complex object. The Stack also contains ordering, > >> where it makes sense, so that you can ensure a db host is built > before > >> an app host tries to use it. > >> > >> A Deployment is an instance of a Stack, where we can record the > exact > >> hosts used, the parameters passed to the puppet classes (so we can > >> correctly share things like DB passwords to the right hosts, > etc), and > >> where we can at a later time update the deployment. > >> > >> For provisioning, the eventual plan is to either have the user > provide > >> a set of prebuilt hosts (configured with appropriate > authentication so > >> that Foreman can log in and configure them), or to have the user > >> specify one (or more) compute resources (or bare metal, via > discovery > >> as staypuft does) and allow Foreman to build the necessary hosts > >> before proceeding to the provisioning step. Clearly this is only > >> needed when creating a Deployment, since a Stack is an abstract > >> representation. > >> > >> There will of course be a UI eventually, but initial efforts > will be > >> focussed on creating these stacks/deployments via the API, and on > >> defining an appropriate schema for the JSON used to create them. > Any > >> UI efforts will likely be just for displaying what has already been > >> created. > >> > >> To get started on this, we've broken it down into some user > stories. > >> This is by no means a final list, but in *rough* order, we have > >> (indents are comments on the story): > >> > >> * A data design for stack and deployments > >> * A JSON schema for creating these objects via the API > >> * Ability to deploy a single-machine 'stack' > >> * Basically what Foreman already does, but allows us to test > stack > >> nesting, etc > >> * Ensures we can deploy independant services > >> * Ensures we can deploy intra-dependant services on a single box > >> (ordering / puppetclasses) > >> * Ability to define network configuration for a stack (single > network > >> only initially) > >> * Ability to detect / act upon when a host has completed it's > >> configuration > >> * Needed to know when to start the deployment of the next > host in > >> sequence > >> * Staypuft uses the incoming Puppet reports, but we can probably > >> improve on this > >> * Ability to deploy a 3 machine stack on the single network > >> * 3 is arbitrary > >> * Introduces inter-depdant services between machines > >> * Ability to deploy multi-network stack > >> * Permits concepts such as compute/management networks, > provisioning > >> networks etc > >> > >> * Ability to consume existing hosts within a deployment > >> * Ability to spin up hosts on a single compute resource for a > deployment > >> * Ability to spin up hosts on multiple compute resources for a > deployment > >> * Ability to consume discovered hosts via the Discovery plugin > for a > >> deployment > >> > >> The ultimate goal is to make these Stack objects shareable, so > that we > >> can all contribute to having well designed stacks running within > our > >> infrastructures. We have a long way to go though. > >> > >> Some of you will want to know what research we've done, and our > >> thoughts on the tools already out there - I'll let Daniel reply > with > >> that info. Petr is already working on the data designs, and will > reply > >> shortly with his work so far. For myself, I'll be picking up the > task > >> of extending how we know when a host is "finished" which, since it > >> involves things like running Puppet after reboot, is more > nuanced than > >> simply receiving the /unattended/built call from the installer. > >> > >> Thanks for reading, and we want to hear your thoughts on the > design! > >> > >> Greg > >> > > > > -- > You received this message because you are subscribed to the Google > Groups "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to foreman-dev+unsubscribe@googlegroups.com > . > For more options, visit https://groups.google.com/d/optout.
19

Petr,

Would it be possible to do a deep dive/ review on the model?

thanks,
Ohad

··· On Mon, Dec 1, 2014 at 1:43 PM, Petr Chalupa wrote:

Updating ComputeResourceResource to allow image based provisioning.
Adding flow.md describing the deployment life-cycle, also copied below.

Deployment flow

If user is not mentioned then the step is done automatically.

  • Show Deployment form to User.
    • User inputs name and description.
    • Form is submitted.
  • Deployment is created.
  • User creates required resources one by one.
  • User assigns existing foreman Subnets with
    SubnetTypesResources.
  • User Starts by creating all the Hostgroups for each
    HostgroupResource.
    (Custom form allowing to select just items bellow.)
    • If parent is not HostgroupResource, it needs to provided by user.
    • Hostgroup inherits configuration from parent, remaining options
      are provided by user.
    • Hostgroup is created.
    • ParamResources adds the parameters to the hostgroup.
    • Puppetclasses are looked up by PuppetClassResources.
    • ParamOverideResources overrides defined puppet class parameters.
  • User continues by creating all the Hosts for each HostResouce.
    (Custom form allowing to select just items bellow.)
    • User assigns Subnets to subnet types,
    • picks ComputeResource based on allow-list defined by
      ComputeResourceResources.
    • User selects provisioning (true is default) (Some managed
      hosts may be
      just reused without provisioning).
    • User defines number of the Hosts required within min, max
      defined on HostResource.
      • Bare metal hosts are assigned if applicable.
    • Hosts are created or if already existing just assigned to proper
      Hostgroup.
    • Provisioning may already start if it cannot be delayed. But the
      puppetrun has to be always delayed, it’s orchestrated later.
  • All hosts are just created waiting for provisioning or only provisioned
    without configuration triggered.
  • User triggers deployment.
    • All hosts are provisioned if required.
    • Waits until all hosts are provisioned.
    • Orchestration of the Ordered resources. e.g.:
      • Puppetrun an all controller nodes in parallel,
      • then puppetrun on all compute nodes in parallel.

Petr

On 28.11.14 9:13, Petr Chalupa wrote:

Removing any associations to existing Foreman model from stacks to allow
stack sharing without problems between users.

On 26.11.14 9:27, Petr Chalupa wrote:

Hi

Updating the model for stacks and adding data model of a deployment.

Petr

On 24.11.14 16:15, Petr Chalupa wrote:

Thanks Greg for the nice writeup. As you’ve mentioned I am working on
data model. The first part representing the Stacks (template of a
Deployment) is attached.

I’ll continue working on Deployment instance representation tomorrow.
Then I’ll send some examples of actual stack instances describing simple
apps/products.

I’ve used PlanUML [1] and Idea plugin [2].

Legend

Entities:
(A) = Abstract class
(C) = Class
(M) = Module

Arrows:
A --|> B (blank closed arrow) = represents inheritance B.is_a?(A)
A --o B (blank diamond) = Aggregation - B is composed of A

Petr

[1] http://plantuml.sourceforge.net/classes.html
[2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm

On 21.11.14 15:39, Greg Sutcliffe wrote:

Hi all,

(TL;DR - multi-host orchestration is coming, help us make it awesome!)

For a while we’ve been discussing the concept of multi-host
orchestration in Foreman, and how to achieve it. We’ve formed a
feature team to start looking into implementing it, and I’m writing
this email to give the community some insight into our planning so
far, and to invite input at this early to make sure we haven’t missed
anything. (There’s also quite a lot of lessons from Staypuft in this
document as well!)

Some terms we’re using:

  • Resource: a Foreman object (Host, Subnet, Puppetclass etc)
  • Stack: the instructions needed to assemble a set of resources for a
    specific purpose (e.g. a database stack)
  • Stacks can be nested (e.g. an app stack combines a db stack with a
    webserver stack, etc)
  • Deployment: the result of performing the instructions in a stack,
    taking in any necessasry input parameters (MyTwitter is a deployment
    of the app stack) [note: I actually favour the term Scenario here, but
    I got outvoted - anyone else prefer this? :P]

The current thinking is that we need some new first-class models in
Foreman for the Stack and Deployment objects, so that we can store
them. The Stack object is intended to be an abstract representation of
the thing to be built, the “receipe” if you will. This contains things
like the number of Hosts to build, the PuppetClasses to execute on
those hosts, and so on. Stacks can be nested, so that one can write a
simple stack for, say, a db cluster, and then use that stack for
defining a more complex object. The Stack also contains ordering,
where it makes sense, so that you can ensure a db host is built before
an app host tries to use it.

A Deployment is an instance of a Stack, where we can record the exact
hosts used, the parameters passed to the puppet classes (so we can
correctly share things like DB passwords to the right hosts, etc), and
where we can at a later time update the deployment.

For provisioning, the eventual plan is to either have the user provide
a set of prebuilt hosts (configured with appropriate authentication so
that Foreman can log in and configure them), or to have the user
specify one (or more) compute resources (or bare metal, via discovery
as staypuft does) and allow Foreman to build the necessary hosts
before proceeding to the provisioning step. Clearly this is only
needed when creating a Deployment, since a Stack is an abstract
representation.

There will of course be a UI eventually, but initial efforts will be
focussed on creating these stacks/deployments via the API, and on
defining an appropriate schema for the JSON used to create them. Any
UI efforts will likely be just for displaying what has already been
created.

To get started on this, we’ve broken it down into some user stories.
This is by no means a final list, but in rough order, we have
(indents are comments on the story):

  • A data design for stack and deployments

  • A JSON schema for creating these objects via the API

  • Ability to deploy a single-machine ‘stack’

    • Basically what Foreman already does, but allows us to test stack
      nesting, etc
    • Ensures we can deploy independant services
    • Ensures we can deploy intra-dependant services on a single box
      (ordering / puppetclasses)

  • Ability to define network configuration for a stack (single network
    only initially)

  • Ability to detect / act upon when a host has completed it’s
    configuration

    • Needed to know when to start the deployment of the next host in
      sequence
    • Staypuft uses the incoming Puppet reports, but we can probably
      improve on this

  • Ability to deploy a 3 machine stack on the single network

    • 3 is arbitrary
    • Introduces inter-depdant services between machines

  • Ability to deploy multi-network stack

    • Permits concepts such as compute/management networks, provisioning
      networks etc

  • Ability to consume existing hosts within a deployment

  • Ability to spin up hosts on a single compute resource for a
    deployment

  • Ability to spin up hosts on multiple compute resources for a
    deployment

  • Ability to consume discovered hosts via the Discovery plugin for a
    deployment

The ultimate goal is to make these Stack objects shareable, so that we
can all contribute to having well designed stacks running within our
infrastructures. We have a long way to go though.

Some of you will want to know what research we’ve done, and our
thoughts on the tools already out there - I’ll let Daniel reply with
that info. Petr is already working on the data designs, and will reply
shortly with his work so far. For myself, I’ll be picking up the task
of extending how we know when a host is “finished” which, since it
involves things like running Puppet after reboot, is more nuanced than
simply receiving the /unattended/built call from the installer.

Thanks for reading, and we want to hear your thoughts on the design!

Greg

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

20

Hi Petr,

Actually it's all one story.
It's the way we define a deployment and how foreman can use it later on as
building blocks.
whether it's a DB, storage, compute (private/public cloud) you name it.
actually every deployment object you add is capable of offering a service
and thus becomes to be a resource (in foreman language).
Foreman should know how to take advantage of this service to be used in
upper layer of the stack (or in another stack), all based on foreman multi
level administration for authorization.

maybe i'm not getting the whole picture you are showing in the model right,
i think going back to Ohad's suggested regarding a deep dive session, can
be very beneficial- at least for me :slight_smile:

Thanks,
Moran.

··· On Monday, December 1, 2014 3:00:50 PM UTC+2, Petr Chalupa wrote: > > Hello Moran, > > I've recognized 3 stories, I'll try to rephrase them. Please correct me > if I got it wrong. > > 1. Sharing a stack which defines DB with multiple deployments allowing > to get the DB API URL for sharing with other stacks in the deployment. > > 2. Adding of an object representing the DB instance which would allow > simple operations like adding DB. > > 3. Produce a configured ComputeResource for RHEV which was just > deployed. Allow to use the RHEV's ComputeResource to provision new hosts. > > Let me address them: > > 1. This should be supported out of the box with this model. There can be > a ParamResource defined in the stack which will represent a API URL. > It's value can be computed automatically with ERB snippet. The value can > be then used in other stacks in the same deployment. > > 2. That could be achieved through plugin, which would provide this > object and possibly other ordered-resources (like create-db) which could > be then used in other stacks. Although I think this is nice-to-have > feature for future. Also we should contemplate if it is a good practice, > it may be preferable to use 1. and the actions (like db creation) > implement with puppet classes (applied in the stack using the DB). > > 3. I was already thinking about that, we will need to do this in fusor. > Rhev stack needs to produce a configured ComputeResource to be able to > deploy CloudForms on it. It can be achieved by adding custom > ordered-resource which will create and configure the ComputeResource as > a last step within the RHEV's stack. > > Petr > > On 27.11.14 17:11, mgol...@redhat.com wrote: > > Hi Petr,all > > > > I'm pretty new to foreman, so please bear with me. > > after reviewing the suggested models i'm mostly missing the child -> > > parent stack/deployment connection possibility. > > > > for example: > > -i have deployed a DB - most basic deployment child in the chain (i > think) > > -now i would like to use it as a central DB for some of my other > > deployments, can you explain how it is being done in the current model? > > > > I think that this result might be achieved by exposing the some of the > > api of the installed product. > > and from where i see it, it's actually been done in foreman before - > > compute resource is good example of it. > > so, the stack would be openstack/oVirt/VMware... i would imagine the > > stack class is inherited from compute abstract class. and this stack > > should tell the story on how foreman can install it, the story should > > include a way to configure their sub services like nova, glance, > storage.... > > now, the outcome of such implantation of this stack is a compute > > resource, which have a common set of functions - run vm /stop vm ... no > > matter what provider/stack is underneath. > > So going back to the DB example, when openstack deployment is taking > > place, and a DB is needed for on of the components i can use the > > central DB which is now available as a DB_resource. with the api to > > create DB, managing users etc. and i don't really care if it's > > postgres/mysql.. i have basic functions to do it without caring too much > > on the layer underneath (if of course the stack allows it) > > > > so everything we deploy using foreman can actually become a resource > > that can be used for other deployments in foreman. > > > > Thanks, > > Moran. > > > > On Wednesday, November 26, 2014 10:27:54 AM UTC+2, Petr Chalupa wrote: > > > > Hi > > > > Updating the model for stacks and adding data model of a deployment. > > > > Petr > > > > On 24.11.14 16:15, Petr Chalupa wrote: > > > Thanks Greg for the nice writeup. As you've mentioned I am > > working on > > > data model. The first part representing the Stacks (template of a > > > Deployment) is attached. > > > > > > I'll continue working on Deployment instance representation > > tomorrow. > > > Then I'll send some examples of actual stack instances describing > > simple > > > apps/products. > > > > > > I've used PlanUML [1] and Idea plugin [2]. > > > > > > ## Legend > > > > > > Entities: > > > (A) = Abstract class > > > (C) = Class > > > (M) = Module > > > > > > Arrows: > > > A --|> B (blank closed arrow) = represents inheritance > `B.is_a?(A)` > > > A --o B (blank diamond) = Aggregation - B is composed of A > > > > > > Petr > > > > > > [1] http://plantuml.sourceforge.net/classes.html > > > > > [2] pluginhttps://plugins.jetbrains.com/plugin/7017?pr=phpStorm > > > > > > > > On 21.11.14 15:39, Greg Sutcliffe wrote: > > >> Hi all, > > >> > > >> (TL;DR - multi-host orchestration is coming, help us make it > > awesome!) > > >> > > >> For a while we've been discussing the concept of multi-host > > >> orchestration in Foreman, and how to achieve it. We've formed a > > >> feature team to start looking into implementing it, and I'm > writing > > >> this email to give the community some insight into our planning > so > > >> far, and to invite input at this early to make sure we haven't > > missed > > >> anything. (There's also quite a lot of lessons from Staypuft in > > this > > >> document as well!) > > >> > > >> Some terms we're using: > > >> - Resource: a Foreman object (Host, Subnet, Puppetclass etc) > > >> - Stack: the instructions needed to assemble a set of resources > > for a > > >> specific purpose (e.g. a database stack) > > >> - Stacks can be nested (e.g. an app stack combines a db stack > > with a > > >> webserver stack, etc) > > >> - Deployment: the result of performing the instructions in a > stack, > > >> taking in any necessasry input parameters (MyTwitter is a > > deployment > > >> of the app stack) [note: I actually favour the term Scenario > > here, but > > >> I got outvoted - anyone else prefer this? :P] > > >> > > >> The current thinking is that we need some new first-class models > in > > >> Foreman for the Stack and Deployment objects, so that we can > store > > >> them. The Stack object is intended to be an abstract > > representation of > > >> the thing to be built, the "receipe" if you will. This contains > > things > > >> like the number of Hosts to build, the PuppetClasses to execute > on > > >> those hosts, and so on. Stacks can be nested, so that one can > > write a > > >> simple stack for, say, a db cluster, and then use that stack for > > >> defining a more complex object. The Stack also contains > ordering, > > >> where it makes sense, so that you can ensure a db host is built > > before > > >> an app host tries to use it. > > >> > > >> A Deployment is an instance of a Stack, where we can record the > > exact > > >> hosts used, the parameters passed to the puppet classes (so we > can > > >> correctly share things like DB passwords to the right hosts, > > etc), and > > >> where we can at a later time update the deployment. > > >> > > >> For provisioning, the eventual plan is to either have the user > > provide > > >> a set of prebuilt hosts (configured with appropriate > > authentication so > > >> that Foreman can log in and configure them), or to have the user > > >> specify one (or more) compute resources (or bare metal, via > > discovery > > >> as staypuft does) and allow Foreman to build the necessary hosts > > >> before proceeding to the provisioning step. Clearly this is only > > >> needed when creating a Deployment, since a Stack is an abstract > > >> representation. > > >> > > >> There will of course be a UI eventually, but initial efforts > > will be > > >> focussed on creating these stacks/deployments via the API, and > on > > >> defining an appropriate schema for the JSON used to create them. > > Any > > >> UI efforts will likely be just for displaying what has already > been > > >> created. > > >> > > >> To get started on this, we've broken it down into some user > > stories. > > >> This is by no means a final list, but in *rough* order, we have > > >> (indents are comments on the story): > > >> > > >> * A data design for stack and deployments > > >> * A JSON schema for creating these objects via the API > > >> * Ability to deploy a single-machine 'stack' > > >> * Basically what Foreman already does, but allows us to test > > stack > > >> nesting, etc > > >> * Ensures we can deploy independant services > > >> * Ensures we can deploy intra-dependant services on a single > box > > >> (ordering / puppetclasses) > > >> * Ability to define network configuration for a stack (single > > network > > >> only initially) > > >> * Ability to detect / act upon when a host has completed it's > > >> configuration > > >> * Needed to know when to start the deployment of the next > > host in > > >> sequence > > >> * Staypuft uses the incoming Puppet reports, but we can > probably > > >> improve on this > > >> * Ability to deploy a 3 machine stack on the single network > > >> * 3 is arbitrary > > >> * Introduces inter-depdant services between machines > > >> * Ability to deploy multi-network stack > > >> * Permits concepts such as compute/management networks, > > provisioning > > >> networks etc > > >> > > >> * Ability to consume existing hosts within a deployment > > >> * Ability to spin up hosts on a single compute resource for a > > deployment > > >> * Ability to spin up hosts on multiple compute resources for a > > deployment > > >> * Ability to consume discovered hosts via the Discovery plugin > > for a > > >> deployment > > >> > > >> The ultimate goal is to make these Stack objects shareable, so > > that we > > >> can all contribute to having well designed stacks running within > > our > > >> infrastructures. We have a long way to go though. > > >> > > >> Some of you will want to know what research we've done, and our > > >> thoughts on the tools already out there - I'll let Daniel reply > > with > > >> that info. Petr is already working on the data designs, and will > > reply > > >> shortly with his work so far. For myself, I'll be picking up the > > task > > >> of extending how we know when a host is "finished" which, since > it > > >> involves things like running Puppet after reboot, is more > > nuanced than > > >> simply receiving the /unattended/built call from the installer. > > >> > > >> Thanks for reading, and we want to hear your thoughts on the > > design! > > >> > > >> Greg > > >> > > > > > > > -- > > You received this message because you are subscribed to the Google > > Groups "foreman-dev" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to foreman-dev...@googlegroups.com > > <mailto:foreman-dev+unsubscribe@googlegroups.com >. > > For more options, visit https://groups.google.com/d/optout. >