Integrating Image Factory with Foreman

Hello all,

I'm part of the group that develops and maintains "Image Factory" a tool
for building, modifying and uploading images to various cloud and virt
platforms.

https://github.com/redhat-imaging/imagefactory

Factory creates images from scratch by running automated native
installations within a virtual machine, either locally or more recently
in a cloud container manager such as Nova. Factory can do additional OS
image customization either as part of the automated native install (for
example, by adding packages or %pre and %post content to a kickstart
file), or by booting the installed system image into a "normal" run
level and triggering additional activities.

We're hoping to proof out an option for integrating Image Factory with
Foreman by creating a new plugin that communicates with the Factory REST
API. The end goal is to provide a mechanism for creating images that is
similar, from a workflow perspective, to the existing Foreman mechanisms
for installing individual physical and virtual machines. In particular,
we'd like to explore how we might apply additional package installs and
large scale customization to images.

We expect that creating this plugin may raise some questions
about how full system images (either locally stored or on remote
cloud/virt environments) should be represented within Foreman and
related tools. For example, should the collections of channels and
repos associated with a RHEL release now include things like a set of
system images, OVAs, AMIs, etc.

We also expect that since our end result is an image, rather than a
unique machine, that we'll have to skip or modify some steps that
finalize a machine's identity.

Dennis Kliban, another Factory developer, has been looking at creating
this plugin in a bit more detail and has some specific thoughts about
how it might be structured.

Dennis, can you chime in?

-Ian

> Hello all,
>
> I'm part of the group that develops and maintains "Image Factory" a tool
> for building, modifying and uploading images to various cloud and virt
> platforms.
>
> https://github.com/redhat-imaging/imagefactory
>
> Factory creates images from scratch by running automated native
> installations within a virtual machine, either locally or more recently
> in a cloud container manager such as Nova. Factory can do additional OS
> image customization either as part of the automated native install (for
> example, by adding packages or %pre and %post content to a kickstart
> file), or by booting the installed system image into a "normal" run
> level and triggering additional activities.
>
> We're hoping to proof out an option for integrating Image Factory with
> Foreman by creating a new plugin that communicates with the Factory REST
> API. The end goal is to provide a mechanism for creating images that is
> similar, from a workflow perspective, to the existing Foreman mechanisms
> for installing individual physical and virtual machines. In particular,
> we'd like to explore how we might apply additional package installs and
> large scale customization to images.
>
> We expect that creating this plugin may raise some questions
> about how full system images (either locally stored or on remote
> cloud/virt environments) should be represented within Foreman and
> related tools. For example, should the collections of channels and
> repos associated with a RHEL release now include things like a set of
> system images, OVAs, AMIs, etc.
>
> We also expect that since our end result is an image, rather than a
> unique machine, that we'll have to skip or modify some steps that
> finalize a machine's identity.
>
> Dennis Kliban, another Factory developer, has been looking at creating
> this plugin in a bit more detail and has some specific thoughts about
> how it might be structured.
>
> Dennis, can you chime in?

The plugin will add menu items for managing images: List images, Build image.

The view for building an image will allow the user to select a host (or hostgroup)
template to be used to dynamically generate a kickstart file for building the image.

The controller for the build image view will then pass the kickstart file to Image
Factory via REST API. Image Factory will then build an image. Images created by IF
are currently stored locally where IF is running, but are made available for download
via API.

The view for listing images will display a list of available images based on
information retrieved from Image Factory.

With such a preliminary design in mind, I have the following questions:

Is foreman currently aware of an image resource? If so, what tables/models
are associated with it?

Does foreman have an image store?

Would it make sense to allow users to use both hostgroups and host based templates to
generate kickstart files for image building?

-Dennis

··· ----- Original Message -----

-Ian


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hello

> We expect that creating this plugin may raise some questions
> about how full system images (either locally stored or on remote
> cloud/virt environments) should be represented within Foreman and

When you say "locally stored", what do you mean? Does the IF have some
kind of image registry?

> We also expect that since our end result is an image, rather than a
> unique machine, that we'll have to skip or modify some steps that
> finalize a machine's identity.

I am not sure I follow, can you elaborate this?

This is exciting news, thanks.

··· -- Later, Lukas #lzap Zapletal

Dang, I meant to reply to this days ago, apologies.

Cool stuff - looking forward to seeing where it goes. I had a stab at basic
image building a while ago for Openstack, and Libvirt, where the basic idea
was to run a provisioning step via Foreman (so create a Host, run the
templates on it etc), then SSH in and run a cleanup template, and then push
the disk back to the Compute Resource at the end, before deleting the VM.
Now what you're doing is far better than these crude efforts, but I thought
I'd post up a link in case it helps you get started with how I rendered and
applied extra templates to things for building generic images.

Feel free to ping me in Freenode ('gwmngilfen') if I can help with the
development.

Greg

> Hello
>
>> We expect that creating this plugin may raise some questions
>> about how full system images (either locally stored or on remote
>> cloud/virt environments) should be represented within Foreman and
> When you say "locally stored", what do you mean?

I'm referring to images that are stored as full formed binary blobs
accessible to factory and/or Foreman, in contrast to images that exist
on some external infrastructure or service (such as AMIs on EC2, image
templates in vSphere, RHEV-M, etc.)

> Does the IF have some
> kind of image registry?
Factory has a basic ability to store and serve up the binary
representation of images that it has created. I'm not sure this
qualifies as a "registry". Can you elaborate on what that might mean?
>
>> We also expect that since our end result is an image, rather than a
>> unique machine, that we'll have to skip or modify some steps that
>> finalize a machine's identity.
> I am not sure I follow, can you elaborate this?

Generally speaking, we'll need to avoid and/or undo anything that
generates a unique identity inside of the image, since it is by
definition something that can and will be used to generate multiple
machine instances. For example, we don't want to register the image
against satellite or use an entitlement and we'll need to remove any
uniqueness or personality that ends up on the filesystem as an indirect
result of doing customization (things like SSH host keys and MAC addresses).

··· On 09/03/2014 09:48 AM, Lukas Zapletal wrote: > > This is exciting news, thanks. >

>
>
> > Hello all,
> >
> > I'm part of the group that develops and maintains "Image Factory" a tool
> > for building, modifying and uploading images to various cloud and virt
> > platforms.
> >
> > https://github.com/redhat-imaging/imagefactory
> >
> > Factory creates images from scratch by running automated native
> > installations within a virtual machine, either locally or more recently
> > in a cloud container manager such as Nova. Factory can do additional OS
> > image customization either as part of the automated native install (for
> > example, by adding packages or %pre and %post content to a kickstart
> > file), or by booting the installed system image into a "normal" run
> > level and triggering additional activities.
> >
> > We're hoping to proof out an option for integrating Image Factory with
> > Foreman by creating a new plugin that communicates with the Factory REST
> > API. The end goal is to provide a mechanism for creating images that is
> > similar, from a workflow perspective, to the existing Foreman mechanisms
> > for installing individual physical and virtual machines. In particular,
> > we'd like to explore how we might apply additional package installs and
> > large scale customization to images.
> >
> > We expect that creating this plugin may raise some questions
> > about how full system images (either locally stored or on remote
> > cloud/virt environments) should be represented within Foreman and
> > related tools. For example, should the collections of channels and
> > repos associated with a RHEL release now include things like a set of
> > system images, OVAs, AMIs, etc.
> >
> > We also expect that since our end result is an image, rather than a
> > unique machine, that we'll have to skip or modify some steps that
> > finalize a machine's identity.
> >
> > Dennis Kliban, another Factory developer, has been looking at creating
> > this plugin in a bit more detail and has some specific thoughts about
> > how it might be structured.
> >
> > Dennis, can you chime in?
>
> The plugin will add menu items for managing images: List images, Build image.
>
> The view for building an image will allow the user to select a host (or
> hostgroup)
> template to be used to dynamically generate a kickstart file for building the
> image.
>
> The controller for the build image view will then pass the kickstart file to
> Image
> Factory via REST API. Image Factory will then build an image. Images created
> by IF
> are currently stored locally where IF is running, but are made available for
> download
> via API.
>
> The view for listing images will display a list of available images based on
> information retrieved from Image Factory.
>
> With such a preliminary design in mind, I have the following questions:
>
> Is foreman currently aware of an image resource? If so, what tables/models
> are associated with it?
>
> Does foreman have an image store?
>
> Would it make sense to allow users to use both hostgroups and host based
> templates to
> generate kickstart files for image building?
>

After playing around with foreman I have decided that a better workflow would look like the following:

Create Image Factory as a compute resource type.
When adding image to Image Factory compute resource, user selects:
Operating System
Any parameters associated with that OS
e.g. root password
The templates associated with the OS selected are used to render a kickstart file for Image Factory.

I found documentation[0] that shows how to render a template for host or hostgroup. Is there a similar api call for rendering templates for an operating system? Does this make sense?

What is still not known to me is how the images in Image Factory should be presented to other compute resources.

Is there currently support for having foreman upload an image to a compute resource? E.g. Provide URL for an image and have it uploaded to either OpenStack or EC2.

This would open up the possibility of either:

  • Importing each image in Image Factory resource individually into each resource that supports it.
  • A smart proxy moving an image from Image Factory to another compute resource when a new host is created within that resource from an image in Image Factory.

-Dennis

[0] - TemplateWriting - Foreman

··· ----- Original Message ----- > ----- Original Message -----

-Dennis

-Ian


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hey,

> Factory has a basic ability to store and serve up the binary
> representation of images that it has created. I'm not sure this
> qualifies as a "registry". Can you elaborate on what that might mean?

well in Foreman images are spread across various services as you noted.
By registry I mean a service that can hold them all. One thing to rule
them all.

This makes a lot of sense in the hybrid environment when users want to
have the ability to use same images (or at least images built from the
same source - kickstarts) on different platforms - OpenStack, EC2,
oVirt…

> Generally speaking, we'll need to avoid and/or undo anything that
> generates a unique identity inside of the image, since it is by
> definition something that can and will be used to generate multiple
> machine instances. For example, we don't want to register the image
> against satellite or use an entitlement and we'll need to remove any
> uniqueness or personality that ends up on the filesystem as an indirect
> result of doing customization (things like SSH host keys and MAC addresses).

That matches Foreman's workflow. Foreman is able to spawn an instance,
wait for an IP and then ssh to it calling what we call "finish script"
which is rendered from our templating system.

··· -- Later, Lukas #lzap Zapletal

> With such a preliminary design in mind, I have the following questions:
>
> Is foreman currently aware of an image resource? If so, what tables/models
> are associated with it?

We are aware of images in several compute resources. For example:
libvirt, oVirt/RHEV, OpenStack, EC2 that comes to my mind. The table
name is "image".

> Does foreman have an image store?

No, we only add images as references to image ids in clouds or
virtualization deployments so we are able to associate these with our
resources (Operating System, Host).

> Would it make sense to allow users to use both hostgroups and host based templates to
> generate kickstart files for image building?

It would make big sense to leverage Foreman templating for image
building, yeah. Although some changes would be required in Foreman Core
perhaps. Currently templates (Kickstats which we generally call
provision templates) renders via tokens. You can spoof them in the webui
but I don't know if this is available in the API.

··· -- Later, Lukas #lzap Zapletal

>>
>>
>>> Hello all,
>>>
>>> I'm part of the group that develops and maintains "Image Factory" a tool
>>> for building, modifying and uploading images to various cloud and virt
>>> platforms.
>>>
>>> https://github.com/redhat-imaging/imagefactory
>>>
>>> Factory creates images from scratch by running automated native
>>> installations within a virtual machine, either locally or more recently
>>> in a cloud container manager such as Nova. Factory can do additional OS
>>> image customization either as part of the automated native install (for
>>> example, by adding packages or %pre and %post content to a kickstart
>>> file), or by booting the installed system image into a "normal" run
>>> level and triggering additional activities.
>>>
>>> We're hoping to proof out an option for integrating Image Factory with
>>> Foreman by creating a new plugin that communicates with the Factory REST
>>> API. The end goal is to provide a mechanism for creating images that is
>>> similar, from a workflow perspective, to the existing Foreman mechanisms
>>> for installing individual physical and virtual machines. In particular,
>>> we'd like to explore how we might apply additional package installs and
>>> large scale customization to images.
>>>
>>> We expect that creating this plugin may raise some questions
>>> about how full system images (either locally stored or on remote
>>> cloud/virt environments) should be represented within Foreman and
>>> related tools. For example, should the collections of channels and
>>> repos associated with a RHEL release now include things like a set of
>>> system images, OVAs, AMIs, etc.
>>>
>>> We also expect that since our end result is an image, rather than a
>>> unique machine, that we'll have to skip or modify some steps that
>>> finalize a machine's identity.
>>>
>>> Dennis Kliban, another Factory developer, has been looking at creating
>>> this plugin in a bit more detail and has some specific thoughts about
>>> how it might be structured.
>>>
>>> Dennis, can you chime in?
>>
>> The plugin will add menu items for managing images: List images, Build image.
>>
>> The view for building an image will allow the user to select a host (or
>> hostgroup)
>> template to be used to dynamically generate a kickstart file for building the
>> image.
>>
>> The controller for the build image view will then pass the kickstart file to
>> Image
>> Factory via REST API. Image Factory will then build an image. Images created
>> by IF
>> are currently stored locally where IF is running, but are made available for
>> download
>> via API.
>>
>> The view for listing images will display a list of available images based on
>> information retrieved from Image Factory.
>>
>> With such a preliminary design in mind, I have the following questions:
>>
>> Is foreman currently aware of an image resource? If so, what tables/models
>> are associated with it?

Amos and Daniel were playing with docker, perhaps there is a pattern
there for how to repsent containers and images? I had assumed images
would be similar to hosts, so that eventually we can do some sort of
drift detection against them

>>
>> Does foreman have an image store?
>>
>> Would it make sense to allow users to use both hostgroups and host based
>> templates to
>> generate kickstart files for image building?
>>
>
> After playing around with foreman I have decided that a better workflow would look like the following:
>
> Create Image Factory as a compute resource type.
> When adding image to Image Factory compute resource, user selects:
> Operating System
> Any parameters associated with that OS
> e.g. root password
> The templates associated with the OS selected are used to render a kickstart file for Image Factory.
>
> I found documentation[0] that shows how to render a template for host or hostgroup. Is there a similar api call for rendering templates for an operating system? Does this make sense?
>
> What is still not known to me is how the images in Image Factory should be presented to other compute resources.
>
> Is there currently support for having foreman upload an image to a compute resource? E.g. Provide URL for an image and have it uploaded to either OpenStack or EC2.
>
> This would open up the possibility of either:
> - Importing each image in Image Factory resource individually into each resource that supports it.
> - A smart proxy moving an image from Image Factory to another compute resource when a new host is created within that resource from an image in Image Factory.

I would be curious how pulp and/or glance could play here. Perhaps the
compute resource is configured to stick them into a certain place?

– bk

··· On 09/04/2014 02:36 AM, Dennis Kliban wrote: > ----- Original Message ----- >> ----- Original Message -----

> Create Image Factory as a compute resource type.
> When adding image to Image Factory compute resource, user selects:
> Operating System
> Any parameters associated with that OS
> e.g. root password
> The templates associated with the OS selected are used to render a kickstart file for Image Factory.

Sounds good, what is the planned workflow? I guess once the image is
built, user is presented with an option to upload it to another Compute
Resource (Foreman could hand over credentials) and also create the image
resource for the Compute Resource.

> I found documentation[0] that shows how to render a template for host or hostgroup. Is there a similar api call for rendering templates for an operating system? Does this make sense?

It does make sense in this case, but Foreman's provision template type
expect @host variable to be present. This type is host-centric.

I guess we'd need to introduce new type of template that would not have
such a constraints.

> What is still not known to me is how the images in Image Factory should be presented to other compute resources.

Why don't you offer the possibility to automatically upload them and
explicitly create them? See above.

> Is there currently support for having foreman upload an image to a compute resource? E.g. Provide URL for an image and have it uploaded to either OpenStack or EC2.

We don't have that. It's a good fit for Factory to do this.

> This would open up the possibility of either:
> - Importing each image in Image Factory resource individually into each resource that supports it.
> - A smart proxy moving an image from Image Factory to another compute resource when a new host is created within that resource from an image in Image Factory.

Whatever can be done on the smart proxy side, do it there. It has plugin
architecture as well and we tend to move more and more business logic
there. Because it's the smart-proxy instance what resides in the network
and has all the access. Foreman web application is usually limited
(although it currently access Compute Resources directly, our long-term
goal is to move this to Smart Proxies).

··· -- Later, Lukas #lzap Zapletal

>
>
>
>>
>>>
>>>
>>>
>>>> Hello all,
>>>>
>>>> I'm part of the group that develops and maintains "Image Factory" a tool
>>>> for building, modifying and uploading images to various cloud and virt
>>>> platforms.
>>>>
>>>> https://github.com/redhat-imaging/imagefactory
>>>>
>>>> Factory creates images from scratch by running automated native
>>>> installations within a virtual machine, either locally or more recently
>>>> in a cloud container manager such as Nova. Factory can do additional OS
>>>> image customization either as part of the automated native install (for
>>>> example, by adding packages or %pre and %post content to a kickstart
>>>> file), or by booting the installed system image into a "normal" run
>>>> level and triggering additional activities.
>>>>
>>>> We're hoping to proof out an option for integrating Image Factory with
>>>> Foreman by creating a new plugin that communicates with the Factory REST
>>>> API. The end goal is to provide a mechanism for creating images that is
>>>> similar, from a workflow perspective, to the existing Foreman mechanisms
>>>> for installing individual physical and virtual machines. In particular,
>>>> we'd like to explore how we might apply additional package installs and
>>>> large scale customization to images.
>>>>
>>>> We expect that creating this plugin may raise some questions
>>>> about how full system images (either locally stored or on remote
>>>> cloud/virt environments) should be represented within Foreman and
>>>> related tools. For example, should the collections of channels and
>>>> repos associated with a RHEL release now include things like a set of
>>>> system images, OVAs, AMIs, etc.
>>>>
>>>> We also expect that since our end result is an image, rather than a
>>>> unique machine, that we'll have to skip or modify some steps that
>>>> finalize a machine's identity.
>>>>
>>>> Dennis Kliban, another Factory developer, has been looking at creating
>>>> this plugin in a bit more detail and has some specific thoughts about
>>>> how it might be structured.
>>>>
>>>> Dennis, can you chime in?
>>>>
>>>
>>> The plugin will add menu items for managing images: List images, Build
>>> image.
>>>
>>> The view for building an image will allow the user to select a host (or
>>> hostgroup)
>>> template to be used to dynamically generate a kickstart file for
>>> building the
>>> image.
>>>
>>> The controller for the build image view will then pass the kickstart
>>> file to
>>> Image
>>> Factory via REST API. Image Factory will then build an image. Images
>>> created
>>> by IF
>>> are currently stored locally where IF is running, but are made available
>>> for
>>> download
>>> via API.
>>>
>>> The view for listing images will display a list of available images
>>> based on
>>> information retrieved from Image Factory.
>>>
>>> With such a preliminary design in mind, I have the following questions:
>>>
>>> Is foreman currently aware of an image resource? If so, what
>>> tables/models
>>> are associated with it?
>>>
>>
> Amos and Daniel were playing with docker, perhaps there is a pattern there
> for how to repsent containers and images? I had assumed images would be
> similar to hosts, so that eventually we can do some sort of drift detection
> against them

ATM mostly Compute Resource => something that can generate VMS, I wonder if
we need more kind of objects like CR to generate stuff? disks/images is
one, network is another, potentially higher level infrastructure such as
database etc?

··· On Thu, Sep 4, 2014 at 3:36 PM, Bryan Kearney wrote: > On 09/04/2014 02:36 AM, Dennis Kliban wrote: >> ----- Original Message ----- >>> ----- Original Message -----

Does foreman have an image store?

Would it make sense to allow users to use both hostgroups and host based
templates to
generate kickstart files for image building?

After playing around with foreman I have decided that a better workflow
would look like the following:

Create Image Factory as a compute resource type.
When adding image to Image Factory compute resource, user selects:
Operating System
Any parameters associated with that OS
e.g. root password
The templates associated with the OS selected are used to render a
kickstart file for Image Factory.

I found documentation[0] that shows how to render a template for host or
hostgroup. Is there a similar api call for rendering templates for an
operating system? Does this make sense?

What is still not known to me is how the images in Image Factory should
be presented to other compute resources.

Is there currently support for having foreman upload an image to a
compute resource? E.g. Provide URL for an image and have it uploaded to
either OpenStack or EC2.

This would open up the possibility of either:

  • Importing each image in Image Factory resource individually into
    each resource that supports it.
  • A smart proxy moving an image from Image Factory to another compute
    resource when a new host is created within that resource from an image in
    Image Factory.

I would be curious how pulp and/or glance could play here. Perhaps the
compute resource is configured to stick them into a certain place?

– bk


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I think it'd be nice to have a separate object, I was thinking of adding
some new class in foreman-docker to handle image building/management but
consuming it from core would make things easier for all plugins (bootdisk,
docker, other CRs…) and some Compute Resources.
Maybe whatever I end up creating for docker images and registries can later
on be moved to core if you guys think its useful and makes sense.

··· On Thu, Sep 4, 2014 at 2:52 PM, Ohad Levy wrote:

On Thu, Sep 4, 2014 at 3:36 PM, Bryan Kearney bryan.kearney@gmail.com > wrote:

On 09/04/2014 02:36 AM, Dennis Kliban wrote:

----- Original Message -----

----- Original Message -----

Hello all,

I’m part of the group that develops and maintains “Image Factory” a
tool
for building, modifying and uploading images to various cloud and virt
platforms.

https://github.com/redhat-imaging/imagefactory

Factory creates images from scratch by running automated native
installations within a virtual machine, either locally or more recently
in a cloud container manager such as Nova. Factory can do additional
OS
image customization either as part of the automated native install (for
example, by adding packages or %pre and %post content to a kickstart
file), or by booting the installed system image into a “normal” run
level and triggering additional activities.

We’re hoping to proof out an option for integrating Image Factory with
Foreman by creating a new plugin that communicates with the Factory
REST
API. The end goal is to provide a mechanism for creating images that
is
similar, from a workflow perspective, to the existing Foreman
mechanisms
for installing individual physical and virtual machines. In
particular,
we’d like to explore how we might apply additional package installs and
large scale customization to images.

We expect that creating this plugin may raise some questions
about how full system images (either locally stored or on remote
cloud/virt environments) should be represented within Foreman and
related tools. For example, should the collections of channels and
repos associated with a RHEL release now include things like a set of
system images, OVAs, AMIs, etc.

We also expect that since our end result is an image, rather than a
unique machine, that we’ll have to skip or modify some steps that
finalize a machine’s identity.

Dennis Kliban, another Factory developer, has been looking at creating
this plugin in a bit more detail and has some specific thoughts about
how it might be structured.

Dennis, can you chime in?

The plugin will add menu items for managing images: List images, Build
image.

The view for building an image will allow the user to select a host (or
hostgroup)
template to be used to dynamically generate a kickstart file for
building the
image.

The controller for the build image view will then pass the kickstart
file to
Image
Factory via REST API. Image Factory will then build an image. Images
created
by IF
are currently stored locally where IF is running, but are made
available for
download
via API.

The view for listing images will display a list of available images
based on
information retrieved from Image Factory.

With such a preliminary design in mind, I have the following questions:

Is foreman currently aware of an image resource? If so, what
tables/models
are associated with it?

Amos and Daniel were playing with docker, perhaps there is a pattern
there for how to repsent containers and images? I had assumed images would
be similar to hosts, so that eventually we can do some sort of drift
detection against them

ATM mostly Compute Resource => something that can generate VMS, I wonder
if we need more kind of objects like CR to generate stuff? disks/images is
one, network is another, potentially higher level infrastructure such as
database etc?

Does foreman have an image store?

Would it make sense to allow users to use both hostgroups and host based
templates to
generate kickstart files for image building?

After playing around with foreman I have decided that a better workflow
would look like the following:

Create Image Factory as a compute resource type.
When adding image to Image Factory compute resource, user selects:
Operating System
Any parameters associated with that OS
e.g. root password
The templates associated with the OS selected are used to render a
kickstart file for Image Factory.

I found documentation[0] that shows how to render a template for host or
hostgroup. Is there a similar api call for rendering templates for an
operating system? Does this make sense?

What is still not known to me is how the images in Image Factory should
be presented to other compute resources.

Is there currently support for having foreman upload an image to a
compute resource? E.g. Provide URL for an image and have it uploaded to
either OpenStack or EC2.

This would open up the possibility of either:

  • Importing each image in Image Factory resource individually into
    each resource that supports it.
  • A smart proxy moving an image from Image Factory to another compute
    resource when a new host is created within that resource from an image in
    Image Factory.

I would be curious how pulp and/or glance could play here. Perhaps the
compute resource is configured to stick them into a certain place?

– bk


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Daniel Lobato

@elobatoss
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30

I've forked foreman on Github [0] and have been able to enable the following functionality:

-Image Factory compute resource type
-Kicking off an Image Factory MockOS image build by adding an image to Image Factory compute resource

The code for kicking off the build had to go into a before_validation callback of an Image model [1].
This new callback has a special case for images being saved for Image Factory compute resource type.
However, Image model is still missing a status field.

Would it be alright to add a status field to Image? What about some more callbacks to handle ImageFactory
images?

Otherwise I would like to create a separate model, FactoryImage, and a FactoryImageController.
FactoryImage would have an additional field called 'Status'. FactoryImage would have an after_initialize
callback that would query Image Factory if the status was not BUILT or ERROR and update the status accordingly.

Where in the code is the UnattendedController used? Would it make sense to extend it to handle ImageProvisioning
TemplateKind? This type of template would basically be a provisioning template without a host field. I'd like to be able
to call UnattendedController in a before_validation callback for either Image or FactoryImage. The rendered
ImageProvisioning template would then be passed to ImageFactory.

-Dennis

[0] https://github.com/dkliban/foreman
[1] https://github.com/dkliban/foreman/commit/29924a64e4d04e96d7a8794dee59874d2ba06a68

··· ----- Original Message ----- > I think it'd be nice to have a separate object, I was thinking of adding > some new class in foreman-docker to handle image building/management but > consuming it from core would make things easier for all plugins (bootdisk, > docker, other CRs..) and some Compute Resources. > Maybe whatever I end up creating for docker images and registries can later > on be moved to core if you guys think its useful and makes sense. > > > On Thu, Sep 4, 2014 at 2:52 PM, Ohad Levy wrote: > > > > > > > > > On Thu, Sep 4, 2014 at 3:36 PM, Bryan Kearney > > wrote: > > > >> > >> > >> On 09/04/2014 02:36 AM, Dennis Kliban wrote: > >> > >>> ----- Original Message ----- > >>> > >>>> > >>>> > >>>> ----- Original Message ----- > >>>> > >>>>> Hello all, > >>>>> > >>>>> I'm part of the group that develops and maintains "Image Factory" a > >>>>> tool > >>>>> for building, modifying and uploading images to various cloud and virt > >>>>> platforms. > >>>>> > >>>>> https://github.com/redhat-imaging/imagefactory > >>>>> > >>>>> Factory creates images from scratch by running automated native > >>>>> installations within a virtual machine, either locally or more recently > >>>>> in a cloud container manager such as Nova. Factory can do additional > >>>>> OS > >>>>> image customization either as part of the automated native install (for > >>>>> example, by adding packages or %pre and %post content to a kickstart > >>>>> file), or by booting the installed system image into a "normal" run > >>>>> level and triggering additional activities. > >>>>> > >>>>> We're hoping to proof out an option for integrating Image Factory with > >>>>> Foreman by creating a new plugin that communicates with the Factory > >>>>> REST > >>>>> API. The end goal is to provide a mechanism for creating images that > >>>>> is > >>>>> similar, from a workflow perspective, to the existing Foreman > >>>>> mechanisms > >>>>> for installing individual physical and virtual machines. In > >>>>> particular, > >>>>> we'd like to explore how we might apply additional package installs and > >>>>> large scale customization to images. > >>>>> > >>>>> We expect that creating this plugin may raise some questions > >>>>> about how full system images (either locally stored or on remote > >>>>> cloud/virt environments) should be represented within Foreman and > >>>>> related tools. For example, should the collections of channels and > >>>>> repos associated with a RHEL release now include things like a set of > >>>>> system images, OVAs, AMIs, etc. > >>>>> > >>>>> We also expect that since our end result is an image, rather than a > >>>>> unique machine, that we'll have to skip or modify some steps that > >>>>> finalize a machine's identity. > >>>>> > >>>>> Dennis Kliban, another Factory developer, has been looking at creating > >>>>> this plugin in a bit more detail and has some specific thoughts about > >>>>> how it might be structured. > >>>>> > >>>>> Dennis, can you chime in? > >>>>> > >>>> > >>>> The plugin will add menu items for managing images: List images, Build > >>>> image. > >>>> > >>>> The view for building an image will allow the user to select a host (or > >>>> hostgroup) > >>>> template to be used to dynamically generate a kickstart file for > >>>> building the > >>>> image. > >>>> > >>>> The controller for the build image view will then pass the kickstart > >>>> file to > >>>> Image > >>>> Factory via REST API. Image Factory will then build an image. Images > >>>> created > >>>> by IF > >>>> are currently stored locally where IF is running, but are made > >>>> available for > >>>> download > >>>> via API. > >>>> > >>>> The view for listing images will display a list of available images > >>>> based on > >>>> information retrieved from Image Factory. > >>>> > >>>> With such a preliminary design in mind, I have the following questions: > >>>> > >>>> Is foreman currently aware of an image resource? If so, what > >>>> tables/models > >>>> are associated with it? > >>>> > >>> > >> Amos and Daniel were playing with docker, perhaps there is a pattern > >> there for how to repsent containers and images? I had assumed images would > >> be similar to hosts, so that eventually we can do some sort of drift > >> detection against them > > > > > > ATM mostly Compute Resource => something that can generate VMS, I wonder > > if we need more kind of objects like CR to generate stuff? disks/images is > > one, network is another, potentially higher level infrastructure such as > > database etc? > > > > > > > >> > >> > >> > >>>> Does foreman have an image store? > >>>> > >>>> Would it make sense to allow users to use both hostgroups and host based > >>>> templates to > >>>> generate kickstart files for image building? > >>>> > >>>> > >>> After playing around with foreman I have decided that a better workflow > >>> would look like the following: > >>> > >>> Create Image Factory as a compute resource type. > >>> When adding image to Image Factory compute resource, user selects: > >>> Operating System > >>> Any parameters associated with that OS > >>> e.g. root password > >>> The templates associated with the OS selected are used to render a > >>> kickstart file for Image Factory. > >>> > >>> I found documentation[0] that shows how to render a template for host or > >>> hostgroup. Is there a similar api call for rendering templates for an > >>> operating system? Does this make sense? > >>> > >>> What is still not known to me is how the images in Image Factory should > >>> be presented to other compute resources. > >>> > >>> Is there currently support for having foreman upload an image to a > >>> compute resource? E.g. Provide URL for an image and have it uploaded to > >>> either OpenStack or EC2. > >>> > >>> This would open up the possibility of either: > >>> - Importing each image in Image Factory resource individually into > >>> each resource that supports it. > >>> - A smart proxy moving an image from Image Factory to another compute > >>> resource when a new host is created within that resource from an image in > >>> Image Factory. > >>> > >> > >> I would be curious how pulp and/or glance could play here. Perhaps the > >> compute resource is configured to stick them into a certain place? > >> > >> -- bk > >> > >> > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "foreman-dev" group. > >> To unsubscribe from this group and stop receiving emails from it, send an > >> email to foreman-dev+unsubscribe@googlegroups.com. > >> For more options, visit https://groups.google.com/d/optout. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "foreman-dev" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to foreman-dev+unsubscribe@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > Daniel Lobato > > @elobatoss > blog.daniellobato.me > daniellobato.me > > GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >

While talking about having Image Factory as compute resource,
is there a plan for Image Factory to run as a service, ideally
having some mechanism of workers and loadbalancing. I expect
the image building to be quite demanding (especially when running
more builds in parallel, or it's a wrong assumption?).

Will the image factory require to run on the same server as the Foreman?

– Ivan

··· ----- Original Message ----- > I've forked foreman on Github [0] and have been able to enable the following > functionality: > > -Image Factory compute resource type > -Kicking off an Image Factory MockOS image build by adding an image to Image > Factory compute resource > > The code for kicking off the build had to go into a before_validation > callback of an Image model [1]. > This new callback has a special case for images being saved for Image Factory > compute resource type. > However, Image model is still missing a status field. > > Would it be alright to add a status field to Image? What about some more > callbacks to handle ImageFactory > images? > > Otherwise I would like to create a separate model, FactoryImage, and a > FactoryImageController. > FactoryImage would have an additional field called 'Status'. FactoryImage > would have an after_initialize > callback that would query Image Factory if the status was not BUILT or ERROR > and update the status accordingly. > > Where in the code is the UnattendedController used? Would it make sense to > extend it to handle ImageProvisioning > TemplateKind? This type of template would basically be a provisioning > template without a host field. I'd like to be able > to call UnattendedController in a before_validation callback for either Image > or FactoryImage. The rendered > ImageProvisioning template would then be passed to ImageFactory. > > -Dennis > > [0] https://github.com/dkliban/foreman > [1] > https://github.com/dkliban/foreman/commit/29924a64e4d04e96d7a8794dee59874d2ba06a68 > > ----- Original Message ----- > > I think it'd be nice to have a separate object, I was thinking of adding > > some new class in foreman-docker to handle image building/management but > > consuming it from core would make things easier for all plugins (bootdisk, > > docker, other CRs..) and some Compute Resources. > > Maybe whatever I end up creating for docker images and registries can later > > on be moved to core if you guys think its useful and makes sense. > > > > > > On Thu, Sep 4, 2014 at 2:52 PM, Ohad Levy wrote: > > > > > > > > > > > > > > On Thu, Sep 4, 2014 at 3:36 PM, Bryan Kearney > > > wrote: > > > > > >> > > >> > > >> On 09/04/2014 02:36 AM, Dennis Kliban wrote: > > >> > > >>> ----- Original Message ----- > > >>> > > >>>> > > >>>> > > >>>> ----- Original Message ----- > > >>>> > > >>>>> Hello all, > > >>>>> > > >>>>> I'm part of the group that develops and maintains "Image Factory" a > > >>>>> tool > > >>>>> for building, modifying and uploading images to various cloud and > > >>>>> virt > > >>>>> platforms. > > >>>>> > > >>>>> https://github.com/redhat-imaging/imagefactory > > >>>>> > > >>>>> Factory creates images from scratch by running automated native > > >>>>> installations within a virtual machine, either locally or more > > >>>>> recently > > >>>>> in a cloud container manager such as Nova. Factory can do additional > > >>>>> OS > > >>>>> image customization either as part of the automated native install > > >>>>> (for > > >>>>> example, by adding packages or %pre and %post content to a kickstart > > >>>>> file), or by booting the installed system image into a "normal" run > > >>>>> level and triggering additional activities. > > >>>>> > > >>>>> We're hoping to proof out an option for integrating Image Factory > > >>>>> with > > >>>>> Foreman by creating a new plugin that communicates with the Factory > > >>>>> REST > > >>>>> API. The end goal is to provide a mechanism for creating images that > > >>>>> is > > >>>>> similar, from a workflow perspective, to the existing Foreman > > >>>>> mechanisms > > >>>>> for installing individual physical and virtual machines. In > > >>>>> particular, > > >>>>> we'd like to explore how we might apply additional package installs > > >>>>> and > > >>>>> large scale customization to images. > > >>>>> > > >>>>> We expect that creating this plugin may raise some questions > > >>>>> about how full system images (either locally stored or on remote > > >>>>> cloud/virt environments) should be represented within Foreman and > > >>>>> related tools. For example, should the collections of channels and > > >>>>> repos associated with a RHEL release now include things like a set of > > >>>>> system images, OVAs, AMIs, etc. > > >>>>> > > >>>>> We also expect that since our end result is an image, rather than a > > >>>>> unique machine, that we'll have to skip or modify some steps that > > >>>>> finalize a machine's identity. > > >>>>> > > >>>>> Dennis Kliban, another Factory developer, has been looking at > > >>>>> creating > > >>>>> this plugin in a bit more detail and has some specific thoughts about > > >>>>> how it might be structured. > > >>>>> > > >>>>> Dennis, can you chime in? > > >>>>> > > >>>> > > >>>> The plugin will add menu items for managing images: List images, Build > > >>>> image. > > >>>> > > >>>> The view for building an image will allow the user to select a host > > >>>> (or > > >>>> hostgroup) > > >>>> template to be used to dynamically generate a kickstart file for > > >>>> building the > > >>>> image. > > >>>> > > >>>> The controller for the build image view will then pass the kickstart > > >>>> file to > > >>>> Image > > >>>> Factory via REST API. Image Factory will then build an image. Images > > >>>> created > > >>>> by IF > > >>>> are currently stored locally where IF is running, but are made > > >>>> available for > > >>>> download > > >>>> via API. > > >>>> > > >>>> The view for listing images will display a list of available images > > >>>> based on > > >>>> information retrieved from Image Factory. > > >>>> > > >>>> With such a preliminary design in mind, I have the following > > >>>> questions: > > >>>> > > >>>> Is foreman currently aware of an image resource? If so, what > > >>>> tables/models > > >>>> are associated with it? > > >>>> > > >>> > > >> Amos and Daniel were playing with docker, perhaps there is a pattern > > >> there for how to repsent containers and images? I had assumed images > > >> would > > >> be similar to hosts, so that eventually we can do some sort of drift > > >> detection against them > > > > > > > > > ATM mostly Compute Resource => something that can generate VMS, I wonder > > > if we need more kind of objects like CR to generate stuff? disks/images > > > is > > > one, network is another, potentially higher level infrastructure such as > > > database etc? > > > > > > > > > > > >> > > >> > > >> > > >>>> Does foreman have an image store? > > >>>> > > >>>> Would it make sense to allow users to use both hostgroups and host > > >>>> based > > >>>> templates to > > >>>> generate kickstart files for image building? > > >>>> > > >>>> > > >>> After playing around with foreman I have decided that a better workflow > > >>> would look like the following: > > >>> > > >>> Create Image Factory as a compute resource type. > > >>> When adding image to Image Factory compute resource, user selects: > > >>> Operating System > > >>> Any parameters associated with that OS > > >>> e.g. root password > > >>> The templates associated with the OS selected are used to render a > > >>> kickstart file for Image Factory. > > >>> > > >>> I found documentation[0] that shows how to render a template for host > > >>> or > > >>> hostgroup. Is there a similar api call for rendering templates for an > > >>> operating system? Does this make sense? > > >>> > > >>> What is still not known to me is how the images in Image Factory should > > >>> be presented to other compute resources. > > >>> > > >>> Is there currently support for having foreman upload an image to a > > >>> compute resource? E.g. Provide URL for an image and have it uploaded > > >>> to > > >>> either OpenStack or EC2. > > >>> > > >>> This would open up the possibility of either: > > >>> - Importing each image in Image Factory resource individually into > > >>> each resource that supports it. > > >>> - A smart proxy moving an image from Image Factory to another > > >>> compute > > >>> resource when a new host is created within that resource from an image > > >>> in > > >>> Image Factory. > > >>> > > >> > > >> I would be curious how pulp and/or glance could play here. Perhaps the > > >> compute resource is configured to stick them into a certain place? > > >> > > >> -- bk > > >> > > >> > > >> -- > > >> You received this message because you are subscribed to the Google > > >> Groups > > >> "foreman-dev" group. > > >> To unsubscribe from this group and stop receiving emails from it, send > > >> an > > >> email to foreman-dev+unsubscribe@googlegroups.com. > > >> For more options, visit https://groups.google.com/d/optout. > > >> > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "foreman-dev" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to foreman-dev+unsubscribe@googlegroups.com. > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > -- > > Daniel Lobato > > > > @elobatoss > > blog.daniellobato.me > > daniellobato.me > > > > GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 > > > > -- > > You received this message because you are subscribed to the Google Groups > > "foreman-dev" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to foreman-dev+unsubscribe@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >

> While talking about having Image Factory as compute resource,
> is there a plan for Image Factory to run as a service, ideally
> having some mechanism of workers and loadbalancing. I expect
> the image building to be quite demanding (especially when running
> more builds in parallel, or it's a wrong assumption?).

Image Factory has a REST api. Right now it is run as a single node.

>
> Will the image factory require to run on the same server as the Foreman?

Image Factory will run on a different server. It will operate the same as
other compute resource types do now.

··· ----- Original Message -----

– Ivan

----- Original Message -----

I’ve forked foreman on Github [0] and have been able to enable the
following
functionality:

-Image Factory compute resource type
-Kicking off an Image Factory MockOS image build by adding an image to
Image
Factory compute resource

The code for kicking off the build had to go into a before_validation
callback of an Image model [1].
This new callback has a special case for images being saved for Image
Factory
compute resource type.
However, Image model is still missing a status field.

Would it be alright to add a status field to Image? What about some more
callbacks to handle ImageFactory
images?

Otherwise I would like to create a separate model, FactoryImage, and a
FactoryImageController.
FactoryImage would have an additional field called ‘Status’. FactoryImage
would have an after_initialize
callback that would query Image Factory if the status was not BUILT or
ERROR
and update the status accordingly.

Where in the code is the UnattendedController used? Would it make sense to
extend it to handle ImageProvisioning
TemplateKind? This type of template would basically be a provisioning
template without a host field. I’d like to be able
to call UnattendedController in a before_validation callback for either
Image
or FactoryImage. The rendered
ImageProvisioning template would then be passed to ImageFactory.

-Dennis

[0] https://github.com/dkliban/foreman
[1]
https://github.com/dkliban/foreman/commit/29924a64e4d04e96d7a8794dee59874d2ba06a68

----- Original Message -----

I think it’d be nice to have a separate object, I was thinking of adding
some new class in foreman-docker to handle image building/management but
consuming it from core would make things easier for all plugins
(bootdisk,
docker, other CRs…) and some Compute Resources.
Maybe whatever I end up creating for docker images and registries can
later
on be moved to core if you guys think its useful and makes sense.

On Thu, Sep 4, 2014 at 2:52 PM, Ohad Levy ohadlevy@gmail.com wrote:

On Thu, Sep 4, 2014 at 3:36 PM, Bryan Kearney bryan.kearney@gmail.com > > > > wrote:

On 09/04/2014 02:36 AM, Dennis Kliban wrote:

----- Original Message -----

----- Original Message -----

Hello all,

I’m part of the group that develops and maintains “Image Factory” a
tool
for building, modifying and uploading images to various cloud and
virt
platforms.

https://github.com/redhat-imaging/imagefactory

Factory creates images from scratch by running automated native
installations within a virtual machine, either locally or more
recently
in a cloud container manager such as Nova. Factory can do
additional
OS
image customization either as part of the automated native install
(for
example, by adding packages or %pre and %post content to a
kickstart
file), or by booting the installed system image into a “normal” run
level and triggering additional activities.

We’re hoping to proof out an option for integrating Image Factory
with
Foreman by creating a new plugin that communicates with the Factory
REST
API. The end goal is to provide a mechanism for creating images
that
is
similar, from a workflow perspective, to the existing Foreman
mechanisms
for installing individual physical and virtual machines. In
particular,
we’d like to explore how we might apply additional package installs
and
large scale customization to images.

We expect that creating this plugin may raise some questions
about how full system images (either locally stored or on remote
cloud/virt environments) should be represented within Foreman and
related tools. For example, should the collections of channels and
repos associated with a RHEL release now include things like a set
of
system images, OVAs, AMIs, etc.

We also expect that since our end result is an image, rather than a
unique machine, that we’ll have to skip or modify some steps that
finalize a machine’s identity.

Dennis Kliban, another Factory developer, has been looking at
creating
this plugin in a bit more detail and has some specific thoughts
about
how it might be structured.

Dennis, can you chime in?

The plugin will add menu items for managing images: List images,
Build
image.

The view for building an image will allow the user to select a host
(or
hostgroup)
template to be used to dynamically generate a kickstart file for
building the
image.

The controller for the build image view will then pass the kickstart
file to
Image
Factory via REST API. Image Factory will then build an image.
Images
created
by IF
are currently stored locally where IF is running, but are made
available for
download
via API.

The view for listing images will display a list of available images
based on
information retrieved from Image Factory.

With such a preliminary design in mind, I have the following
questions:

Is foreman currently aware of an image resource? If so, what
tables/models
are associated with it?

Amos and Daniel were playing with docker, perhaps there is a pattern
there for how to repsent containers and images? I had assumed images
would
be similar to hosts, so that eventually we can do some sort of drift
detection against them

ATM mostly Compute Resource => something that can generate VMS, I
wonder
if we need more kind of objects like CR to generate stuff? disks/images
is
one, network is another, potentially higher level infrastructure such
as
database etc?

Does foreman have an image store?

Would it make sense to allow users to use both hostgroups and host
based
templates to
generate kickstart files for image building?

After playing around with foreman I have decided that a better
workflow
would look like the following:

Create Image Factory as a compute resource type.
When adding image to Image Factory compute resource, user selects:
Operating System
Any parameters associated with that OS
e.g. root password
The templates associated with the OS selected are used to render a
kickstart file for Image Factory.

I found documentation[0] that shows how to render a template for host
or
hostgroup. Is there a similar api call for rendering templates for an
operating system? Does this make sense?

What is still not known to me is how the images in Image Factory
should
be presented to other compute resources.

Is there currently support for having foreman upload an image to a
compute resource? E.g. Provide URL for an image and have it uploaded
to
either OpenStack or EC2.

This would open up the possibility of either:

  • Importing each image in Image Factory resource individually into
    each resource that supports it.
  • A smart proxy moving an image from Image Factory to another
    compute
    resource when a new host is created within that resource from an
    image
    in
    Image Factory.

I would be curious how pulp and/or glance could play here. Perhaps the
compute resource is configured to stick them into a certain place?

– bk


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Daniel Lobato

@elobatoss
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

> > While talking about having Image Factory as compute resource,
> > is there a plan for Image Factory to run as a service, ideally
> > having some mechanism of workers and loadbalancing. I expect
> > the image building to be quite demanding (especially when running
> > more builds in parallel, or it's a wrong assumption?).
>
> Image Factory has a REST api. Right now it is run as a single node.

Ah, I missed that. When looking at https://github.com/redhat-imaging/imagefactory#using-image-factory,
the sudo imagefactory --template f12_64.tdl --target ec2 made me think
it works just as a CLI (I would not expect needed sudo for a REST API service).

Where could I find the documentation for the REST API and how to get the service
configured? Is there some sort of queuing mechanism involved?

Also, I'm interested into the docker build support. Any docs on that?

– Ivan

··· ----- Original Message ----- > ----- Original Message -----

Will the image factory require to run on the same server as the Foreman?

Image Factory will run on a different server. It will operate the same as
other compute resource types do now.

– Ivan

----- Original Message -----

I’ve forked foreman on Github [0] and have been able to enable the
following
functionality:

-Image Factory compute resource type
-Kicking off an Image Factory MockOS image build by adding an image to
Image
Factory compute resource

The code for kicking off the build had to go into a before_validation
callback of an Image model [1].
This new callback has a special case for images being saved for Image
Factory
compute resource type.
However, Image model is still missing a status field.

Would it be alright to add a status field to Image? What about some more
callbacks to handle ImageFactory
images?

Otherwise I would like to create a separate model, FactoryImage, and a
FactoryImageController.
FactoryImage would have an additional field called ‘Status’.
FactoryImage
would have an after_initialize
callback that would query Image Factory if the status was not BUILT or
ERROR
and update the status accordingly.

Where in the code is the UnattendedController used? Would it make sense
to
extend it to handle ImageProvisioning
TemplateKind? This type of template would basically be a provisioning
template without a host field. I’d like to be able
to call UnattendedController in a before_validation callback for either
Image
or FactoryImage. The rendered
ImageProvisioning template would then be passed to ImageFactory.

-Dennis

[0] https://github.com/dkliban/foreman
[1]
https://github.com/dkliban/foreman/commit/29924a64e4d04e96d7a8794dee59874d2ba06a68

----- Original Message -----

I think it’d be nice to have a separate object, I was thinking of
adding
some new class in foreman-docker to handle image building/management
but
consuming it from core would make things easier for all plugins
(bootdisk,
docker, other CRs…) and some Compute Resources.
Maybe whatever I end up creating for docker images and registries can
later
on be moved to core if you guys think its useful and makes sense.

On Thu, Sep 4, 2014 at 2:52 PM, Ohad Levy ohadlevy@gmail.com wrote:

On Thu, Sep 4, 2014 at 3:36 PM, Bryan Kearney > > > > > bryan.kearney@gmail.com > > > > > wrote:

On 09/04/2014 02:36 AM, Dennis Kliban wrote:

----- Original Message -----

----- Original Message -----

Hello all,

I’m part of the group that develops and maintains "Image Factory"
a
tool
for building, modifying and uploading images to various cloud and
virt
platforms.

https://github.com/redhat-imaging/imagefactory

Factory creates images from scratch by running automated native
installations within a virtual machine, either locally or more
recently
in a cloud container manager such as Nova. Factory can do
additional
OS
image customization either as part of the automated native
install
(for
example, by adding packages or %pre and %post content to a
kickstart
file), or by booting the installed system image into a "normal"
run
level and triggering additional activities.

We’re hoping to proof out an option for integrating Image Factory
with
Foreman by creating a new plugin that communicates with the
Factory
REST
API. The end goal is to provide a mechanism for creating images
that
is
similar, from a workflow perspective, to the existing Foreman
mechanisms
for installing individual physical and virtual machines. In
particular,
we’d like to explore how we might apply additional package
installs
and
large scale customization to images.

We expect that creating this plugin may raise some questions
about how full system images (either locally stored or on remote
cloud/virt environments) should be represented within Foreman and
related tools. For example, should the collections of channels
and
repos associated with a RHEL release now include things like a
set
of
system images, OVAs, AMIs, etc.

We also expect that since our end result is an image, rather than
a
unique machine, that we’ll have to skip or modify some steps that
finalize a machine’s identity.

Dennis Kliban, another Factory developer, has been looking at
creating
this plugin in a bit more detail and has some specific thoughts
about
how it might be structured.

Dennis, can you chime in?

The plugin will add menu items for managing images: List images,
Build
image.

The view for building an image will allow the user to select a
host
(or
hostgroup)
template to be used to dynamically generate a kickstart file for
building the
image.

The controller for the build image view will then pass the
kickstart
file to
Image
Factory via REST API. Image Factory will then build an image.
Images
created
by IF
are currently stored locally where IF is running, but are made
available for
download
via API.

The view for listing images will display a list of available
images
based on
information retrieved from Image Factory.

With such a preliminary design in mind, I have the following
questions:

Is foreman currently aware of an image resource? If so, what
tables/models
are associated with it?

Amos and Daniel were playing with docker, perhaps there is a pattern
there for how to repsent containers and images? I had assumed images
would
be similar to hosts, so that eventually we can do some sort of drift
detection against them

ATM mostly Compute Resource => something that can generate VMS, I
wonder
if we need more kind of objects like CR to generate stuff?
disks/images
is
one, network is another, potentially higher level infrastructure such
as
database etc?

Does foreman have an image store?

Would it make sense to allow users to use both hostgroups and host
based
templates to
generate kickstart files for image building?

After playing around with foreman I have decided that a better
workflow
would look like the following:

Create Image Factory as a compute resource type.
When adding image to Image Factory compute resource, user
selects:
Operating System
Any parameters associated with that OS
e.g. root password
The templates associated with the OS selected are used to render
a
kickstart file for Image Factory.

I found documentation[0] that shows how to render a template for
host
or
hostgroup. Is there a similar api call for rendering templates for
an
operating system? Does this make sense?

What is still not known to me is how the images in Image Factory
should
be presented to other compute resources.

Is there currently support for having foreman upload an image to a
compute resource? E.g. Provide URL for an image and have it
uploaded
to
either OpenStack or EC2.

This would open up the possibility of either:

  • Importing each image in Image Factory resource individually
    into
    each resource that supports it.
  • A smart proxy moving an image from Image Factory to another
    compute
    resource when a new host is created within that resource from an
    image
    in
    Image Factory.

I would be curious how pulp and/or glance could play here. Perhaps
the
compute resource is configured to stick them into a certain place?

– bk


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it,
send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it,
send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Daniel Lobato

@elobatoss
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.