Foreman discovery question?

Development
1

> Hello Lukas

Hello, please use our mailing list which I am CCing.

> I came across your foreman-discovery slides which was very informative & an
> interesting read, thank you.
>
> Would it be possible for you to give me some advice please?
>
> We are switching to use the discovery image as part of our buils but it
> does not exactly meet our use case.
>
> We need to ensure we can pass boot parameters so that we can configure
> hardware raid devices & we also need to ensure that when hosts are added to
> foreman, that it happens seamlessly without needing someone to select
> "build"

For the boot parameters, you can set whatever you want in the "Default
PXELinux" template. If you need to add RAID-related kernel command line
options, you can just add them. I am not sure if this is what you are
asking about.

Provisioning without interaction is our short-term goal, expect this
kind of feature pretty soon.

We have a new discovery image which is currently in alpha stage, feel
free to test it out:

https://lzap.fedorapeople.org/temp/foreman-discovery-2.0pre1/

··· -- Later, Lukas #lzap Zapletal
2

Hi Lukas

The raid controller on the node is external so I have to use the binary
provided by our vendor to configure it, I was hoping to do this during
discovery stage by slotting in the binary somewhere during the discovery
process, reconfiguring the RAID controller & letting foreman-discovery do
its thing?

Can you please advise? ( hope I've made it more clearer)

Regards

Frank

··· On Wednesday, 5 November 2014 13:26:38 UTC, Lukas Zapletal wrote: > > > Hello Lukas > > Hello, please use our mailing list which I am CCing. > > > > I came across your foreman-discovery slides which was very informative & > an > > interesting read, thank you. > > > > Would it be possible for you to give me some advice please? > > > > We are switching to use the discovery image as part of our buils but it > > does not exactly meet our use case. > > > > We need to ensure we can pass boot parameters so that we can configure > > hardware raid devices & we also need to ensure that when hosts are added > to > > foreman, that it happens seamlessly without needing someone to select > > "build" > > For the boot parameters, you can set whatever you want in the "Default > PXELinux" template. If you need to add RAID-related kernel command line > options, you can just add them. I am not sure if this is what you are > asking about. > > Provisioning without interaction is our short-term goal, expect this > kind of feature pretty soon. > > We have a new discovery image which is currently in alpha stage, feel > free to test it out: > >
3

> The raid controller on the node is external so I have to use the binary
> provided by our vendor to configure it, I was hoping to do this during
> discovery stage by slotting in the binary somewhere during the discovery
> process, reconfiguring the RAID controller & letting foreman-discovery do
> its thing?

Hello,

discovery image 0.6 does not support that at all.

For discovery 2.0 (currently pre1) there are plans to provide
possibility to download custom "plugins". The initial goal for these
plugins are custom facts, but we can extend this a little bit to your
use case.

The plugin could be either RPM package, or just a ZIP file with some
kind of manifest. Once you provide the plugin via kernel command line,
discovery would install/extract it and execute postscriplet (or in case
of simple ZIP some kind of shell script).

I am not sure when we get there, are you able to give the new image a
shot, build your own one, try it and if it works, let's discuss how you
could implement this yourself. I could write the discovery part
(download ZIP, extract it, run a script) and you could prepare your own
plugin.

··· -- Later, Lukas #lzap Zapletal
4

>
> The plugin could be either RPM package, or just a ZIP file with some
> kind of manifest. Once you provide the plugin via kernel command line,
> discovery would install/extract it and execute postscriplet (or in case
> of simple ZIP some kind of shell script).
>
> I am not sure when we get there, are you able to give the new image a
> shot, build your own one, try it and if it works, let's discuss how you
> could implement this yourself. I could write the discovery part
> (download ZIP, extract it, run a script) and you could prepare your own
> plugin.

>
Hi Lukas

Yes, would really like to help & resolve this, I'm still trying to figure
out how foreman-discovery works, Once I've clocked the understanding it
should be a doddle!

I downloaded the image you sent me, Ive extracted the ramdisk, but cant see
any ruby scripts which take care of updating foreman with the facts &
registration?

Does it pull the ruby discovery scripts from the foreman server?

5

> I downloaded the image you sent me, Ive extracted the ramdisk, but cant see
> any ruby scripts which take care of updating foreman with the facts &
> registration?
>
> Does it pull the ruby discovery scripts from the foreman server?

Hey,

all the files are here:

https://github.com/theforeman/foreman-discovery-image/tree/master/root

This is the script that collects all the facts:

https://github.com/theforeman/foreman-discovery-image/blob/master/root/usr/bin/discover-host

If you want to provide own facts, just put them into
/usr/share/fdi/facts (this dir is in the FACTERLIB)

https://github.com/theforeman/foreman-discovery-image/tree/master/root/usr/share/fdi/facts

The image is pretty much standard CentOS7 minimal installation,
investigate the root/ directory to see what we install. In addition to
that, we install foreman-proxy which is running on port 8443 and have
BMC configured to do restarts.

··· -- Later, Lukas #lzap Zapletal
6

I'm still missing something here from my understanding. The files you
mentioned are nowhere to be seen on the image you sent me, do they live in
the squashfs.img located on the node which is then extracted?

Im trying to find information on how the files you mentioned end up on the
discovery image?

Can you please explain that part?

Regards

··· On Friday, 7 November 2014 08:34:12 UTC, Lukas Zapletal wrote: > > > I downloaded the image you sent me, Ive extracted the ramdisk, but cant > see > > any ruby scripts which take care of updating foreman with the facts & > > registration? > > Hi Lukaz
7

The files are located in the OS when you boot your node.

https://lzap.fedorapeople.org/temp/foreman-discovery-2.0pre1/

Follow the README instructions how to add a kernel parameter so you can
set root password and enable ssh daemon:

Then:

ls /usr/share/fdi/facts

for example.

LZ

··· On Fri, Nov 07, 2014 at 01:06:53AM -0800, Frank wrote: > > > On Friday, 7 November 2014 08:34:12 UTC, Lukas Zapletal wrote: > > > > > I downloaded the image you sent me, Ive extracted the ramdisk, but cant > > see > > > any ruby scripts which take care of updating foreman with the facts & > > > registration? > > > > Hi Lukaz > > I'm still missing something here from my understanding. The files you > mentioned are nowhere to be seen on the image you sent me, do they live in > the squashfs.img located on the node which is then extracted? > > Im trying to find information on how the files you mentioned end up on the > discovery image? > > Can you please explain that part? > > Regards > >

–
Later,
Lukas #lzap Zapletal

8

The files are located in the OS when you boot your node.

https://lzap.fedorapeople.org/temp/foreman-discovery-2.0pre1/

Follow the README instructions how to add a kernel parameter so you can
set root password and enable ssh daemon:

Hi Lukaz

You state the files are there, they obviously are as I can see them when i
login to the discovery_image over ssh.

But if i look at the image fdi image you gave me & open the initrd ramdisk
(without booting), i do not see any files, I cannot even find any users. (
apart from root,nobody)

But booting the image tells me a different story?

Can you please some further insight into why that is?

··· On 7 November 2014 09:26, Lukas Zapletal wrote:

The files are located in the OS when you boot your node.

https://lzap.fedorapeople.org/temp/foreman-discovery-2.0pre1/

Follow the README instructions how to add a kernel parameter so you can
set root password and enable ssh daemon:

GitHub - theforeman/foreman-discovery-image: Foreman discovery image live distro

Then:

ls /usr/share/fdi/facts

for example.

LZ

On Fri, Nov 07, 2014 at 01:06:53AM -0800, Frank wrote:

On Friday, 7 November 2014 08:34:12 UTC, Lukas Zapletal wrote:

I downloaded the image you sent me, Ive extracted the ramdisk, but
cant
see
any ruby scripts which take care of updating foreman with the facts &
registration?

Hi Lukaz

I’m still missing something here from my understanding. The files you
mentioned are nowhere to be seen on the image you sent me, do they live
in
the squashfs.img located on the node which is then extracted?

Im trying to find information on how the files you mentioned end up on
the
discovery image?

Can you please explain that part?

Regards

–
Later,
Lukas #lzap Zapletal

9

> You state the files are there, they obviously are as I can see them when i
> login to the discovery_image over ssh.

Pastebin me the commands you are doing on the image, the result and
expectations. Also pastebin me output of:

discovery-debug

··· -- Later, Lukas #lzap Zapletal
10

Hi Lukas

I have expanded the initrd0.img from the image you sent me & have looked
for any filename with the string discovery, which returns nothing…

http://pastebin.com/uhtH4i17

I have then logged into the fdi image after booting which produces
different results.

http://pastebin.com/d5yB5jWi

I would like to know the logic behind whats going on, I also noticed that
the following loop devices are mounted & used…

[root@fdi /]# losetup
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
/dev/loop0 0 0 0 0 /fdi.iso (deleted)
/dev/loop1 0 0 0 1 /osmin.img (deleted)
/dev/loop2 0 0 0 1 /osmin
/dev/loop3 0 0 0 1
/run/initramfs/live/LiveOS/squashfs.img
/dev/loop4 0 0 0 1 /LiveOS/ext3fs.img
/dev/loop5 0 0 0 0 /overlay (deleted)
/dev/loop6 0 0 1 0 /dev/dm-0

So I'm assuming something is also happening here?

Please advise?

Thanks

Frank

··· On 10 November 2014 08:36, Lukas Zapletal wrote:

You state the files are there, they obviously are as I can see them when
i
login to the discovery_image over ssh.

Pastebin me the commands you are doing on the image, the result and
expectations. Also pastebin me output of:

discovery-debug

–
Later,
Lukas #lzap Zapletal

–
You received this message because you are subscribed to a topic in the
Google Groups “foreman-dev” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-dev/RYYojIj1xco/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

11

Hi Lukas

Your request for me to paste commands has had a enlightening effect on me
as ive now read up on Live images & understand the way it now works -->
squashfs --> ext3fs.img, etc

So we can now jump this thread forward by three or four spaces as ive now
caught up…:slight_smile: thank you…!

Moving on to what i stated in the beginning, i want to exec the following

  1. Provide a kernel option to allow an external raid binary ( to be
    included in the live image) to execute or not
  2. I want to have the image pause & request a hostname, this again would be
    a kernel option ( option would decide if images pauses & asks for hostname
    or provides the hostname on kernel cmd line)
  3. auto provision the node into foreman & allow the host to be dynamically
    added to a hostgroup ( bypassing Discovered hosts completely)

Point one is simple i assume as I need to provide repo for the external
binary to be installed, should I use /etc/rc.d/rc.local to execute?
Point two, again we could do this in the same file as point one ( rc.local)
Point three, can you please provide some insight into the best way to
achieve this.

Many thanks for your time on this

Frank

··· On 10 November 2014 15:57, J Imran wrote:

Hi Lukas

I have expanded the initrd0.img from the image you sent me & have looked
for any filename with the string discovery, which returns nothing…

http://pastebin.com/uhtH4i17

I have then logged into the fdi image after booting which produces
different results.

http://pastebin.com/d5yB5jWi

I would like to know the logic behind whats going on, I also noticed that
the following loop devices are mounted & used…

[root@fdi /]# losetup
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
/dev/loop0 0 0 0 0 /fdi.iso (deleted)
/dev/loop1 0 0 0 1 /osmin.img (deleted)
/dev/loop2 0 0 0 1 /osmin
/dev/loop3 0 0 0 1
/run/initramfs/live/LiveOS/squashfs.img
/dev/loop4 0 0 0 1 /LiveOS/ext3fs.img
/dev/loop5 0 0 0 0 /overlay (deleted)
/dev/loop6 0 0 1 0 /dev/dm-0

So I’m assuming something is also happening here?

Please advise?

Thanks

Frank

On 10 November 2014 08:36, Lukas Zapletal lzap@redhat.com wrote:

You state the files are there, they obviously are as I can see them
when i
login to the discovery_image over ssh.

Pastebin me the commands you are doing on the image, the result and
expectations. Also pastebin me output of:

discovery-debug

–
Later,
Lukas #lzap Zapletal

–
You received this message because you are subscribed to a topic in the
Google Groups “foreman-dev” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-dev/RYYojIj1xco/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

12

Hey,

glad you solved it out.

> Moving on to what i stated in the beginning, i want to exec the following

Please subscribe to this feature request:

http://projects.theforeman.org/issues/6407

We can carry on our discussion there. Note the presentation link, I'd
like to follow that pattern. The best would be to have the Razor
extensions compatible, but this is not really the goal here. I was
thinking the very similar design.

> 1) Provide a kernel option to allow an external raid binary ( to be
> included in the live image) to execute or not

My vision is to have a ZIP file that would be downloaded by the image,
extracted and executed (a shell script). In this script, you can load a
kernel module (assuming that we only support RHEL/CentOS 7 based images
on x86-64) and do what you need.

> 2) I want to have the image pause & request a hostname, this again would be
> a kernel option ( option would decide if images pauses & asks for hostname
> or provides the hostname on kernel cmd line)

You can of course parse kernel command line options and do your very own
commands.

> 3) auto provision the node into foreman & allow the host to be dynamically
> added to a hostgroup ( bypassing Discovered hosts completely)

This is a work in progress this sprint, actually I have a fully working
prototype I will be showing on the demo a week after the next one.

My priority for the image is to wait until reboot issue is merged
(hopefully today) https://github.com/theforeman/smart-proxy/pull/235 and
release 2.0 image rc1 which I hope will be the last RC.

After 2.0 is released, we can start working on the extensions which will
likely land in the 2.1 release. Of course you can start working now
already, feel free to raise a pull request with the support for this in
the image.

··· -- Later, Lukas #lzap Zapletal
13

How would you deal with the (thankfully nowadays less common) case where
you need to provide additional network drivers? You can't download
those.

··· On Tue, Nov 11, 2014 at 03:49:27PM +0100, Lukas Zapletal wrote: > > 1) Provide a kernel option to allow an external raid binary ( to be > > included in the live image) to execute or not > > My vision is to have a ZIP file that would be downloaded by the image, > extracted and executed (a shell script). In this script, you can load a > kernel module (assuming that we only support RHEL/CentOS 7 based images > on x86-64) and do what you need.
14

> > My vision is to have a ZIP file that would be downloaded by the image,
> > extracted and executed (a shell script). In this script, you can load a
> > kernel module (assuming that we only support RHEL/CentOS 7 based images
> > on x86-64) and do what you need.
>
> How would you deal with the (thankfully nowadays less common) case where
> you need to provide additional network drivers? You can't download
> those.

Good point, in that case we can extend the build scripts to include
extensions in the image directly. Made the note, I will implement that
as well.

··· -- Later, Lukas #lzap Zapletal