Can I provision a system using a non-primary interface?

Support
1

My nodes have two Ethernet ports on the back, and I would like to use the
ports this way:

  • Use the first port (NIC1, which defaults to eth0 in Linux) as the
    permanent network connection connected to the public network. I will not
    put DHCP server on the public network.
  • Use the secondary NIC (NIC2, e.g. eth1) as a temporary network
    connection which is only used during the provisioning. This network has the
    DHCP server, TFTP server, etc. When I'm done with provisioning, I remove
    the cable.

This is a pretty common configuration at many sites.

Foreman and Puppet can speak on both networks.

I've noticed that Foreman will only create files under
/var/lib/tftpboot/pxelinux.cfg/00-AA-BB-CC-DD-EE-FF for the Primary
Interface, and only if it matches a known subnet. Foreman will not create
any files for the secondary interface.

Can I use this secondary interface for provisioning?

If I do use the Primary Interface for provisioning, I noticed that the
network settings for the Primary Interface will get changed as soon as
Puppet updates the facts, and I believe I can disable this behavior using
ignore_puppet_facts_for_provisioning, per [the Wiki: Foreman configuration<Foreman configuration - Foreman>.

Thank you,

-= Stefan

2

> My nodes have two Ethernet ports on the back, and I would like to use
> the ports this way:
>
> * Use the first port (NIC1, which defaults to eth0 in Linux) as the
> permanent network connection connected to the public network. I will
> not put DHCP server on the public network.
> * Use the secondary NIC (NIC2, e.g. eth1) as a temporary network
> connection which is only used during the provisioning. This network
> has the DHCP server, TFTP server, etc. When I'm done with
> provisioning, I remove the cable.
>
> This is a pretty common configuration at many sites.
>
> Foreman and Puppet can speak on both networks.
>
> I've noticed that Foreman will only create files under
> /var/lib/tftpboot/pxelinux.cfg/00-AA-BB-CC-DD-EE-FF for the Primary
> Interface, and only if it matches a known subnet. Foreman will not
> create any files for the secondary interface.
>
> Can I use this secondary interface for provisioning?

In theory it could be added, but currently the non-primary NICs only get
DHCP and DNS orchestration - no TFTP. I wonder if we'd need it
configurable or not? I'd suggest adding a feature request.

> If I do use the Primary Interface for provisioning, I noticed that the
> network settings for the Primary Interface will get changed as soon as
> Puppet updates the facts, and I believe I can disable this behavior
> using ignore_puppet_facts_for_provisioning, per [the Wiki: Foreman
> configuration
> <Foreman configuration - Foreman> .

This issue's been filed (from another bug report) to change the default
to true, which might be more useful than the current default:
Bug #3864: Change ignore_puppet_facts_for_provisioning default to true - Foreman

··· On 14/12/13 00:13, Stefan Lasiewski wrote:

–
Dominic Cleal
Red Hat Engineering

3

>
> > My nodes have two Ethernet ports on the back, and I would like to use
> > the ports this way:
> >
> > * Use the first port (NIC1, which defaults to eth0 in Linux) as the
> > permanent network connection connected to the public network. I will
> > not put DHCP server on the public network.
> > * Use the secondary NIC (NIC2, e.g. eth1) as a temporary network
> > connection which is only used during the provisioning. This network
> > has the DHCP server, TFTP server, etc. When I'm done with
> > provisioning, I remove the cable.
> >
> > This is a pretty common configuration at many sites.
> >
> > Foreman and Puppet can speak on both networks.
> >
> > I've noticed that Foreman will only create files under
> > /var/lib/tftpboot/pxelinux.cfg/00-AA-BB-CC-DD-EE-FF for the Primary
> > Interface, and only if it matches a known subnet. Foreman will not
> > create any files for the secondary interface.
> >
> > Can I use this secondary interface for provisioning?
>
> In theory it could be added, but currently the non-primary NICs only get
> DHCP and DNS orchestration - no TFTP. I wonder if we'd need it
> configurable or not? I'd suggest adding a feature request.
>

Looks like someone beat me to it: Feature #3554: Enable provisioning on
non-primary interface <Feature #3554: Enable provisioning on non-primary interface - Foreman>

> > If I do use the Primary Interface for provisioning, I noticed that the
> > network settings for the Primary Interface will get changed as soon as
> > Puppet updates the facts, and I believe I can disable this behavior
> > using ignore_puppet_facts_for_provisioning, per [the Wiki: Foreman
> > configuration
> > <Foreman configuration - Foreman>
> .
>
> This issue's been filed (from another bug report) to change the default
> to true, which might be more useful than the current default:
> Bug #3864: Change ignore_puppet_facts_for_provisioning default to true - Foreman
>

Thanks again Dominic.

Another side effect of this: If the DNS proxy is active, and I name a host
like 'web01' with the private IP address of 172.16.100.100 then that
hostname is assigned to the temporary address on the private, provisioning
network.

If I create a second interface on the public network, and try to assign the
hostname 'web01.example.org' to that interface, I run into a DNS conflict
like:

Conflicts have been detected
>
> The following entries were found conflicting with what foreman wanted to
> apply.
>
> Please review them carefully, if you are certain that they should be
> removed, please click on overwrite.
> DNS A Records web01/172.16.100.100 already exists
>

As a result, I either need to deactivate the DNS proxy, rename my nodes
within foreman or chose a temporary and inaccurate name for the interface.

-= Stefan

··· On Monday, December 16, 2013 1:05:03 AM UTC-8, Dominic Cleal wrote: > On 14/12/13 00:13, Stefan Lasiewski wrote:
4

Hi folks,

Now that Foreman 1.7 is release, I thought I would check in regarding this
feature. Can I provision bare-metal systems using the secondary interface,
not using the primary interface?

I see that Feature #3554: Enable provisioning on non-primary interface - Foreman and it's dependent
bugs have been changed, and I see other 1.7 notes that look substantial and
possibly what I need. But the Foreman :: Manual
manual is a bit vague on what is possible, and the Template Writing wiki
<TemplateWriting - Foreman>
still has the notes that I added last year.

-= Stefan

··· On Monday, December 16, 2013 3:43:34 PM UTC-8, Stefan Lasiewski wrote: > > On Monday, December 16, 2013 1:05:03 AM UTC-8, Dominic Cleal wrote: >> >> On 14/12/13 00:13, Stefan Lasiewski wrote: >> > My nodes have two Ethernet ports on the back, and I would like to use >> > the ports this way: >> > >> > * Use the first port (NIC1, which defaults to eth0 in Linux) as the >> > permanent network connection connected to the public network. I >> will >> > not put DHCP server on the public network. >> > * Use the secondary NIC (NIC2, e.g. eth1) as a temporary network >> > connection which is only used during the provisioning. This network >> > has the DHCP server, TFTP server, etc. When I'm done with >> > provisioning, I remove the cable. >> > >> > This is a pretty common configuration at many sites. >> > >> > Foreman and Puppet can speak on both networks. >> > >> > I've noticed that Foreman will only create files under >> > `/var/lib/tftpboot/pxelinux.cfg/00-AA-BB-CC-DD-EE-FF` for the Primary >> > Interface, and only if it matches a known subnet. Foreman will not >> > create any files for the secondary interface. >> > >> > Can I use this secondary interface for provisioning? >> >> In theory it could be added, but currently the non-primary NICs only get >> DHCP and DNS orchestration - no TFTP. I wonder if we'd need it >> configurable or not? I'd suggest adding a feature request. >> > > Looks like someone beat me to it: Feature #3554: Enable provisioning on > non-primary interface > > >> > If I do use the Primary Interface for provisioning, I noticed that the >> > network settings for the Primary Interface will get changed as soon as >> > Puppet updates the facts, and I believe I can disable this behavior >> > using `ignore_puppet_facts_for_provisioning`, per [the Wiki: Foreman >> > configuration >> > >> . >> >> This issue's been filed (from another bug report) to change the default >> to true, which might be more useful than the current default: >> http://projects.theforeman.org/issues/3864 >> > > > Thanks again Dominic. > > Another side effect of this: If the DNS proxy is active, and I name a host > like 'web01' with the private IP address of 172.16.100.100 then that > hostname is assigned to the temporary address on the private, provisioning > network. > > If I create a second interface on the public network, and try to assign > the hostname 'web01.example.org' to that interface, I run into a DNS > conflict like: > > Conflicts have been detected >> >> The following entries were found conflicting with what foreman wanted to >> apply. >> >> Please review them carefully, if you are certain that they should be >> removed, please click on overwrite. >> DNS A Records web01/172.16.100.100 already exists >> > > > As a result, I either need to deactivate the DNS proxy, rename my nodes > within foreman or chose a temporary and inaccurate name for the interface. > > -= Stefan > >
5

Sadly not, it wasn't ready in time. This week I demo'd the current
state of the network refactoring. Take a look at
https://www.youtube.com/watch?v=xmYmMQONq_0 and feel free to leave
feedback on the discussion thread at
https://groups.google.com/forum/#!topic/foreman-dev/JC3B35d8xcY. We
really want user feedback on this to check we're going the right way.

Thanks,
Greg

··· On 4 December 2014 at 20:18, Stefan Lasiewski wrote: > Hi folks, > > Now that Foreman 1.7 is release, I thought I would check in regarding this > feature. Can I provision bare-metal systems using the secondary interface, > not using the primary interface?
6

We worked around the issue and got a fairly good process now.

We assign the MAC address of the 2nd NIC (eth1) in the MAC address
parameter and no IP address.That takes care of the PXE boot stuff embedded
into Foreman.

I then create host parameters with the MAC address and IP details of the
primary IP address (eth0). Then use the host parameters in the finish
script and rewrite the ifcfg-ethX interface files based on our cabling and
patching standards.

I also turn off update values from puppet runs after the server is built
otherwise it will attempt to rebuild on the primary network.

Works well.

Puppet manages drift on the network stuff after the host has been built.

I have just managed to get this system up and going with Windows 2008 R2.

We will relook at this when the feature has been release but so far happy
sitting on version 1.6.3 for a bit.

Cheers

Chris

··· On Friday, December 5, 2014 9:31:53 AM UTC+11, Greg Sutcliffe wrote: > > On 4 December 2014 at 20:18, Stefan Lasiewski > wrote: > > Hi folks, > > > > Now that Foreman 1.7 is release, I thought I would check in regarding > this > > feature. Can I provision bare-metal systems using the secondary > interface, > > not using the primary interface? > > Sadly not, it wasn't ready in time. This week I demo'd the current > state of the network refactoring. Take a look at > https://www.youtube.com/watch?v=xmYmMQONq_0 and feel free to leave > feedback on the discussion thread at > https://groups.google.com/forum/#!topic/foreman-dev/JC3B35d8xcY. We > really want user feedback on this to check we're going the right way. > > Thanks, > Greg >
7

Hi Chris,
I'm facing the very same issue, and, though I know it will be addressed in
future Foreman releases, I would like to start ASAP with a workaround like
yours.
Would you kindly share some more details about your solution, specially
concerning finish script and and turning off update values from puppet.
Many thanks in advance
federico

··· Il giorno lunedì 8 dicembre 2014 01:44:28 UTC+1, Chris Gibbs ha scritto: > > We worked around the issue and got a fairly good process now. > > We assign the MAC address of the 2nd NIC (eth1) in the MAC address > parameter and no IP address.That takes care of the PXE boot stuff embedded > into Foreman. > > I then create host parameters with the MAC address and IP details of the > primary IP address (eth0). Then use the host parameters in the finish > script and rewrite the ifcfg-ethX interface files based on our cabling and > patching standards. > > I also turn off update values from puppet runs after the server is built > otherwise it will attempt to rebuild on the primary network. > > Works well. > > Puppet manages drift on the network stuff after the host has been built. > > I have just managed to get this system up and going with Windows 2008 R2. > > We will relook at this when the feature has been release but so far happy > sitting on version 1.6.3 for a bit. > > Cheers > > Chris > > > On Friday, December 5, 2014 9:31:53 AM UTC+11, Greg Sutcliffe wrote: >> >> On 4 December 2014 at 20:18, Stefan Lasiewski >> wrote: >> > Hi folks, >> > >> > Now that Foreman 1.7 is release, I thought I would check in regarding >> this >> > feature. Can I provision bare-metal systems using the secondary >> interface, >> > not using the primary interface? >> >> Sadly not, it wasn't ready in time. This week I demo'd the current >> state of the network refactoring. Take a look at >> https://www.youtube.com/watch?v=xmYmMQONq_0 and feel free to leave >> feedback on the discussion thread at >> https://groups.google.com/forum/#!topic/foreman-dev/JC3B35d8xcY. We >> really want user feedback on this to check we're going the right way. >> >> Thanks, >> Greg >> >
8

Hello,

this should be possible since 1.8 where you select which interface is primary
and which is provisioning.

Hope this helps

··· On Friday 26 of June 2015 08:37:16 Federico Nebiolo wrote: > Il giorno lunedì 8 dicembre 2014 01:44:28 UTC+1, Chris Gibbs ha scritto: > > We worked around the issue and got a fairly good process now. > > > > We assign the MAC address of the 2nd NIC (eth1) in the MAC address > > parameter and no IP address.That takes care of the PXE boot stuff embedded > > into Foreman. > > > > I then create host parameters with the MAC address and IP details of the > > primary IP address (eth0). Then use the host parameters in the finish > > script and rewrite the ifcfg-ethX interface files based on our cabling and > > patching standards. > > > > I also turn off update values from puppet runs after the server is built > > otherwise it will attempt to rebuild on the primary network. > > > > Works well. > > > > Puppet manages drift on the network stuff after the host has been built. > > > > I have just managed to get this system up and going with Windows 2008 R2. > > > > We will relook at this when the feature has been release but so far happy > > sitting on version 1.6.3 for a bit. > > > > Cheers > > > > Chris > > > > On Friday, December 5, 2014 9:31:53 AM UTC+11, Greg Sutcliffe wrote: > >> On 4 December 2014 at 20:18, Stefan Lasiewski > >> > >> wrote: > >> > Hi folks, > >> > > >> > Now that Foreman 1.7 is release, I thought I would check in regarding > >> > >> this > >> > >> > feature. Can I provision bare-metal systems using the secondary > >> > >> interface, > >> > >> > not using the primary interface? > >> > >> Sadly not, it wasn't ready in time. This week I demo'd the current > >> state of the network refactoring. Take a look at > >> https://www.youtube.com/watch?v=xmYmMQONq_0 and feel free to leave > >> feedback on the discussion thread at > >> https://groups.google.com/forum/#!topic/foreman-dev/JC3B35d8xcY. We > >> really want user feedback on this to check we're going the right way. > >> > >> Thanks, > >> Greg > > Hi Chris, > I'm facing the very same issue, and, though I know it will be addressed in > future Foreman releases, I would like to start ASAP with a workaround like > yours. > Would you kindly share some more details about your solution, specially > concerning finish script and and turning off update values from puppet. > Many thanks in advance > federico

–
Marek