One of the problems I find myself struggling with frequently with Foreman
is the choice to have the discovered MAC address as the unique identifier
for a system. I'm going to lay out the issues I've found recently, and
perhaps others can chime in on how they dealt with it.
Servers often times have many NICs, our standard platform is the Dell
PowerEdge M610, M620, and M630. Each of these systems has 6 NICs each.
The "Deployment Interface" is actually only used for deployment, and after
that, one of the other NICs takes over. This creates an extra (manual)
step during provisioning to work out which MAC we should be tracking.
Dell blade server enclosures have a concept called FlexAddressing where
the MAC address is actually a virtual MAC provisioned by the blade. This
is an option that can be turned off of course, but by default it is on.
The purpose of Flex Addresses is to be able to swap out a blade and that
blade then takes on the MAC address of the slot in the enclosure. This can
cause mayhem when moving from a Flex Address configured enclosure to a
non-Flex one. I'm not sure if other vendor's blade enclosures have these
same feature sets.
We have a project to upgrade all the NICs from 1GbE to 10GbE in about 50
servers, which will mean 50 new MAC addresses on the provisioning
interface, and 50 new MAC addresses on the "production" interface. I
haven't even begun to scope the changes that will be necessary for Foreman.
Virtual Machines, especially in the VMware universe seem to like
changing MAC addresses for just about any change imaginable. Yes, we can
usually propagate the old MAC address, but this is a manual process and has
to be part of the "manual steps" run-book.
We are starting to get into the realm of booting on a virtual interface
that is a bonded interface. Because Bonded interfaces are virtual, it is
up to the bonding driver developer to decide how that MAC will be
generated, and to my knowledge there is not standard on how to do so. I've
seen variations from "lowest MAC" to "lowest PCI Slot" to "make it up as we
go along" depending on the OS. What's more, the rules can change causing
mayhem later on.
This list is by no means exhaustive, in fact I stopped at 5 just to show
the point that MAC address of a NIC is a very transitory ID to latch onto.
So, why was MAC address chosen for the unique identifier? If we're
looking for something that permanently follows a system, Why not the BIOS
UUID or Board Serial Number? Is there any hope for including an option to
track by something other than MAC address?
I can offer insight why BIOS UUID is problematic. I have 400 Super micro servers and all of them have the bios UUID of 0123456789. This is somewhat common for supermicro systems as in ovirt community others had issues with ovirt nodes due to this generic UUID value being the same between systems.
I think one reason MAC can be a useful identifier is Foreman can manage DHCP and so ensuring your not adding multiple overlapping dhcp entries can be useful.
#1 is something I faced and I decided the best route was to manage all interfaces in Puppet and define their information in hiera. I'm going to move to using the foreman 1.7 interface ENC data for this but haven't come up with method yet.
When you say "the MAC is the identifier for the host", what action are
you referring to? The only one I can think of is for the TFTP records
for puppet reports it's matched on certname/FQDN (certname wins if
supplied and different from the FQDN); for retrieving provisioning
templates, the token is the default way to identify the host (although
MAC is a fallback, it's true); and the API can use the ID or the FQDN
of the host.
For TFTP records, you're right that 1.8 will help - and it's worth
knowing that we are considering allowing the "provisioning flag" to be
assigned to multiple interfaces, which will help with bonds and so on.
Also, TFTP can only use MAC or IP as an identifier (as far as I
recall, it' looks for 01-<mac> followed by the IP in hex), so we're
limited there.
···
On 23 March 2015 at 14:58, lawre wrote:
> One of the problems I find myself struggling with frequently with Foreman is
> the choice to have the discovered MAC address as the unique identifier for a
> system. I'm going to lay out the issues I've found recently, and perhaps
> others can chime in on how they dealt with it.
Oi, I hadn't thought about lazy manufacturers That reminds me of the
days of yore when certain ODMs were counterfeiting Intel 1GbE NICs and you
would find duplicate MAC addresses in large installations.
I'm hopeful for that the 1.8 "deploy interface" feature will help with some
of these gotchas.
···
On Saturday, March 28, 2015 at 10:16:21 AM UTC-5, treydock wrote:
>
> I can offer insight why BIOS UUID is problematic. I have 400 Super micro
> servers and all of them have the bios UUID of 0123456789. This is somewhat
> common for supermicro systems as in ovirt community others had issues with
> ovirt nodes due to this generic UUID value being the same between systems.
>
>
The Default hostname is the MAC address. I would prefer it be something
like "Board serial number" or "BIOS UUID"… or even better yet, let this
be a configuration option. I always change the hostname to the Board
Serial myself, but this is a manual process that requires me going into the
Facts during discovery, copy the data, paste over the mac00202032…
I guess I'm not getting too technical with the "identifier" bit, more at
the UI level.
···
On Tuesday, March 31, 2015 at 2:12:19 PM UTC-5, Greg Sutcliffe wrote:
>
> On 23 March 2015 at 14:58, lawre <all...@gmail.com > wrote:
> > One of the problems I find myself struggling with frequently with
> Foreman is
> > the choice to have the discovered MAC address as the unique identifier
> for a
> > system. I'm going to lay out the issues I've found recently, and
> perhaps
> > others can chime in on how they dealt with it.
>
> When you say "the MAC is the identifier for the host", what action are
> you referring to? The only one I can think of is for the TFTP records
> - for puppet reports it's matched on certname/FQDN (certname wins if
> supplied and different from the FQDN); for retrieving provisioning
> templates, the token is the default way to identify the host (although
> MAC is a fallback, it's true); and the API can use the ID or the FQDN
> of the host.
>
> For TFTP records, you're right that 1.8 will help - and it's worth
> knowing that we are considering allowing the "provisioning flag" to be
> assigned to multiple interfaces, which will help with bonds and so on.
> Also, TFTP can only use MAC or IP as an identifier (as far as I
> recall, it' looks for 01- followed by the IP in hex), so we're
> limited there.
>
···
On Thu, Apr 16, 2015 at 10:24 PM, lawre wrote:
The Default hostname is the MAC address. I would prefer it be something
like “Board serial number” or “BIOS UUID”… or even better yet, let this
be a configuration option. I always change the hostname to the Board
Serial myself, but this is a manual process that requires me going into the
Facts during discovery, copy the data, paste over the mac00202032…
I guess I’m not getting too technical with the “identifier” bit, more at
the UI level.
On Tuesday, March 31, 2015 at 2:12:19 PM UTC-5, Greg Sutcliffe wrote:
One of the problems I find myself struggling with frequently with
Foreman is
the choice to have the discovered MAC address as the unique identifier
for a
system. I’m going to lay out the issues I’ve found recently, and
perhaps
others can chime in on how they dealt with it.
When you say “the MAC is the identifier for the host”, what action are
you referring to? The only one I can think of is for the TFTP records
for puppet reports it’s matched on certname/FQDN (certname wins if
supplied and different from the FQDN); for retrieving provisioning
templates, the token is the default way to identify the host (although
MAC is a fallback, it’s true); and the API can use the ID or the FQDN
of the host.
For TFTP records, you’re right that 1.8 will help - and it’s worth
knowing that we are considering allowing the “provisioning flag” to be
assigned to multiple interfaces, which will help with bonds and so on.
Also, TFTP can only use MAC or IP as an identifier (as far as I
recall, it’ looks for 01- followed by the IP in hex), so we’re
limited there.
Ah, you're talking about the Discovery plugin - there's no such
thing as a default hostname for Foreman itself, since we always ask
the user to supply hostnames. In that case, Ori is right - the way we
construct hostnames should be more flexible. You can control it
directly, but currently only through the quota system of Discovery
Rules (which can take an arbitrary pattern for the hostname) - by
default we still use the mac for hosts which don't match a rule. Once #9490 is done that should fix your problem.
For a minute there, I was worried I'd missed an entire part of the UI
that was mac-specific
Greg
···
On 16 April 2015 at 20:24, lawre wrote:
> The Default hostname is the MAC address. I would prefer it be something
> like "Board serial number" or "BIOS UUID"... or even better yet, let this be
> a configuration option. I always change the hostname to the Board Serial
> myself, but this is a manual process that requires me going into the Facts
> during discovery, copy the data, paste over the mac00202032....
>
> I guess I'm not getting too technical with the "identifier" bit, more at the
> UI level.
You know what? you're right… this is about the Discovery plugin. It has
been working so well and seems so ingrained in our process that I forgot it
was a plugin! I believe #9490 that you and Ori pointed out will answer
most, if not all of my concerns.
···
On Sunday, April 19, 2015 at 4:17:27 PM UTC-5, Greg Sutcliffe wrote:
>
> On 16 April 2015 at 20:24, lawre <all...@gmail.com > wrote:
> > The Default hostname is the MAC address. I would prefer it be something
> > like "Board serial number" or "BIOS UUID"... or even better yet, let
> this be
> > a configuration option. I always change the hostname to the Board
> Serial
> > myself, but this is a manual process that requires me going into the
> Facts
> > during discovery, copy the data, paste over the mac00202032....
> >
> > I guess I'm not getting too technical with the "identifier" bit, more at
> the
> > UI level.
>
> Ah, you're talking about the Discovery *plugin* - there's no such
> thing as a default hostname for Foreman itself, since we always ask
> the user to supply hostnames. In that case, Ori is right - the way we
> construct hostnames should be more flexible. You can control it
> directly, but currently only through the quota system of Discovery
> Rules (which can take an arbitrary pattern for the hostname) - by
> default we still use the mac for hosts which don't match a rule. Once
> #9490 is done that should fix your problem.
>
> For a minute there, I was worried I'd missed an entire part of the UI
> that was mac-specific ;)
>
> Greg
>
