Ideas for IPv6 netboot provisioning

On of the main Foreman’s strength is PXE management (bare-metal provisioning). It’s built around management of the TFTP directory boot files and DHCP reservations. When a host is created in Foreman, DHCP configuration for particular client is generated and the server identifies the client using it’s unique key (MAC address), bootloader is downloaded from TFTP and it reads either generic or unique configuration leveraging again the unique key.

However in IPv6 world it’s different. DHCPv6 was designed in a way that unique identifier is always different, SLAAC protocol is, if I understand it correctly, unmanaged. There are issues with dual-stack networks, ISC DHCP server only support IPv4 and brand-new software which is not supported by Foreman yet (Kea) must be used for DHCPv6. I’ve seen several new implementations of DHCPv6 which aims to solve the problem of unique identifiers. It’s a mess.

So what I would like to hear from you guys is:

  • If you netboot IPv6 clients, how do you do it?
  • How did you solve the issues mentioned above?
  • Which workflow would you like to see in Foreman?


Adding some of my my rough ideas:

Option 1: Chainbooting iPXE over unmanaged DHCPv6/TFTP

This should work with both BIOS and UEFI bare-metal. For VMs with iPXE embedded the chainbooting can be simply skipped. The workflow:

  • a node initiates PXE boot requesting IPv6 address
  • a DHCPv6 server replies with filename “http://foreman/ipxe.efi” for UEFI or ipxe.lkrn for BIOS
  • iPXE initializes network and performs another IPv6 request
  • a DHCPv6 server replies with a different filename “http://foreman/script.ipxe
  • The bootstrap configuration file contains statement to load “http://foreman/script.ipxe?mac=AA:BB:CC:DD:EE:FF
  • The final configuration file contains kernel/initramdisk statements to boot Anaconda

Changes required in Foreman:

  • none if bootdisk plugin is installed
  • add MAC parameter for unattended controller otherwise (bootdisk provides this feature)

This is the easiest workflow I think, the only thing to do is to setup unmanaged DHCPv6 (ISC can be used), TFTP server with iPXE files, configuring it for chainbooting and that should be it. Foreman ships with required templates and bootstrapping has been recently added by Timo (intermediate iPXE template). One slight disadvantage is that tokens cannot be used, MAC-address spoofing is possible. On the other hand, this has been always possible when bootdisk is installed.

Option 2: HTTP UEFI Boot over unmanaged DHCPv6

This is a similar workflow but with HTTP UEFI bootstrapping procedure (no PXE/TFTP). Both iPXE or grub2 can be used. Since iPXE workflow is almost the same as the one above (except there is no TFTP bootstrap) I describe grub2 case:

This is a nice alternative for hardware where iPXE is not compatible. Again, there are no changes required in Foreman as we already have support for HTTP UEFI boot (the last PR is pending merge). I have tested HTTP UEFI Boot over IPv4, the question is if it works over IPv6 as well.

Option 3: PXE over managed IPv6

DHCPv6 does not advertise MAC address in the request so it is not possible to create host reservations and in PXE environment DUID (unique client identifier) changes every boot so the MAC address is the only reasonable way of identifing clients I think. To workaround that, the workflow would require DHCPv6 relay server with Client Link-Layer Address Option in DHCPv6 feature. In short:

Currently, the DHCPv6 specification [RFC3315] does not define a way
to communicate the client link-layer address to the DHCP server in
cases where the DHCP server is not connected to the same network link
as the DHCP client. The DHCPv6 specification mandates that all
clients prepare and send a DHCP Unique Identifier (DUID) as the
client identifier option in all the DHCPv6 message exchanges.
However, none of these methods provide a simple way to extract a
client’s link-layer address. This presents a problem to an operator
who is using an existing DHCPv4 system with the client link-layer
address as the customer identifier and who desires to correlate
DHCPv6 assignments using the same identifier. [RFC4361] describes a
mechanism for using the same DUID in both DHCPv4 and DHCPv6.
Unfortunately, this specification requires modification of existing
DHCPv4 clients, and has not seen broad adoption in the industry
(indeed, we are not aware of any commercial implementations).

Providing an option in DHCPv6 Relay-Forward messages to carry the
client link-layer address explicitly will help the above mentioned
scenarios. For example, it can be used along with other identifiers
to associate DHCPv4 and DHCPv6 messages from a dual-stack client.
Further, having the client link-layer address in DHCPv6 will help by
providing additional information for event debugging and logging
related to the client at the relay agent and the server. The
proposed option may be used in a wide range of networks; two notable
deployment models are service provider and enterprise network

So the workflow would be the typical PXE workflow as we have it today (new host or discovery) with the following:

  • DHCPv6 relay with MAC address feature
  • DHCPv6 server that can be managed via Foreman and supports identifying hosts by MAC
  • changes in orchestration and smart-proxy API to support IPv6 provisioning interface

The only DHCPv6 server that can be managed today and fulfills the requirements is dnsmasq. From its man page:

A single --dhcp-host may contain an IPv4 address or an IPv6 address, or both. IPv6 addresses must be brack‐eted by square brackets thus: --dhcp-host=laptop,[1234::56] IPv6 addresses may contain only the host-identifier part: --dhcp-host=laptop,[::56] in which case they act as wildcards in constructed dhcp ranges, with the appropriate network part inserted. Note that in IPv6 DHCP, the hardware address may not be available, though it normally is for direct-connected clients, or clients using DHCP relays which support RFC 6939.

The question is how to approach dual-stack nodes, the current idea treats DHCPv4 and DHCPv6 as two completely separate entities. There are advantages in managing the DHCP addresses in some combined way.

I just wanted to mention something, as it didn’t seem clear in your description.

From my understanding typically SLAAC is used for address assignment, and DHCPv6 is used for everything else (known as Stateless DHCP). So SLAAC assigns the address, and DHCPv6 provides everything else (DNS, Boot Info, etc). The Host communicates with the router to determine a IPv6 address, and the other info comes from the DHCP server.

Is this a model we are planning to support?

Note entirely true. RFC 8106 IPv6 Router Advertisement Options for DNS Configuration means SLAAC can provide the DNS info and don’t need DHCPv6 to be present.

Sure, but for our purposes (Network Boot) DHCPv6 needs to be present.

I haven’t looked at all the options so I can’t say for sure, but I was trying to say that many networks don’t need DHCPv6 so often it won’t be present yet.