Booting a diskless serverfarm

timbuchwaldt · February 9, 2021, 1:33pm

Hey Guys,
we are trying to set up a diskless compute setup, where a portion of our servers would netboot a complete OS on every single boot.
I found an older thread with a similar question (There’s a way to provision diskless hosts?) - @lzap commented this should be doable by modifying PXE templates. I would assume the template to modify would be the “iPXE default local boot”, as I guess the host wouldn’t be in build-mode all the time. Can you give any further hints on this? My initial thought would be creating a similar template, checking if the host in question belongs to our netbooted clusters, then just doing the usual iPXE netboot + CloudInit magic, does that sound half-correct?

Thanks,

Tim

lzap · February 9, 2021, 2:18pm

I assume your problem is that the default PXE template boots from localhost, but you want for some group or servers to boot it from network with some extra arguments.

Well, I looked into the codebase and @host variable should be available in the local templates, so you can basically use hostgroup name or some host/hostgroup parameter to find out if a host should boot from network. Modify PXELinux, Grub2 or iPXE template:

github.com

theforeman/foreman/blob/develop/app/views/unattended/provisioning_templates/PXELinux/pxelinux_default_local_boot.erb

<%#
kind: PXELinux
name: PXELinux default local boot
model: ProvisioningTemplate
%>
<%# Used to boot provisioned hosts, do not associate or change the name. %>

UI menu.c32
MENU TITLE Booting local disk (ESC to stop)
TIMEOUT 200
ONTIMEOUT <%= global_setting("default_pxe_item_local", "local_chain_hd0") %>
DEFAULT <%= global_setting("default_pxe_item_local", "local_chain_hd0") %>

<%= snippet "pxelinux_chainload" %>

Then all hosts which exit build mode (or never enter it) are deployed with these templates and will boot from HDD or network depending on how you set it up.

Alternatively, you can create some rules that will be executed in the bootloader based on hostname, IP/subnet, BIOS/EFI serial names or MAC addresses and boot either from HDD or network. You can’t do this with PXELinux but both Grub2 and iPXE have pretty powerfull scripting languages you can use.

In any case, get back to us with your solution so others can leverage what you find out!

ferrao · February 15, 2021, 5:57pm

If you find a way to properly do this, please report it back here. I wasn’t able to made it work in past. I confess that I didn’t tried that much because I didn’t have the time required, but as @lzap stated it should be doable.

timbuchwaldt · February 22, 2021, 3:05pm

So, i can confirm that @lzap is right and you can just modify this file and set it as a new default. I used <% if host_param_true?('..') %> to check if my host is to be netbooted. To work around foreman wanting to proxy all OS images and such we currently just copy them to the tftp-directory and hard-link them in the template, but this is probably not the most elegant solution.

lzap · February 24, 2021, 7:04am

Feel free to make a PR into the template, find a nicely named parameter and provide some good example item so other users could inspire.

Can you elaborate?

Foreman currently downloads PXE images after host is created, we could probably create a webhook when OS is modified and when Katello synchronizes or publishes content to initiate downloading too.