Discovery Image Boot Issue

tlauwk · November 23, 2018, 3:33am

Hi,

Having some issue getting Discovery to work properly. I’ve followed the following doco:

Foreman :: Plugin Manuals

Foreman version is 1.19.1. I’ve …

foreman-proxy-plugin-discovery-install-images=true
/var/lib/tftpboot/boot/fdi-image directory created with relevant files in it
change from local to discovery in Default PXE global template entry
clicked on Build PXE Default button
/var/lib/tftpboot/pxelinux.cfg/default created with the following portion included:

LABEL discovery
MENU LABEL Foreman Discovery Image
KERNEL boot/fdi-image/vmlinuz0
APPEND initrd=boot/fdi-image/initrd0.img rootflags=loop root=live:/fdi.iso rootfstype=auto ro rd.live.image acpi=force rd.luks=0 rd.md=0 rd.dm=0 rd.lvm=0 rd.bootif=0 rd.neednet=0 nomodeset proxy.url=https://foreman-server.company.com proxy.type=foreman
IPAPPEND 2

When booting a UEFI host via PXE, it gets a DHCP lease but then boots into the bootx64.efi as indicated by the dhcpd.conf:

#Bootfile Handoff
next-server 10.11.12.4;
option architecture code 93 = unsigned integer 16 ;
if option architecture = 00:06 {
filename “grub2/bootia32.efi”;
} elsif option architecture = 00:07 {
filename “grub2/bootx64.efi”;
} elsif option architecture = 00:09 {
filename “grub2/bootx64.efi”;
} else {
filename “pxelinux.0”;
}

The above bootfile handoff works fine for our esxi deployment as once the host is built, PXE Loader is changed to none and the host will fail on PXE boot and then boot to the next sequence which is local disk.

So I’m now trying to add a further step with Discovery to deploy esxi hosts but can’t get it to boot into the discovery image. Any help would be much appreciated.

Thanks.

tlauwk · November 23, 2018, 5:58am

Changing dhcpd.conf bootfile handoff to use shim.efi allows the host to boot into the Discovery Image.

Upon discovery and provisioning the host in Foreman, it then deploys esxi successfully. However, the subsequent PXE boot then defaults to:

Chainload into BIOS bootloader on first disk

Manually changing it (as its UEFI) to …

Chainload Grub2 EFI from ESP

… however fails to boot:

File grubx64.efi not found on ESP.
The system will halt in 2 minutes or
press ESC to halt immediately.

So the current option would be to not have PXE as the first boot order. Then the engineer would have to manually select PXE boot during the deployment process.

Wonder if anyone has gotten around this chain loading issue with esxi.

Thanks.

lzap · November 23, 2018, 1:35pm

Hey, from our manual:

In the Foreman UI, go to Settings and on the Provisioning tab, set Default PXE global template entry to discovery instead of local. This enables booting discovery for unknown hosts. To commit the change, click on Build PXE Default button.

Have you performed this? After you click the button, both Grub2 and PXELinux global configuration files are deployed and default menu entry changes to discovery, so it should work out of box. However, as we state below:

This only works for Foreman 1.18 or higher, for older versions go to Provisioning Templates, edit (or clone) PXELinux global default template and change the default entry from localboot entry to Discovery.

This needs to be done also for PXEGrub2 template as well, this bit is missing in the manual. I filed a PR:

I have never provisioned ESXi via Foreman. Is it really EFI installation?

If it is, we look for particular EFI files on ESP volume that is only Grub2 and Windows boot loader. You need to figure out which one to load and add it here:

github.com

theforeman/community-templates/blob/11f63675790cb5f2a74f4e28498077daa5d6dc59/provisioning_templates/snippet/pxegrub2_chainload.erb

<%#
kind: snippet
name: pxegrub2_chainload
model: ProvisioningTemplate
snippet: true
%>
<%
  paths = [
    '/EFI/fedora/shim.efi',
    '/EFI/fedora/grubx64.efi',
    '/EFI/redhat/shim.efi',
    '/EFI/redhat/grubx64.efi',
    '/EFI/centos/shim.efi',
    '/EFI/centos/grubx64.efi',
    '/EFI/debian/grubx64.efi',
    '/EFI/ubuntu/grubx64.efi',
    '/EFI/sles/grubx64.efi',
    '/EFI/opensuse/grubx64.efi',
    '/EFI/Microsoft/boot/bootmgfw.efi'
  ]

This file has been truncated. show original

tlauwk · November 23, 2018, 2:09pm

Yes, I’ve done this part as I’m using Foreman 1.19.1.

As for …

… I’ll give this a go next week and research further on esxi’s efi on esp installation.

Thanks for the heads up.

Cheers.

lzap · November 23, 2018, 2:14pm

All you need to do basically is to list files on the ESP partition and find the proper EFI file which ESXi uses. Then add it to the list and update your template according to github (this one is fresh, we recently refactored it a bit). Then enter/exit build mode to re-render the template for the host and reboot it to test it. Please file a PR if you find working combination, we get more and more questions about ESXi but we are not heavy VMWare users here (well some of us are but I am not)

tlauwk · November 26, 2018, 5:36am

Editing the pxegrub2_chainload provisioing template and adding the path …

/EFI/BOOT/BOOTx64.EFI

… worked for the esxi host … somewhat. Upon reboot into PXE, the host gets presented with the following options:

Chainload Grub2 EFI from ESP
*Chainload into BIOS bootloader on first disk
Chainload into BIOS bootloader on second disk
Chainload Grub2 EFI from ESP
Chainload into BIOS bootloader on first disk
Chainload into BIOS bootloader on second disk
Foreman Discovery Image

I have to manually select the 1st option “Chainload Grub2 EFI from ESP” in order to boot into the disk with esxi installed.

How can I get it to default to the 1st option “Chainload Grub2 EFI from ESP”?
Why are the first 3 options repeated again before the very last Foreman Discovery Image option?

These are the messages on screen after selecting the 1st option:

Chainload Grbu2 EFI from ESP, available devices:
(hd0) (hd0,gpt8) (hd0,gpt7) (hd0,gpt6) (hd0,gpt5) (hd0,gpt1) (hd1) (cd0) (cd1)
error: failure reading sector 0x0 from ‘hd1’.
error: failure reading sector 0x0 from ‘cd1’.
Probing ESP partition … found hd0,gpt1
Press any key to continue …

Takes about 10-15 seconds before it automatically boots into esxi if you don’t press any key.

Thanks.

tlauwk · November 26, 2018, 6:33am

My bad!

I should have just used the pxegrub2_chainload file at the Github location provided by Izap. Instead, I only modified the existing template.

Once I replaced the pxegrub2_chainload template with the code from Github …

*Chainload Grub2 EFI from ESP

… became the default option.

Woo Hoo! I’m almost there with deploying esxi. Now to figure out how to take into account the different custom kickstart scripts for configuring the esxi host.

Thanks.

lzap · November 26, 2018, 8:58am

In older versions you had to modify the “DEFAULT” line. In 1.19+ I think (maybe 1.20) you have global option available you can choose in the WebUI/CLI/API. There is a macro that will expand to value chosen.

The default setting is to boot unknown hosts from HDD, only after you enable discovery option it’s used as the default entry.

Foreman PXE workflow is not very EFI friendly, we relied on fact that in BIOS it’s very easy to PXE chainload from HDD. In EFI we can only do this via this “hack”, that’s why the experience is sub-par. You may experience long delays, error messages and depending on hardware/firmware various errors.

Hopefully the last version from git is better on ESXi, we regularly test this on bare-metal and KVM VMs with Red Hat and Debian distros. Feel free to send patches to improve ESXi experience, thanks.