Boot Issue Using Secondary Foreman Proxy

[tlauwk]

Hi,

I’m using Foreman 1.19.1 to deploy esxi 6.5 on Dell servers and its working well on my main site. I’ve now setup a secondary Foreman proxy server at a remote location with just dhcp, discovery and tftp.

Server will PXE boot to the Discovery image and be “discovered” on the master Foreman proxy server and then it’ll install esxi 6.5 successfully.

Problem comes up when it reboots and defaults to the Discovery Image instead of “Chainload Grub2 EFI from ESP”.

Manually selecting “Chainload Grub2 EFI from ESP” then gives the following messages:

Probing ESP partition … found hd0.gpt1
File grubx64.efi not found on ESP.
Update pxegrub2_chainload paths array with:
./ …/ boot/ VMware/ Dell/

pxegrub2_chainload has already been updated previously to get Foreman working at my main site:

<%
  paths = [
    '/EFI/BOOT/BOOTx64.EFI',
    '/EFI/fedora/shim.efi',
    '/EFI/fedora/grubx64.efi',
    '/EFI/redhat/shim.efi',
    '/EFI/redhat/grubx64.efi',
    '/EFI/centos/shim.efi',
    '/EFI/centos/grubx64.efi',
    '/EFI/debian/grubx64.efi',
    '/EFI/ubuntu/grubx64.efi',
    '/EFI/sles/grubx64.efi',
    '/EFI/opensuse/grubx64.efi',
    '/EFI/Microsoft/boot/bootmgfw.efi'
  ]
-%>

I’ve loaded a live CD and verified that “/EFI/BOOT/BOOTx64.EFI” does exists on the local disk (which is a mirrored pair of SD cards, there’re no other disks on this server.)

Its like the secondary foreman proxy is using a separate pxegrub2_chainload compared to the master foreman proxy.

Any advice would be much appreciated.

Thanks.

[lzap]

Hello,

this is really a horrible hack. There are two options - make this working and you really need trial and error here as I don’t have these Dell servers with SD cards here to test it. Or use the new approach I tend to recommend these days - change boot order to boot from HDD (or SD card whatever) instead.

EFI Boot order with CentOS 7: Network Boot vs. Local Boot

It needs to be custom for each individual vendor I guess. More than that, if there are multiple NICs, there are also multiple entries: [root@mac525400131301 ~]# efibootmgr BootCurrent: 0008 Timeout: 0 seconds BootOrder: 0002,0003,0004,0005,0007,000B,000C,000D,0008,0001,0000,0006,0009,000A Boot0000* UiApp Boot0001* UEFI Misc Device Boot0002* UEFI QEMU DVD-ROM QM00005 Boot0003* UEFI PXEv4 (MAC:525400131301) Boot0004* UEFI PXEv6 (MAC:525400131301) Boot0005* UEFI HTTPv4 (MAC:525400131301) Boot0006*…

There is a snippet which should help you with this a bit. It really depends on EFI vendor if the system will change boot order after new EFI entry putting it as the first or as the last. Some vendors allows you to configure this, so you can decide. Or you can use this snippet and change the ordering via script, however there are challenges (boot entry name is always different).

https://github.com/theforeman/community-templates/pull/557/files

If you want to try to chainloading stuff, then make sure you use the latest and greatest version from community-templates as there were some changes. Also put DELL/VMWARE EFI files into paths array instead of the BOOTx64.EFI which is I think just some shim:

  paths = [
    '/EFI/VMWARE/XXXX.EFI',
  ... ]

You need to find the path first, make sure not to make a typo. That should work.

[tlauwk]

Thing is it all works fine at my main site where the master Foreman server is located. Physical host has PXE selected as first boot option:

1. Server defaults to “discovery image” on first boot.
2. Then I configure it for build and it reboots into the esxi installer.
3. Once esxi is installed, it reboots and defaults to “Chainload Grub2 EFI from ESP”.
4. Then it loads esxi and runs the customised kickstart script from a Provisioning Template.
5. Finally it reboots and defaults again to “Chainload Grub2 EFI from ESP” and loads esxi all ready to go.

But when I point a physical host to use the remote smart proxy (with dhcp, tftp, discovery and templates features installed), it fails at step 3 where it defaults to “discovery image”. And if I select “Chainload Grub2 EFI from ESP” manually, it says “File grubx64.efi not found on ESP”.

So I reckon I’m not understanding how the remote smart proxy is suppose to work. The physical host gets a dhcp lease correctly, boots into the esxi installer correctly but doesn’t seem to be able to read the pxegrub2_chainload template from the master Foreman server. Hence it can’t find /EFI/BOOT/BOOTx64.EFI even though its there.

I’ve enabled templates.yml on the remote proxy:

:enabled: true
:template_url: http://remote-proxy:8000

And on settings.yml on the remote proxy:

:foreman_url: https://foreman-master
:https_port: 8443
:http_port: 8000

The tftpboot directory from the Foreman master is synced to the remote proxy (every 15 seconds). I can see the mac address “directory” created under tftpboot with the boot.cfg file. But the corresponding mac address “file” under the pxelinux.cfg directory isn’t created.

Thanks.

[tlauwk]

Getting closer.

Noticed several “MAC Address” directories/files were not being created in the tftpboot folder:

tftpboot/grub/01-MAC-ADDRESS
tftpboot/grub/menu.lst.01MACADDRESS
tftpboot/grub2/grub.cfg-01-mac-address

Adding the 3rd entry above “tftpboot/grub2/grub.cfg-01-mac-address” allowed "…reboots and defaults to “Chainload Grub2 EFI from ESP …” to work. It found the /EFI/BOOT/BOOTx64.EFI file and ESXi loaded successfully.

So I need to get “tftpboot/grub2/grub.cfg-01-mac-address” created automatically. It does when the physical host points to the master Foreman server for the build process. But it does not when pointed to the remote smart proxy.

Thanks.

[tlauwk]

My bad! Doh!!!

The “mac address files” were actually being created on the remote smart proxy. But the script syncing the tftpboot folder from the Foreman Master to the Remote Proxy was deleting them. Thanks to rsync’s “–delete” option.

I need to get away from computers for a while.

Sorry for wasting your time lzap. Have a great weekend.

[lzap]

You too @tlauwk!