Provisioning EL7 on older Foreman versions

Hello!

EL7 x86_64 hosts provisioned through UEFI PXE can stop at Booting 'Kickstart default PXEGrub2' after a provisioning server or TFTP proxy is upgraded to EL9. The common grub2/grubx64.efi then comes from EL9, and is not compatible with EL7’s provisioning kernel/initramfs in this configuration.

Replacing the common grubx64.efi with an EL7 (or EL8) binary can make EL7 work again, but it is not a general solution: it affects every UEFI host, prevents suitable EL9/EL10 provisioning, and can be undone by installer runs.

Now, we have a good solution for that called bootloader universe which solve the problem completely. But this is a relatively new feature and we keep hearing from users and customers of older versions of Foreman, Katello and Satellite. Therefore, I took a look on how to create a opt-in solution that is small enough so it can be easily backported.

Solution

This opt-in patch for Foreman Server (no Smart Proxy change needed) selects one of four additional GRUB EFI files only for non-Secure-Boot, TFTP Grub2 UEFI provisioning of RHEL 7–10 x86_64 hosts. Everything else is unchanged and unsupported. This is how the filename option changes depending on the OS information:

Provisioned OS Boot filename
RHEL 7 grub2/grubx64-el7.efi
RHEL 8 grub2/grubx64-el8.efi
RHEL 9 grub2/grubx64-el9.efi
RHEL 10 grub2/grubx64-el10.efi

It has no effect unless FOREMAN_RHEL_VERSIONED_GRUB=true is set in the Foreman process environment. With the variable absent or any other value, Foreman retains the existing grub2/grubx64.efi filename for every host.

I prepared a tiny patch that applies cleanly on any Foreman version 3.12+ (the test applies cleanly from 3.14+). It is intentionally small and should be straightforward to adapt to older supported branches.

Scope and limitations

  • Applies only when the Operating System family name is exactly RedHat, the major version is 7, 8, 9, or 10, the architecture is x86_64, and the PXE loader is Grub2 UEFI.
  • Does not affect other operating systems, BIOS, other architectures, HTTP Boot, or the existing generic loader.
  • Secure Boot is deliberately unsupported. Do not select Grub2 UEFI SecureBoot: shim has a hard-coded GRUB filename and cannot select these versioned names.
  • No Smart Proxy change is required. The proxy only serves static files from its existing TFTP root.

Installation

Apply the attached Core patch to the Foreman server and deploy the four GRUB EFI binaries to every TFTP proxy that provisions these hosts:

/var/lib/tftpboot/grub2/grubx64-el7.efi
/var/lib/tftpboot/grub2/grubx64-el8.efi
/var/lib/tftpboot/grub2/grubx64-el9.efi
/var/lib/tftpboot/grub2/grubx64-el10.efi

Use the grubx64.efi binary from the matching RHEL major release for each file. Preserve the files through local configuration management or a post-upgrade step. After copying them, restore the TFTP SELinux labels if SELinux is enabled:

restorecon -Rv /var/lib/tftpboot/grub2

Enable the behavior with a systemd drop-in for the service that runs Foreman. On installations where Foreman runs in Apache, use httpd.service.d; where it runs as foreman.service, use foreman.service.d. Do not set this on the Smart Proxy service.

install -d -m 0755 /etc/systemd/system/httpd.service.d

cat >/etc/systemd/system/httpd.service.d/90-rhel-versioned-grub.conf <<'EOF'
[Service]
Environment=FOREMAN_RHEL_VERSIONED_GRUB=true
EOF

systemctl daemon-reload
systemctl restart httpd

Replace httpd with foreman in all paths and commands when appropriate.

Finally, rebuild the affected hosts’ PXE/DHCP configuration so their DHCP filename changes from grub2/grubx64.efi to the versioned filename. New builds use the selected filename automatically.

To disable the workaround, remove the drop-in or the environment setting, reload systemd, restart the Foreman service, and rebuild PXE/DHCP configuration. The old generic behavior is then restored.

The patch

diff --git a/app/models/operatingsystem.rb b/app/models/operatingsystem.rb
index c81cf7b06..0ba6f786d 100644
--- a/app/models/operatingsystem.rb
+++ b/app/models/operatingsystem.rb
@@ -267,7 +267,7 @@ class Operatingsystem < ApplicationRecord
     return default_boot_filename if host.nil? || host.pxe_loader.nil?
     return host.foreman_url('iPXE') if host.pxe_loader == 'iPXE Embedded'
     architecture = host.arch.nil? ? '' : host.arch.bootfilename_efi
-    if host.subnet&.httpboot? && host.pxe_loader =~ /UEFI HTTP/
+    filename = if host.subnet&.httpboot? && host.pxe_loader =~ /UEFI HTTP/
       if host.pxe_loader =~ /HTTPS/
         port = host.subnet.httpboot.httpboot_https_port!
       else
@@ -279,6 +279,20 @@ class Operatingsystem < ApplicationRecord
       raise(::Foreman::Exception.new(N_("HTTP UEFI boot requires proxy with httpboot feature"))) if host.pxe_loader =~ /UEFI HTTP/
       self.class.all_loaders_map(architecture)[host.pxe_loader]
     end
+
+    rhel_versioned_grub_filename(host, filename)
+  end
+
+  # EL 7-10 need a GRUB EFI binary built for their respective major release.
+  # Secure Boot is deliberately excluded: shim has a hard-coded GRUB filename.
+  def rhel_versioned_grub_filename(host, filename)
+    return filename unless ENV['FOREMAN_RHEL_VERSIONED_GRUB'] == 'true'
+    return filename unless family == 'Redhat'
+    return filename unless host.pxe_loader == 'Grub2 UEFI'
+    return filename unless host.arch&.name == 'x86_64'
+    return filename unless %w[7 8 9 10].include?(major.to_s)
+
+    filename.sub('grub2/grubx64.efi', "grub2/grubx64-el#{major}.efi")
   end
 
   # Does this OS family use release_name in its naming scheme

The patch gist also contains a simple unit test.

Disclaimer

I haven’t tested this myself, this is just a theory of mine, but I got it checked with AI. As you can see, there are articles in the post so I also fixed grammar with AI :slight_smile:

Looking at the smart proxy repo, the original commit was merged on Nov 19, 2024, which was almost two years ago.

I guess it makes the bootloader universe fit the relatively new feature category, but IMHO instead of providing other fixes we should promote it more, especially in docs, tutorials, and maybe some articles on the Foreman blog. (I volunteer to help with this effort)
I think it’s still an unknown feature that deserves more spotlight.

Yes, this thread is specifically meant for users of old Foreman and Satellite who no longer can provision EL7.

One problem with the new feature I have personally is that this is something that does not work out of box. So I started a new thread about this moments I finished this one:

1 Like

Yeah, we talked about it, agreed that we want to deal with it, and … the rest is history.

Once you have time, I pushed the first step in managing universe bootloaders already. The link to the PR is in the thread. :smiley:

1 Like