Hello,
for upcoming work Strict SecureBoot/HTTPS Provisioning of Bare-Metal and VMs I need to solve distribution of netboot.iso files onto TFTP smart proxies. Because Foreman already does support bootloader universe directory structure - see more in Provisioning hosts - I think it makes sense to combine both things and think about how smart-proxy might manage this directory.
Foreman today can download PXE files (kernel, initramdisk) from Installation Media and Katello/Pulp, when installed. The way this works is there is a simple API call /tftp/fetch_boot_file (exposed by tftp capability) that takes two parameters: source, destination (they are named path and prefix but for simplicity I changed them). Source is a URL (from installation media or Pulp), destination is a relative path within TFTP directory. Two files are downloaded today: kernel and initramdisk.
A new capability called bootloader_universe was added recently, this activates when Foreman deploys one of its TFTP templates. Instead of simply writing the PXE configuration directly to boot/ directory, it creates host-config/MAC directory as documented and symlinks all files that were manually deployed, for example bootloader-universe/pxegrub2/rhel/9.6/x86_64/grubx64.efi.
What I propose is that Foreman Server + Proxy starts managing the universe directory automatically. AFAIR this was the ultimate goal but would require more work, so we ended up just implementing manual approach. And it looks like this is not too much work!
The feature can be implemented on top of what we have today. We just need to change the current download API /tftp/fetch_boot_file
- source (is present today)
- destination (is present today)
- extract: list of files (source, destination) and type (iso, rpm, extensible)
By using the current API, Foreman can download files not only to boot/ (current behavior) but also to bootloader-universe/. More than that, by adding little bit of code, it can extract various files from formats like ISO, RPM or DEB packages. This is important, because bootloaders are typically not available easily as simple files. This all can be done with little effort and minimal changes.
What is great about this design is it works with or without Katello and with any OS, the only thing is to add a bit of code into Operating System to provide the correct list of files to download. We would implement Red Hat and Debian systems initially so the community can do followups. Also, this is fully backward compatible - existing OSes which will not have support for this bootloader universe management will continue working.
Finally, the change is compatible too from API standpoint, we would be only adding new flags and behavior to the /tftp/fetch_boot_file endpoint. Proxy would need to advertise this capability tho, perhaps using a new bootloader_artifacts and for extraction capabilities we would have bootloader_iso_extract so new formats can be added at later date.
One thing that is open question is the more files Foreman downloads the more the probability of host not being able to boot because files are not yet downloaded. Specifically if we go the extracting Grub2 from boot.iso path, then it is an issue. For that, I think Foreman should trigger downloading of TFTP files the moment Operating System is created or edited (when it has all required associations like Installation Media).
