Problem:
Provisioning via foreman provisioning templates / kickstart; Pinning /dev/sda to the root/var/logs/tmp, etc filesystem when it builds rather than it randomly selecting the other larger disk say sdc. Very frustrating. Expected outcome:
The root filesystem always installs on /dev/sda
First off:
This is a kernel “problem” rather than a Foreman thing. Over the last years, the kernel tried to improve boot times quite a lot, one of the things in this effort was parallelization in hardware initialization. This results in dynamically generated device names like sda, sdb, etc to not be consistent across reboots anymore, which is what you see here. When the installer is booted, the order the kernel initializes harddrives in might be different then after the next boot (technically, every boot will lead to this). After installation, you will most likely not notice it changing anymore because usually, you have other layers to refer to the FSes you mount then (lables, uuids, LVM volumes, etc), but under the hood the same thing happens.
This behaviour already became apparent to us with RHEL8, and RHEL9 is even more “dynamic” in this regard.
Now for a solution:
I assume you are using custom templates already, because afair, the default templates can not handle multi-disk systems anyways.
The kernel offers several consistent paths to access disks that are not dependent on hardware initialization order. These are all under /dev/disk/by-* and can be used to solve your problem.
We found that using /dev/disk/by-path was the best solution for our needs, since it refers to the disks by their path on the PCI/SCSI tree. Like you, we always want to have the first disk be the OS disk. We accomplished this via the following script in our partition table file:
This reads all the disks on the PCI tree that are real disks (not existing partitions or similar things) and declares the first disk as OS disk.
You can then use something like this for partitioning:
bootloader --boot-drive=$OS_DISK
part /boot --fstype=ext4 --size=750 --ondrive=$OS_DISK
part pv.01 --size=1000 --grow --ondrive=$OS_DISK
volgroup system pv.01
The DISKCOUNT variable let’s you react to additional disks being present and you can always cut out other disks from the DISKS variable to handle additional disks being present.
PS: Depending on your environment, you might need to alter the egrep command for the DISKS variable. This grep should work for bare-metal, VMWare and OVirt/OLVM based systems, but depending on your virtualization solution, the names in by-path may vary.
The code I posted all needs to go into a partitioning template (if you are using those). Then the default KS template should do all the needed things. Here is a small example how it would look in a rendered KS file:
url --url http://example.com/fooo
%include /tmp/diskpart.cfg
%pre
#Dynamic - this line tells Foreman this is a script rather then a static layout
DISKS=$(ls -d1 /dev/disk/by-path/* | egrep "/(pci-.*scsi.*:[0-9]|virtio-pci-.*.0)$")
DISKCOUNT=$(echo ${DISKS} | wc -w)
OS_DISK=$(echo ${DISKS} | cut -d' ' -f1)
cat <<EOF >> /tmp/diskpart.cfg
zerombr
clearpart --all --initlabel
bootloader --location=mbr --append="nofb net.ifnames=0 biosdevname=0 audit=1 audit_backlog_limit=8192" --boot-drive=$OS_DISK
part /boot --fstype=ext4 --size=750 --ondrive=$OS_DISK
part pv.01 --size=1000 --grow --ondrive=$OS_DISK
volgroup system pv.01
...
EOF
%end
Essentially, the %pre section is a shellscript that gets executed before the actuall install part. There you can write out a file to the local fs (/tmp/diskpart.cfg in this case) that uses KS partitioning syntax.
The %include statement in the main KS section can then pick up that file and use it as though it was part of the actual Kickstart file.
The line with the comment #Dynamic - .... is for Foreman’s partition tables feature so Foreman knows it needs to go into %pre instead of the main section.
Basically, you can just shove everything that should go into %pre into a PTable template, assign that template to your hosts and everything should just work.
