Automatically Install Arch Linux

A Guide To Install Arch Linux Using Foreman

Hey people :slight_smile:

Introduction

I am pretty new to the Foreman project and was just recently introduced to it.
Since I am a pretty big fan of A) automation and B) Arch Linux I instantly tried to integrate Arch.

I am not deep into ERB or the Foreman ecosystem itself. So please bear with me and my code style.

TLDR

I got Arch Linux working!
But the scripts used still need some love.

Requirements

  • Foreman
  • Arch Linux ISO
  • Test system

The General Idea

Arch Linux does not provide anything similar to preseed or autoyast. It does however offer the ability to run a script when booting into the live ISO.

Arch Linux also offers the archinstall script, which can be used to install the system either interactively or non-interactively.
You can point the parameter - that determines which script to run at boot - to a URL → e.g. a Foreman provision script.

This way we can use host specific information within the archinstall script (hostname, etc.).

archinstall can save your manually provided configuration to files which you can then use to run the script non-interactively.
Those configuration files can be baked into partition tables and provisioning scripts.


The next thing we need is a way to start the live ISO’s kernel. It will be provided via http.
For this we will also use some symlinks in order to point Foreman to the correct path.

We also need to adjust where Arch Linux can find the actual ISO files. Those will be provided via http as well.


The archinstall script is not mature enough (currently). For that reason we need to adjust a few things after the script has run (setting passwords for example).


In summary:

  • provide the Arch Linux ISO’s files via http
  • boot into the live ISO’s kernel
  • adjust kernel parameters to run a provision script
  • create
    • Installation Media
    • Operating System
    • partition table (Arch Linux default)
    • PXELinux template (Arch Linux default PXELinux)
    • Provisioning script (Arch Linux default)

The Guide

Providing ISO Files

Get the Arch Linux ISO (example URL):

wget http://ftp.fau.de/archlinux/iso/2023.05.03/archlinux-x86_64.iso

Create a temporary directory and mount the ISO:

mkdir tmp_iso
mount archlinux.iso tmp_iso

Create a directory for the ISO’s files to live in:

mkdir /var/www/html/pub/archlinux

Copy the ISO’s files:

cp -r tmp_iso/* /var/www/html/pub/archlinux/

Create directories and symlinks so Foreman finds ‘linux’ and ‘initrd’:

mkdir -p /var/www/html/pub/archlinux/boot/x86_64/loader

cd /var/www/html/pub/archlinux/boot/x86_64/loader/
ln -s ../../../arch/boot/x86_64/vmlinuz-linux linux
ln -s ../../../arch/boot/x86_64/initramfs-linux.img initrd

You should now have the contents of the Arch Linux ISO at http://YOUR_FOREMAN_INSTANCE/pub/archlinux.

Installation Media

Go to Hosts → Provisioning Setup → Installation Media and click Create Medium.
Give your medium a name (“Arch Linux Mirror”).
For Path insert the URL to your Arch Linux ISO’s contens (“http://YOUR_FOREMAN_INSTANCE/pub/archlinux”).
Lastly for Operating System Family choose Arch Linux.

Operating System - Part 1

Go to Hosts → Provisioning Setup → Operating Systems and click Create Operating System.
Give your operating system a name (“Arch Linux”).
Give it a major version (“1.0”).
For Family choose Arch Linux.
For Root Password Hash choose SHA512.
For Architectures choose x86_64.

After creating the templates you will have to come back to your operating system.

PXELinux Script

Go to Hosts → Templates → Provisioning Templates and click Create Template.
Give your template a name (“Arch Linux default PXELinux”).

In the editor insert the following:

Arch Linux default PXELinux
<%#
kind: PXELinux
name: Arch Linux default PXELinux
model: ProvisioningTemplate
oses:
- Archlinux
description: |
  The template to render PXELinux bootloader configuration for Arch Linux.
  The output is deployed on the host's subnet TFTP proxy.
test_on:
- host4dhcp
- host6dhcp
- host4and6dhcp
- host4static
- host6static
-%>
# This file was deployed via '<%= template_name %>' template
<%
  timeout = host_param('loader_timeout').to_i * 10
  timeout = 100 if timeout.nil? || timeout <= 0
  arch_iso_url    = foreman_server_url.gsub('https', 'http') + '/pub/archlinux/'
  arch_iso_kernel = arch_iso_url + 'arch/boot/x86_64/vmlinuz-linux'
  arch_iso_initrd = arch_iso_url + 'arch/boot/x86_64/initramfs-linux.img'
-%>
DEFAULT menu
MENU TITLE Foreman Arch Linux
TIMEOUT <%= timeout %>
ONTIMEOUT archlinux

LABEL archlinux
    MENU LABEL Arch Linux
    LINUX <%= arch_iso_kernel %>
    INITRD <%= arch_iso_initrd %>
    APPEND archisobasedir=arch archiso_http_srv=<%= arch_iso_url %> cms_verify=y ip=::: script=<%= foreman_url('provision') %>
    IPAPPEND 2
    TEXT HELP
    Arch Linux - Autoprovision
    ENDTEXT

LABEL archlinux_no_script
    MENU LABEL Arch Linux - No Script
    LINUX <%= arch_iso_kernel %>
    INITRD <%= arch_iso_initrd %>
    APPEND archisobasedir=arch archiso_http_srv=<%= arch_iso_url %> cms_verify=y ip=:::
    IPAPPEND 2
    TEXT HELP
    Arch Linux - Simply the Arch Linux live iso
    ENDTEXT

Let’s take a closer look at the relevant menue entry (resolved):

LABEL archlinux
    MENU LABEL Arch Linux
    LINUX http://foreman.localdomain/pub/archlinux/arch/boot/x86_64/vmlinuz-linux
    INITRD http://foreman.localdomain/pub/archlinux/arch/boot/x86_64/initramfs-linux.img
    APPEND archisobasedir=arch archiso_http_srv=http://foreman.localdomain/pub/archlinux/ cms_verify=y ip=::: script=http://foreman.localdomain/unattended/provision?token=4109fe26-40fd-4821-a839-d626f680d521
    IPAPPEND 2
    TEXT HELP
    Arch Linux - Autoprovision
    ENDTEXT

Here we point to the kernel and initrd provided via http (lines LINUX and INITRD respectively).

In the APPEND line we tell it where to find the the actual ISO’s content and what base directory to use.
We also provide ip=::: in order to keep the current network settings within the live ISO’ environment (IPAPPEND 2 apparently also needed. Further testing needed!).

Last but not least we also tell it to fetch a script to run after booting. This is where the actual magic will happen.

Under Type select “PXELinux Template”.
Under Association associate it with your Arch Linux Operating System.
Save the template.

Partition Table

The configuration files provided by archinstall are simply JSON. It is a bit picky about what information goes into which file.
For this partition table information is provided where it does not actually belong. But this allows us to keep information about the partitioning in one place (within Foreman).
The problem is reconciled later within the provisioing script.

Go to Hosts → Templates → Partition Tables and click Create Partition Table.
Give your template a name (“Arch Linux default”).

In the editor insert the following:

Arch Linux default (Partition Table)
<%#
kind: ptable
name: Arch Linux default
model: Ptable
oses:
- Archlinux
-%>
<%
boot_start   = 3
boot_size    = 512
partitioning = {
   "user_disk_layout": {
      "$disk": {
         "partitions": [
            {
               "boot": true,
               "encrypted": false,
               "filesystem": {
                  "format": "fat32"
               },
               "mountpoint": "/boot",
               "size": "#{boot_size}MiB",
               "start": "#{boot_start}MiB",
               "type": "primary",
               "wipe": true
            },
            {
               "encrypted": false,
               "filesystem": {
                  "format": "ext4",
                  "mount_options": []
               },
               "mountpoint": "/",
               "size": "100%",
               "start": "#{boot_size + boot_start}MiB",
               "type": "primary",
               "wipe": true
            }
         ],
         "wipe": true
      }
   }
}
-%>
<%= to_json(partitioning) %>

Save the template.

Provisioning Script

Go to Hosts → Templates → Provisioning Templates and click Create Template.
Give your template a name (“Arch Linux default”).

In the editor insert the following:

Arch Linux default (Provisioning Template)
<%#
kind: provision
name: Arch Linux default
model: ProvisioningTemplate
oses:
- Archlinux
test_on:
- host4dhcp
- host6dhcp
- host4and6dhcp
- host4static
- host6static
description: |
  The provisioning template for Arch Linux.
  To customize the installation, modify the host parameters.

  This template accepts the following parameters:
  - lang: string (default="en_US.UTF-8")
  - keyboard: string (default="us")
  - additional-packages: string (default=undef)
  - lukspass: string (default=@host.name)
-%>
<%
packages     = [ 'vim', 'openssh' ]
lang         = host_param('lang') || 'en_US.UTF-8'
keyboard     = host_param('keyboard') || 'us'
sys_language = lang.split('.')[0]
sys_encoding = lang.split('.')[1]
lukspass     = host_param('lukspass') || @host.name
ptable_hash  = parse_json(@host.diskLayout)
%>


# Get the very first disk (lsblk) and replace the variable '$disk' in the json strings
disk="$(lsblk --nodeps --paths --output NAME,TYPE | grep disk | cut -d ' ' -f 1 | head -1)"

# Read in disk layout
cat << EOF > user_disk_layout.json
<%= to_json(ptable_hash['user_disk_layout']) %>
EOF

<% if ptable_hash.key?('user_configuration') and ptable_hash['user_configuration'].key?('disk_encryption') %>
cat << EOF > user_credentials.json
{
    "encryption_password": "<%= lukspass %>"
}
EOF
<% end %>



# The actual install settings
cat << EOF > user_configuration.json
{
    "config_version": "2.5.5",
    "debug": false,
    <%- if ptable_hash.key?('user_configuration') and ptable_hash['user_configuration'].key?('disk_encryption') -%>
    "disk_encryption": <%= to_json(ptable_hash['user_configuration']['disk_encryption']) %>,
    <%- end -%>
    "harddrives": [
        "$disk"
    ],
    "hostname": "<%= @host.name -%>",
    "keyboard-layout": "<%= keyboard -%>",
    "bootloader": "grub-install",
    "profile": {
	"path": "/usr/lib/python3.10/site-packages/archinstall/profiles/minimal.py"
    },
    "nic": {
        "dhcp": true,
        "dns": null,
        "gateway": null,
        "iface": null,
        "ip": null,
        "type": "nm"
    },
    "no_pkg_lookups": false,
    "packages": <%= packages -%>,
    "offline": false,
    "script": "guided",
    "silent": false,
    "sys-encoding": "<%= sys_encoding -%>",
    "sys-language": "<%= sys_language -%>",
    "version": "2.5.5"
}
EOF

# Actual install
archinstall --silent --config user_configuration.json --disk_layouts user_disk_layout.json <% if ptable_hash.key?('user_configuration') and ptable_hash['user_configuration'].key?('disk_encryption') %> --creds user_credentials.json <% end %>

# Access to root account (passwords are not well implemented yet)
arch-chroot /mnt/archinstall usermod -p '<%= root_pass %>' root

# SSH (EOW = End Of Wrapper since EOF is used in the snippet)
arch-chroot /mnt/archinstall systemctl enable sshd.service
cat << 'EOW' > deploy_ssh_keys.sh
<%= snippet('remote_execution_ssh_keys') %>
EOW
arch-chroot /mnt/archinstall /bin/bash < deploy_ssh_keys.sh

# DONE - Let Foreman know and reboot
<%= indent(2, skip1: true) { snippet('built', :variables => { :endpoint => 'built', :method => 'POST' }) } -%>
reboot

Under Type select “Provisioning template”.
Under Association associate it with your Arch Linux Operating System.
Save the template.

Operating System - Part 2

Go to Hosts → Provisioning Setup → Operating Systems and click on your previously created operating system.
Under Partition Table select your partition table.
Under Installation Media select your installation medium.
Under Templates select

  • “Arch Linux default” for Provisioning template
  • “Arch Linux default PXELinux” for PXELinux template
    Save the template.

Test It

Now you should be able automatically install Arch Linux onto your machines using Foreman.
Also feel free to leave me some feedback :slight_smile:

Some Sources

3 Likes

Bro, thanks for this tutorial. I have been trying to replicate this, but the thing is I cannot host files in pub folder, tried everything, but browser always says 404 not found.
Which version of Foreman do you have installed and on which system?

braj, i managed to put files in /usr/share/foreman/public/archlinux and can get to them by going directly to http://foreman.mirko.local/archlinux/arch/boot/x86_64/vmlinuz-linux for example

but still when i load the grub menu after PXE booting and choose Archinstall, there is a brief meessage that it could not find anything on http://foreman.real.local/archlinux/arch/boot/x86_64/vmlinuz-linux

i am completely lost and defeated.

I would think that cloud-init might be the best way to automate a Arch install.

can you elaborate this, i am barely familiar with this?

Me either lol, I am still learning.

It is the method that many Linux distros such as Ubuntu etc, are using in the cloud to do automated installs these days which do not have any other advanced legacy automated install method like kickstart or yast available.

Cloud-init lets you pretty much script a system configuration like kickstart and it has modules built in to do do things like run various post scripts, set up the network for you, fix /etc/hosts, handle client repos setup, and more. It is actually quite kickstart like in that it has many modules that do system configuration things for you and you just provide the options, and then you also can add your own custom scripts to it.

The general idea is you copy a base core prebuilt image down onto the hardware, and then provide the cloud-init configuration settings, and it does all the customization.

Based on my playing with Fedora Silverblue, I have a feeling that this will eventually replace the traditional kickstart method of installing RHEL, once RHEL becomes an immutable core os, it will have to switch to the image based install + cloud-init.

Hey, I’ve developed / tested this on

  • CentOS Stream 8
  • Foreman 3.6
  • Katello 4.8

Also, you’ve sent two different links. Maybe you were changing things up when testing but maybe somehow your links got messed up. Just want to make sure :slight_smile:

http://foreman.mirko.local/archlinux/arch/boot/x86_64/vmlinuz-linux

http://foreman.real.local/archlinux/arch/boot/x86_64/vmlinuz-linux

Could it possbily be selinux getting in your way? Even when (supposedly) setting all permissions correctly, selinux often begs to differ.

I might come around to testing again this week. I’ll let you know if it fails for me now, too, or if it still works as before :wink:

Okay, I’ve quickly tested this setup using my (old) Foreman instance (the one I’ve originally tested on) but using a new Arch ISO (2023-08-01).


TLDR

This setup currently does not work without tweaks / rewrites.


They - being Arch - seem to have changed a few things regarding the archinstall script.
I do get my PXE menu, Arch Linux starts up and it starts running the correct script.
The script itself fails however.

This might be a good opportunity to reconsider the way

  • partitions
  • the actual installation

are being handled and how they should be handled in the future.

For now I will just try to kinda update the scripts to the current archinstall as shipped in ISO of 2023-08-01.

I’ll provide updated scripts as I get around to doing so.

great man, i appreciate it. i will try to install it on centos8 with 2023.05.03 iso image and see if i can make it work.

i just created a new system on cent0s with foreman 3.6 and katello. i copied your instructions to the t. again, the same message http://foreman.mirko.local/pub/archlinux/arch/boot/x86_64/initramfs-linux.img cannot be found, but if i copy it to my browser it downloads the file.
i checked selinux on foreman and this is the output:

http_cache_port_t tcp 8080, 8118, 8123, 10001-10010
http_cache_port_t udp 3130

one more thing that occured to me, this provision token might be the culprit. how did you obtain it?

You mean the token in the URL to the script?
This is generated by Foreman for each host individually.
After adding a host, while building this specific host the host can access this URL to obtain the actual installation script.
Within Foreman you can have a look at all the different scripts and how they change for each host respectively (-> Preview).

This still does not explain why your host can’t access http://foreman.mirko.local/pub/archlinux/arch/boot/x86_64/initramfs-linux.img however.

Depending on your network configuration I think this could also be a firewall issue, e.g. you - with your browser - can access the file via the URL, while your client (the one that should be provisioned) can not. You could be within different subnets.

If I recall correctly, the client gets the boot menu via tftp but gets the installation script, kernel and initrd via http.
I’d check firewall settings/ports.

i checked on another machine from the same subnet. it wont resolve foreman.mirko.local link, but when i wget http://192.168.236.25/pub/archlinux/arch/boot/x86_64/vmlinuz-linux it downloads it, so i have adjusted the script to point to IP. still no luck.
these are my opened ports on foreman server:

22/tcp 53/tcp 80/tcp 443/tcp 3000/tcp 3306/tcp 5432/tcp 5910-5930/tcp 8080/tcp 8118/tcp 8123/tcp 8140/tcp 8443/tcp 10001-10010/tcp 67-69/udp

You want to allow DNS?
In most cases DNS uses UDP, not TCP. There are cases where TCP has to be used but UDP would be ‘normal’.
You might want to add it. Although I doubt that it makes any difference here.
Other than this the ports seem fine at first glance.

Would you please double check that under

host → templates → provisioning templates
Arch Linux default PXELinux
→ Preview (here choose the host you created)

the path to vmlinzu-linux is the same as when booting and getting the menu?
While the menu is open, pressing TAB on an entry gives you information about its parameters.

For me it looks like this for example:

Screenshot 2023-08-30 093113

You seem to be missing the token identifier ("token= ").
I would guess that your Foreman does not know that it is supposed to even build this host.

When you check your host within Foreman, what is its status? Is it currently ‘building’? If not, start the build.

When editing

Arch Linux default PXELinux

there could be a typo or something. Or your host is missing some information. Regardless, this script seems to be the best place for you to keep searching.
You could also add other hosts and check the preview on this script.

As I mentioned before, Foreman should create this token automatically.

i had a space before token. now i edited Arch Linux default PXELinux and it is showing now.
when i create a host and go to it’s details i can find the token and i edit the script and paste the token there.
BUT, host is already provisioned with the old script and token showing up in the menu is from the previous build.

Just re-build the built host. Let us know :wink:

there is some error when trying to rebuild host. i had this error message no matter which foreman version i had installed.

  • Danger alert:Error

Content_Host_Status rebuild failed for host: lou-fosher.mirko.local. Refresh_Content_Host_Status rebuild failed for host: lou-fosher.mirko.local.