Installation via Containers and Podman Quadlets
Problem Statement
In order to run Foreman on a given operating system, hundreds of native packages need to be created and then tested. This has to be repeated for every operating system the project wishes to support. Across the ecosystem of projects included in Foreman installations there are multiple languages involved, Ruby, Python, and Java to name a few. For each new operating system the Foreman project must ensure each of these has a corresponding runtime, and dependencies, present to operate which makes onboarding multiple operating systems cumbersome.
The current installation, when upgraded many times, often suffers from packages that linger on the system and trigger security scanners. There have been times where projects with the same runtime language, but requiring different versions, for example Pulp and Ansible, have caused conflicts for installation.
Proposal
Align on using containers as our deployment artifact, taking a build once and deploy many approach. This would cut down on our build overhead, while opening up more deployment possibilities. In order to maintain continuity, initially the containers would be built with RPMs instead of source. This provides the project the ability to maintain the same build artifacts, for traditional and container based, while developing the container-based installation.
The approach will be to first encapsulate our current systemd services as containers running as Podman Quadlet’s that can provide a common service runtime framework. This will enable making the deployment more operating system agnostic, avoiding clashes with the operating system and providing a path to a production level Kubernetes deployment.
There is on going research into this idea occurring at GitHub - theforeman/foreman-quadlet repository. This repository uses light-weight Ansible to orchestrate and allow development to focus on the container and quadlet piece.
Considerations
The research up to this point has identified some major areas of additional investigation and open questions that will need answered. These are tracked as issues. There are some major ones worth highlighting here:
Installer
A container and quadlet model introduces a new way of thinking about our deployment and the services running within it. This presents the question of whether the existing puppet-based foreman-installer should be adapted or if we should start with a fresh installer focused on this deployment model free of the past. Evolving the current installer allows for progressive enhancement but at the cost of coupling multiple deployment methods together. A new installer allows us to start with a fresh approach, aiming for simplicity and a container-native approach but comes at the cost of having to upgrade existing installations and translate the current installer.
Upgrades of existing setups
Regardless of the installer decision, we will need to research and plan how the upgrade path for existing installations will look like.
Development Environment
We will need to ensure that this is built with developers in mind to ensure that an environment can be created for production and development. The goal being to ensure that they mirror each other closely, while providing a fun and efficiency environment for developers.
Packaging
The containers used for research were built based on our RPM packages, but could also be run on other operating systems (like Debian or even SUSE), as long as that operating system can run containers.
One could argue this makes dedicated (Debian) packaging obsolete, so time and effort could be saved. Later on also RPM packaging could be re-considered.
Smart Proxy
The initial container work will focus on Foreman and not the smart-proxy itself. The smart-proxy has many considerations with how it interacts with external services, and handles plugins. This will need it’s own dedicated exploration.