RFC: Image mode host management in Katello

Development RFCs
,
1

Hello community,

As alluded to in my Katello future feature development updates section in the last community demo, Katello is preparing to add support for managing image mode hosts. This new host type involves using bootc to deploy “bootable containers” as immutable Linux systems. More information can be found here: Getting Started with Bootable Containers :: Fedora Docs

The implementation proposal for the next couple of releases split into two goals:

Connect and Analyze

  • Identify image mode hosts
  • See the staged, booted, available, and rollback container images

Update

  • Conveniently trigger remote bootc actions on hosts
  • Identify bootc systems that can upgrade
    • Consideration: image mode hosts do auto-update
  • Easily view booted container images and the hosts running them

Here’s the current design plan:

Connect and Analyze:

  • Prerequisite: update container image naming scheme to use slashes instead of dashes
    • Since a long time ago, Katello distributes container repositories with dashes between the organization, product, content view, etc. Slashes should be used instead since that is the new standard. It’ll make container repositories more readable on podman search.
  • Collect bootc facts from subscription-manager and add them to new bootc fact attributes on content facets.
    • Facts: image path (e.g. katello.example.com/org/product/centos9-bootc) + digest for:
      • staged, booted, available, and rollback container images
    • Try to associate the booted image to a container manifest / manifest list in Katello. We’ll likely need a polymorphic relationship here because manifests and manifest lists are separate in Katello.
    • For facts relaying, bootc is being requested to make a facts file for sub-man under /etc/rhsm/facts/
      • The backup plan is to have sub-man collect the facts for us. There is no other reliable transport mechanism (today) on registered hosts besides subscription-manager.
  • Differentiate image mode hosts from other hosts
    • The existence of a booted container image from bootc facts will determine this.
    • The API and Hammer will report if a host is an image mode host, likely via an image mode boolean attribute
    • The new all-host listing page will have a column option to view the host type (image mode or not)
  • See collected bootc fact information on hosts
    • The API and Hammer will return host bootc fact information for hosts via the content facet
    • The UI will have a new card on the host details tab for showing image information.

Update

  • Add a new page (“deployed images”) in the UI that shows a list of all booted images aggregated from host bootc facts
    • For each image, if clicked, open up a table (side panel like with alternate content source details?) of all hosts running that image
  • Disable package and errata actions on image mode hosts
  • Add new remote execution actions for bootc image update actions:
    • bootc upgrade
    • bootc rollback
    • bootc switch (this will require input to choose the new target image)
  • Bonus: display in the UI if a host’s image is stale
    • We can tell this if the host is consuming from Katello’s registry. We’ll just check if the digest associated with the host’s booted image tag matches the digest of the manifest associated with the same tag in the same container image in Katello’s registry.

Notes and Questions

  • Can we get away with not associating host booted images directly to Katello synced container manifests / manifest lists?
    • We cannot assume that hosts will be using images from Katello’s registry. So, the benefits of tracking synced image associations would have limits
    • The bonus we’d get here is an easy way in the future to display host/image relationships on container mainfest UI pages
    • There’s an overhead of keeping track of the association with both time and code complexity
    • In the current design, the only feature that requires an image mode host to container manifest relationship would be telling if a host’s current image is stale. However, this is considered a stretch goal
  • Will using host bootc facts entirely to populate the new “deployed images” page suffice?
    • We’ll need to ensure that the facts are properly indexed in the database so that searching them is fast
    • Queries:
      • List the images (aggregate + make unique)
      • Find all hosts related to a specific image may be expensive 
        • * Query would look like "fetch all hosts where bootc_booted_image == 'image'"
    • Pagination + loading the host list only when clicking on a specific image may help with performance.
  • Why is this in Katello and not Foreman?
    • subscription-manager is currently identified as the best way to get bootc facts into Foreman/Katello. Other options are remote execution (not installed in all environments) and making a new host service to provide the facts (which we would not want to make mandatory to install on clients).
    • Image mode hosts are powered by containers, and Katello has a container registry that users will likely use to feed their image mode hosts. So, if we couple image mode content facets to indexed container information, it’ll be easier to add new features in the future.

I appreciate any feedback for this new feature. Image mode hosts are a new concept that people are still getting used to, so feel free to ask any general questions about the technology here as well.

Thanks!

3 Likes