RFC: Automatically Generated Foreman Component Compatibility Matrix

RFC: Automatically Generated Foreman Component Compatibility Matrix

Decision Due Date: None (Request for Feedback)

Context and Problem Statement

Foreman and Katello are actively developed projects with regular releases and upgrades.

When administrators plan upgrades, troubleshoot issues, or validate repository configurations, they may need to determine which versions of related components are intended to be used together for a given release.

Examples include:

  • Foreman

  • Katello

  • Candlepin

  • Pulpcore

  • Java runtime requirements

The information already exists within project metadata and packaging sources. However, discovering the expected component combination may require navigating multiple repositories, branches, packaging files, and release metadata.

As an example, determining the expected component versions for a Foreman/Katello release may involve:

  • Identifying the correct branch

  • Locating the relevant packaging repository

  • Determining which metadata files are authoritative

  • Comparing multiple version definitions

In practice, multiple administrators may independently perform the same investigation.

This can lead to:

  • Increased troubleshooting effort

  • Incorrect repository selection

  • Misidentified compatibility issues

  • Community support requests that ultimately turn out to be unsupported component combinations rather than software defects

The objective of this proposal is not to introduce new compatibility data, but to improve discoverability of information that already exists within the project.

Technical Objectives

  • Provide a single reference for expected component versions.

  • Reduce compatibility-related support discussions.

  • Improve release planning and upgrade validation.

  • Avoid manual maintenance whenever possible.

  • Derive information from existing project metadata.

Non-Goals

This proposal does not:

  • Change supported installation methods.

  • Introduce new support commitments.

  • Define new compatibility policies.

  • Replace existing installation documentation.

  • Provide documentation for disconnected deployments specifically.

Proposal

Generate a compatibility matrix automatically from existing release metadata.

Example output:

Foreman | Katello | Candlepin | Pulpcore | Java

3.19 | 4.21 | 4.7.x | 3.105.x | 17

5.1 | 5.1 | 4.8.x | 3.x | 25

``

The matrix could be generated from metadata already maintained by the project, such as:

foreman-packaging

katello-repos

release packaging manifests

Possible publication formats:

  • Markdown document in project documentation

  • Generated page on the website

  • Generated release artifact

The key requirement is that the matrix is automatically generated from existing metadata rather than maintained manually.

Technical Impact

Positive impacts:

  • Improves discoverability of release compatibility information.

  • Reduces duplicated investigation work.

  • Provides a common reference during troubleshooting.

  • Helps distinguish unsupported version combinations from actual software defects.

Maintenance impact:

  • Minimal if generation is automated.

  • No additional compatibility data needs to be maintained manually.

Project impact:

  • No changes to release processes are required.

  • No changes to support policies are required.

Alternative Designs

Continue Using Existing Metadata Sources

The current information already exists within project repositories.

Advantages:

  • No additional work required.

  • Existing workflow remains unchanged.

Disadvantages:

  • Requires knowledge of packaging internals.

  • Information is distributed across multiple sources.

  • Multiple users may repeat the same investigation process.

Manually Maintained Compatibility Documentation

A manually maintained compatibility page could be created.

Advantages:

  • Simple to understand.

  • Flexible presentation.

Disadvantages:

  • Introduces ongoing maintenance burden.

  • May become outdated.

  • Requires updates for every release.

For these reasons, a generated solution is preferred.

Decision Outcome

Pending community feedback.

Potential outcomes:

  1. No change.

  2. Community-maintained generated matrix.

  3. Integration into project documentation.

  4. Integration into release generation workflows.

Impacts

If adopted:

  • Administrators gain a single compatibility reference.

  • Release validation becomes easier.

  • Troubleshooting effort may be reduced.

  • Compatibility-related support discussions may decrease.

Limitations:

  • Depends on the availability and consistency of release metadata.

  • Reflects documented metadata only.

  • Does not replace release notes or installation documentation.

I think generally it’s a good idea and something we can provide. As we move towards containers, I feel like a lot of this should be alleviated as the container image will be the source of truth, and our intended tagging structure will indicate this. See foremanctl/docs/developer/container-image-builds.md at master · theforeman/foremanctl · GitHub

While we’re at it, maybe we can add the supported OS versions, Ruby versions for Foreman/Smart Proxy/Katello, and Python for Pulp.

I like that foreman-documentation has a PR template with at least Foreman+Katello+downstream versions; and there’s matrix.json in foreman. Looking at it, we could also provide the NodeJS versions and PostgreSQL versions. I also looked at python-pulpcore.spec in the past.