RFC: Applying crypto policies to Foreman containers
Context and Problem Statement
Containerized Foreman needs a way to apply the host’s selected crypto policy to its services. Ideally, users would continue using update-crypto-policies on the host, then run foremanctl deploy to apply the change to their containers.
We cannot simply mount the host’s generated files into every container. Those files depend on the OS and crypto libraries that generated them, and we need to support configurations such as EL10 containers on EL9 hosts. Podman provides automatic handling for plain FIPS, but does not propagate other policy selections or FIPS subpolicies.
This proposal covers configuration managed by update-crypto-policies. Custom-written policies, application-specific TLS configuration, and automatic synchronization outside deployment are out of scope.
Proposal
I’m proposing that each container image generate its own crypto-policy configuration. This avoids requiring every image to use exactly the same base OS version.
During foremanctl deploy, we would:
- Read the host’s selected crypto-policy string, unless the user supplies a container-policy override.
- Start a temporary container from each distinct target image, using the exact image version that will run the service (or that is already in storage).
- For existing deployments, this step uses the container image that exists in storage.
- I originally wanted to use the running container itself to generate the crypto files, however,
podmandoes not support adding bind-mounts to already running containers. The container thus needs to be recreated anyway, so it’s simpler to always use the image first to generate the crypto config files, and then create the permanent container after.
- Run
update-crypto-policiesas non-root via the ephemeral containers, usingbase_dirto write into a temporary directory (and--no-reloadsince it’s pointless in an ephemeral container). - Export the generated crypto configuration to the main Foreman host machine.
- Run the containers with their respective crypto policy bind mounts.
Configuration would be regenerated when images change. Each relevant image must provide update-crypto-policies and its execution dependencies.
Policy compatibility
A host selection may not exist in the container’s OS. For example, an EL9 host could use LEGACY:NO-SHA1 while an EL10 image lacks NO-SHA1.
In that case, deployment would fail with an explanation and direct the user to a new container-policy override option. The override would be persisted, with some way to return to host inheritance.
Matching policy names means using each image’s native interpretation, not guaranteeing identical restrictions across OS versions. This is a compromise to deal with the complexities of trying to sync up crypto policies between different OS versions.
FIPS
We would use the same generation path for FIPS and supported FIPS subpolicies. Explicit mounts at /etc/crypto-policies/config and /etc/crypto-policies/back-ends would take precedence over Podman’s automatic plain-FIPS mounts without disabling the underlying FIPS runtime state.
This mount behavior needs testing on supported Podman versions. Overrides must remain compatible with the deployment’s FIPS requirements.
Take a look at container-libs/common/pkg/subscriptions/subscriptions.go at c231aab00595fbd79ae05d8f01028f6e49b2e999 · podman-container-tools/container-libs · GitHub for some comments on FIPS from podman.
Alternative Designs
- One shared generator image: Reduces the number of ephemeral containers that must run
update-crypto-policies, but requires compatible OS and crypto-package versions across the generator and all consuming images. A common base image is expected to be created regardless, however one shared generator image forces every container image to update together when there is an OS update. I also think it requires this common base image to be a prerequisite development item. I’d like to start development of this feature sooner rather than later to make the Foreman 5.2 deadline. - Build locally configured image layers: Instead of bind-mounting the generated crypto policy files, build them into a new layer in the container image. Provides a place to install missing tools and bake in configuration, but adds image-building and lifecycle management. We can revisit this later if the bind mounting introduces unexpected issues.
- Generating crypto config from already-running containers: I wanted to avoid ephemeral containers altogether, but as discussed above, bind-mounts must be declared at container image running time. Locally rebuilding container images with crypto config is one way to avoid this bind-mounting complexity, but we must also consider the overhead of building container images. At the moment, I’m assuming that local container builds are slower, but I have no data to back that up.
Decision Outcome
TBD, following community feedback.
I want us as a project to begin implementing the solution for this overall crypto-policy-matching issue sooner rather than later to ensure that the Foreman 5.2 deadline is not a problem.
Impacts/Prerequisites
- IoP images need the
crypto-policies-scriptsRPM. - FIPS testing is necessary to qualify the proposal up above.
Open questions
- When do we reload the crypto config?
- The crypto config needs to be regenerated when container images are updated, and when the requested crypto policy changes. Is it easy via
foremanctlto detect those specific situations? Or, since updating those files may be cheap, is it better to simplify the workflow and always regenerate the crypto config atforemanctl deploytime?
- The crypto config needs to be regenerated when container images are updated, and when the requested crypto policy changes. Is it easy via
- What do we do if an image we do not control stops shipping the
crypto-policies-scriptsRPM in the container?- Perhaps we can either start maintaining that container ourselves since we need a joint base container anyway. Or, we can look into how local rebuilds can solve this.