RFC: Add intermediate base image to our containers
Decision Due Date: June, 18
Context and Problem Statement
Currently we are building our containers with the base image set to centos:stream9.
Centos containers are constantly updated, and the old images are discarded.
This means we cannot effectively pin the images to a specific SHA version of the container, making the build non-repeatable.
This can also pose a problem if we will want to impose a policy where all our containers should be built from the same base image. This may become important if we would like to support crypto-policies in our containers.
Proposal
Maintain an intermediate base container to be used as the base for all our containers.
This will mean we will have:
It makes sense, but will not solve the problem fully as now the base image is not reproduceable. Not sure if this is a problem for anyone, but could be for downstream depending of what is included in it.
To me this sounds more like basing builds on centos:stream9 is the wrong choice of base OS for a containerized Foreman ecosystem. It sounds like there is a desire for a stable base OS, which is simply not what CentOS stream is?
No, you’d have the very same “problem” of drift if you use debian:13, or registry.access.redhat.com/ubi9/ubi as long as you’re not pinning to specific SHA.
If the old images are in fact discarded from the registry, then CentOS is surely unsuitable for pinning to SHA? Or is it just that image tags are discarded, but not the actual image layer itself?
I know that tags are pointers pointing at images, and what they point at may change. “floating” in your words. But the original claim was that the old images (not tags) are discarded. If that is really the case then pinning against image SHA is useless, since that SHA will be pointing at nothing.
Anyway, I am starting to feel like I am hijacking this thread/going off on a tangent.
It does for Centos:Stream9, the digest from the previous Centos:stream9 is deleted after the new one is pushed, I did check that with @Shimon_Shtein when we talked about this idea of a base image.
We can keep the digest alive by pushing 2 tags for the same digest, centos don’t have a 2nd tag when it pushes centos:stream9 or stream10, that’s why it vanishes from the repo.
Those will be problematic, since we won’t be able to ensure that the base OS is the same for all the containers that we ship.
On the other hand, specifically the postgres container is not exposed directly to the user. How important it is to have it the same crypo policy as others, if it is not exposed?
We will be in control when the foreman base image will be changed. Imagine a situation where we build the foreman container today, but candlepin container tomorrow. If centos decides to replace the image after the foreman build, we will not have the same base for all our images.
If we manage the base image, we will build it once and use it for all the containers, both today’s foreman and tomorrow’s candlepin. We can decide on retention policy and make sure we can rebuild the relevant containers regardless the centos policy.
My point was that you still need to pin to a SHA (or similarly unique-in-time identifier) to ensure you can do such rebuilds as you mention if the “latest” base image was updated by us.
Also, where is this CentOS policy defined? I think we should have a clear reference to be able to say “this is not what we want”.
I didn’t see an official image retention policy, but since centos uses a rolling tag, it means that the older SHA will become an orphan, so it will be a valid candidate for garbage collection.
Tomorrow if we get one new image from Centos:Stream9, the one above will vanish and we will lose the reproducibility unless we build the image using the Source Container that we push to quay as part of the Konflux pipeline.
