I am asking myself the same question: Do we want this? In my opinion, Foreman should be the tool to set up a kubernetes cluster. That is going to be hard, when you already need a working cluster to deploy your Foreman instance. How are we going to address this chicken-and-egg problem? Maybe foreman-installer could setup a one-node kubernetes cluster (via puppet modules) and deploy foreman in there? Just a thought.
We decided not so deploy Foreman in our Kubernetes Cluster to reduce the blast radius if we have issues with one of the systems. Let’s say we need to redeploy our broken Kubernetes Cluster, we can currently still do that because Foreman runs outside of the cluster. If we’d deploy Foreman inside Kubernetes, we’d be screwed.
Another concern I have is if the software is actually ready for containers. We tell our developers that their apps need to follow the 12factor rules. When we looked into deploying Katello on multiple hosts, there were some architectural issues we hit. I think we should address these before moving to containers. Also Katello relies on a lot of apache configuration. I think we should talk about moving these to the rails app to reduce the dependencies on the web server configuration.
To sum up, I’m still pretty reluctant on this topic. Kubernetes is definitely the new industry standard for container deployments. But Foreman needs to happen before containers in my opinion. And I think the app (core + common plugins) is not ready for the move yet.
Happy to hear more thoughts on these topics.