Containers: Project Management

Development RFCs
,
1

Project Management

Project Name

The current project lives under the umbrella of the Forklift project. Given the scope of the containerized effort, this should likely be moved within it’s own project structure. Proposed new names:

  • Forklift 2.0 (same name and project, fresh vision)
  • Excavator
  • Pylon
  • Mast (https://www.eureka-forklifts.com.au/info/container-masts/)
  • Stevedore
  • Longshoreforeman
  • Dockworker
  • Dockhand
  • Loading Bay
  • SkidSteer - I like the two syllables, similar to forklift
  • Grapple
  • StraddleTruck - I really like how this device is carrying the containers - we could probably call it Straddler (bonus is there is not such a project on github, unlike the previous two names)
  • Spreader - containers again
  • BallastTamper - I like the irony here + not existing github project
  • Lowboy - There is a risk we would need to rename it to Lowbot at some point

Change Management

  • Use Github issues on either Forklift or yet to be named new project
  • Changes should follow standard PR + maintainer approval workflow
  • Changes should attempt to add tests for new functionality

Developer Community

Developers should be encouraged and able to develop using a containerized deployment that models the production deployment of containers as closely as possible. Enabling developers to use a container deployment in day to day operations provides value by:

  • Developers become more comfortable with container deployment
  • Developers are more likely to contribute fixes back the containerized deployment
  • Developers would developing in a nearly production environment reducing environmental bugs

User community

Users are the main target for use of a containerized deployment through testing and use in their production environments. The change to containers presents not only a physical shift in user infrastructure but more so a mental shift in how to think about a new deployment model and net new knowledge in how to operate the application managing their infrastructure. In some cases users will be looking to deploy the new containerized platform in their local or company production environments. In a lot of cases, users will need to upgrade their traditional deployments to a containerized version. Further, upstream users have a variety of deployments from vanilla core Foreman to Foreman with some combination of simple plugins to Foreman with complex plugins.

Concerns that need to be accounted for with user community:

  • User education on container orchestration technology, concerns for deploying and how to manage it
  • Upgrades of existing deployments to new platform

One of the major concerns with the transition to a containerized deployment for the user community is gaining their support that this new direction makes sense as the next evolution of Foreman. To enable this, the community needs to be shown through clear documentation and communication the value for individual users and the community as whole (users and developers). Further, users should be presented with tooling that aides in the setup, migration and maintenance of this new deployment model. Users will need tools to aide them in building a container or combination of services for their needs (e.g. provides their set of plugins), deployment tooling, handling upgrades and configuring the environment to their requirements.

User value communication:

  • Smaller, faster updates for bug fixes and features
  • Smaller, lightweight plugin/services for customized infrastructure management per user need
  • More robust releases via development on production like environments
  • Customizable scaling based on user workloads

In order to communicate the shift in direction, the value add and keep a dialog with the user community, the following activities should be undertaken:

  • Regular contributions to community demo showing current state of container work
  • Regular communication of container development plan and activities to the mailing list
  • An upstream repository of documentation related to containers, design, and how to contribute
  • Engage early adopters to test container deployments in their infrastructure

Partner Communities

The Formean project and plugins integrate with a variety of services and in some cases use them as backends for off loading critical workloads. The Katello plugin in particular makes use of and collaborates heavily with Pulp, Candlepin and Qpid projects. In the traditional, puppetized deployment of Foreman, puppet modules were developed to configure the backend services within the Foreman community without collaboration with these partner communities. Each project continued to handle RPM builds but throughout the history Katello has carried the projects RPMs in it’s repositories at one time or another.

As the containerization project progresses, artifacts will be created that would provide benefit for our partner communities. These are artifacts such as container images, deployment configuration onto an orchestration engine, Helm charts or operators, etc. In other words, this effort will produce functionality necessary to run and operate a partner communities technology as a stand alone deployment. This provides value to the partner community by giving users a way to deploy a given project via containers and onto an orchestration system. This potentially provides opportunities for partner communities to upgrade to the Foreman community by bringing Foreman services online with their existing services.

The project should strive to engage our partner communities for engagement in and consumption of containerized artifacts. This can can come in the following formats:

  • Make partner communities aware of our efforts through mailing list posts and demos
  • Encourage contribution from partner communities through efforts to ensure partner services run stand-alone
  • Engage developers from partner communities to take ownership of parts relating to their projects
  • Work cross-functionally with members of partner communities during all stages of the SDLC
2

Apologies for the bikeshedding… “Stevedore”, “Longshoreforeman” “Dockworker”, “Dockhand”.

In general what I understand from all your RFCs is that we would be using very generic, commonly used. The only exception is the Operator framework which I don’t expect most people to know. Why are we using it? Does it improve consumability of Foreman by end-users, or is it mostly a choice that affects only the Operator developers?

3

Overall I’m OK with this. Forklift is getting overloaded and rather big. To be concrete: you suggest a separate git repository?

Not a big fan of this.

Neutral

Given https://pylonsproject.org/ already exists :-1:

Neutral about this.

Other option: loading bay? I avoided loading dock because of the strong connection to docker.

Bikeshedding aside, I don’t have a strong opinion on the actual implementation (kubernetes, openshift, etc). I do like treating external services really like external services rather than as a part of the deployment. They should be able to scale independently. That means ideally an expert on Pulp should be able to maintain the installation and Katello should be able to consume this. It should also be less tightly coupled in terms of upgrades. Katello should only state a minimum supported version and you can upgrade to a newer version without upgrading Katello. Your proposal suggests you’re thinking about it in a similar way but let’s be explicit :slight_smile:

4

To avoid confusion, I’d suggest to pick something else. I have no preference, letting this on folks who will actually do the work.

5

Naming conversation, here we go:

https://en.wikipedia.org/wiki/Heavy_equipment#Types has wide variety of names to look at :slight_smile:

I like:

  • SkidSteer - I like the two syllables, similar to forklift
  • Grapple
  • StraddleTruck - I really like how this device is carrying the containers - we could probably call it Straddler (bonus is there is not such a project on github, unlike the previous two names)
  • Spreader - containers again
  • BallastTamper - I like the irony here + not existing github project
  • Lowboy - There is a risk we would need to rename it to Lowbot at some point :slight_smile:

I have to say I enjoyed looking at all those awesome machines: thanks @ehelms for giving us the opportunity :slight_smile: Now back to work

6

You never have to apologize for that with me. Naming is half the fun :slight_smile:

7

Thanks for the digging and sharing of more ideas! I included these and @dLobatog suggestions in the main post for others to consider.

8

To some degree yes. Being explicit as you ask though, I am not suggesting this as part of this work. I don’t think the two require each other to become a reality and that it’s well understood that making independent Pulp updates a reality requires changes to Katello not the deployment strategy. I am suggesting that we work with a community like Pulp so we can consume their deployment methods and artifacts to deploy the version Katello needs. In that specific case, Katello still needs to control which version being used and is tested against but not necessarily how Pulp is deployed. By sharing resources on deployment artifacts, both communities benefit.

This gets a bit off topic but I think it’s worth re-iterating, to allow Katello to work with multiple versions of Pulp, or a user provided Pulp requires some fundamental changes to both Katello and Pulp. For example, there is no multi-tenancy in Pulp and Katello builds higher level abstract concepts on top of Pulp building blocks. We can get there if we feel this would be beneficial, but there is work to be done outside the scope of containers.

9

Yes, one of the unstated goals is to use as much commonly accepted within the Kubernetes ecosystem tools and methods for the effort to reduce barrier to entry and uniqueness. The Operator idea is older than most people think but has only recently been getting large amounts of press due to the release of the operator-sdk and Operator Lifecycle Management. The idea of an Operator is a Kubernetes native application that manages your application in a Kubernetes native way. The operator-sdk is a toolkit designed to help developers build operator’s for their applications.

A few release back Kubernetes introduced the idea of a custom resource defintiion. You can think of this as a way to define your own Kubernetes resource. That is, you can define your application just like you might define a pod or deployment and create instances of that resource. An operator is then the nuts and bolts that keeps an eye on your resource and ensures that it’s current state matches the state you told Kubernetes it should be in. For example, if you created a Foreman resource instance named ‘myforeman’ and indicated you wanted 3 dynflows as part of the instance. The operator would ensure 3 dynflows existed. If something killed 1 of the dynflow instances, the operator would reconcile the state of the world and spin another dynflow up.

This is the basic example of what an operator can do. There are projects that use operators to handle backup and restore of their applications. One way of thinking about them I have heard is turning the operation of your application into software. The target audience is the person who is administrating the operation of the Foreman instance.

10

A variety of names have been proposed to name this overall container effort and any artifacts that come from it. I am putting up a poll to let folks vote on the new name with a selection from the larger list above.

  • Excavator
  • Pylon
  • Mast
  • Longshoreforeman
  • Dockworker
  • Loading Bay
  • SkidSteer
  • Grapple
  • StraddleTruck
  • BallastTamper

0 voters

11

@ehelms: What about spreader?

12

I can add that back it just has a lot of projects on github named that already.

13

It’s all right, I just love the name. But I guess I’m not the only one.

14

Giving this a gentle bump for more developer votes:

15

The poll having been open nearly a month landed with the highest vote count going to Grapple. I plan to create a Grapple repository on theforeman organization and begin moving existing container work to that repository and out of Forklift. If you disagree with this approach speak now or forever hold your piece.

My idea for approaching this based on feedback across the threads:

  • Split the forklift containers/ directory as a subtree (to retain history) and put under a prior_work folder as an initial commit to grapple
  • Treat grapple as a fresh repository otherwise
  • Start by “moving” the Dockerfiles to a top level directory one by one via PRs to be reviewed by those interested
  • Use this “fresh” approach to add CI from the get go for every aspect

My goal is to preserve the bulk of prior work but with an approach towards adding things a new to allow easier entry for those interested in contributing rather than doing this on the side as a blackbox. The other part of this is looking for others interested in actively participating in Grapple for reviews, code contributions, CI buildout, design, etc.

1 Like