Background
Forklift started as a way to easily get production testing environments and development configurations setup for developers and to some degree users. The target was using Vagrant, a well known developer platform, and single machine setups. Over time the project expanded to some limited multi-host topologies but never anything truly past Vagrant with Libvirt. This has served us well to this point.
Problems and Limitations of Current Design
- Vagrant focused
- Good at single node, bad at multinode
- Starts with Vagrant but meat is in the Ansible
Proposal: Bring Your Own Inventory (BYOI)
With the influx of users and use cases I think the time has come to expand Forklift by modifying the workflow to allow for myriad use cases. The proposal is to modify the workflow of Forklift to a bring your own inventory model with some limited support for âeasyâ mode via Vagrant.
The way Forklift works today assumes an all in one solution of provisioning a machine, then configuring the machine and providing facilities to destroy that machine. This proposal attempts to split this into:
- Provisioning capabilities that produce an Ansible inventory somehow
- Playbooks and roles that are the bulk of the configuration
Provisioning
The focus on provisioning will no longer be the forefront of Forkliftâs intent but it would still provide the current baseline functionality and perhaps other tools in the future. In the case of Vagrant, the project would shift to providing the ability to spin up a Vagrant box, thereby producing an Ansible inventory through the dynamic Vagrant inventory. The user would then be responsible for configuring that box with one of the available playbooks. This would expand the one command today into two:
vagrant up centos7
ansible-playbook -l centos7 playbooks/devel.yml
Another example would be someone who acquires a machine through beaker. The user would provision that beaker machine through some-tool and write out an inventory (or using a dynamic inventory):
acquire-beaker-machine
echo "[katello]\ntest.example.com" > inventories/my_beaker
ansible-playbook -l katello playbooks/katello.yml
Configuration
The configuration of machines would be a continued focus on self-contained and focused roles with a general set of playbooks combining those roles for use. Facilities would be provided to allow users to configure their own setups in three ways:
- User defined playbooks
- Inventory configurations
- Common and user supplied vars files
User defined playbooks would expand on the existing user playbooks to allow users to define playbooks locally and not risk committing them to git. Playbooks that can be generalized for a broader audience would be encouraged to be committed to Forklift.
Inventory configurations would involve documenting how to create inventories that also define the variables one needs to customize a deployment with. This might be something akin to configuring a development environment with your own machine:
cat > inventories/devel_inventory <<EOF
[devel]
test-dev.example.com
[devel:vars]
devel_github_username=ehelms
katello_repositories_environment=staging
EOF
Common and user supported vars files would be allowing playbooks to remain general (e.g. katello.yml) and providing common configurations through vars files that can be used to customize the playbooks. This would also be at the user level for common configurations a user wants locally. An example might be wanting to have production nightly and staging nightly installations:
$ cat vars/nightly_staging.yml
---
katello_repositories_environment: staging
foreman_repositories_environment: staging
$ ansible-playbook -l centos7 playbooks/katello.yml -e @vars/nightly_staging.yml
Looking Ahead
Looking ahead, this shift will allow us to provide a wide range of users and compute resources to be used for environments. Advanced cases such as multi-host topologies can be more readily brought to bear by adding tooling to spin that up and generate an inventory or having a user bring their own. For example, a tool like Linchpin or Terraform could be used to spin up multi-host or even our own custom Vagrant forklift role. For example, say we have one of those spin up a multi-host setup and produce an inventory like:
[server]
server.example.com
[proxy]
proxy.example.com
[clients]
client-el7.example.com
client-deb.example.com
We can then have a playbook that is designed to target host groups server, proxy and clients to configure, orchestrate and install a server, proxy and set of clients for testing. A user could do this individually as well, e.g.
ansible-playbook -l server playbooks/katello.yml
ansible-playbook -l proxy playbooks/proxy.yml
ansible-playbook -l client-deb.example.com playbooks/puppet_client.yml
ansible-playbook -l client-ell7.example.com playbooks/client.yml
As we get into a more container oriented world, this focus on the roles and playbooks opens up the possibility to simply shift the focus of the âmachineâ to that of a container and still apply the same set of core logic built up in the roles and playbooks.
Feedback
Obviously, I am looking for feedback on this proposal as a whole but I am also looking for feedback on user and developer experiences. Things like:
- Does this proposal help meet your needs?
- Are there changes in Forklift youâd like to see that this does not help with?
to making ansible-playbook the primary interface. To me linchpin looks like a much more complete alternative. It has multiple providers that appear to cover the alternatives we currently use (libvirt, openstack) and those desired (beaker) although it is missing an easy way to get a console. Also not sure about the pure bare metal case.