Foreman-bats and Jenkins

Some new testing of RPMs published to yum.theforeman.org has been added
to Jenkins. It uses the foreman-bats project started by lzap to run a
basic smoke test of our packages, the installer and hammer:

https://github.com/theforeman/foreman-bats

Jenkins is configured to run this as part of the RPM "publish" process
(the packaging_publish_rpm job), so a repo is created on koji (e.g. the
nightly or a release repo), this job is called and runs repoclosure,
then foreman-bats and then publishes it to yum.tf.org if successful.

http://ci.theforeman.org/job/systest_foreman/

The Jenkins job is using Vagrant and the vagrant-rackspace plugin to run
foreman-bats on the Rackspace public cloud. Many thanks to Rackspace,
who have donated this account to the project.

Some more info about the setup and links are here:

This isn't a replacement for tests in our projects, but if there are
additional smoke tests that we should include, please send PRs to
foreman-bats. Hopefully it'll increase the quality of our nightly
builds, and also save us a lot of manual smoke testing when doing releases.

The same is nearly ready for Debian as a nightly testing job (see
systest_foreman_debian and #3934).

I'll show this in tomorrow's sprint demo too.

ยทยทยท -- Dominic Cleal Red Hat Engineering