Changes to Jenkins and PR testing

Recently I've been working on Jenkins and in the last week, testing of
open PRs. Some of you are probably more than aware of this, having been
spammed during my recent efforts :slight_smile:

So just a quick summary of what we're now doing in terms of testing:

foreman "develop", matrix job testing Ruby 1.8.7, 1.9.2, 1.9.3, MySQL,
PostgreSQL, SQLite, Puppet 2.6 and 3.0.
http://ci.theforeman.org/job/test_develop/

smart-proxy "develop", matrix testing Ruby 1.8.7, 1.9.2, 1.9.3, Puppet
2.6, 2.7 and 3.0.
http://ci.theforeman.org/job/test_proxy_develop/

Any opened PRs now get tested through the same system, with the patch
being applied to the latest develop HEAD. The matrix is almost the
same, but with the Puppet axis removed for the main foreman project to
reduce the numbers.

We're using a modified version of the test-pull-requests script from
OpenShift:
https://github.com/openshift/test-pull-requests
https://github.com/domcleal/test-pull-requests

This will test any PR from a "trusted" user (member of theforeman or
Katello organisations), or a PR that has a comment containing "[test]"
from a trusted user. I'd like to get regular contributors added to the
org so they're tested automatically (Ohad?).

The bot will note the commit sha that's been tested on the PR, plus link
to the test results, which are kept for 45 days.

There's also the deep dive video from this week if you want to hear more
about it. Let me know if you have any questions.

··· -- Dominic Cleal Red Hat Engineering