I did two things to the Jenkins PR test "system" today which should
reduce the time between opening or updating a PR and it starting to
build it.
The first was adding some retry logic to the test-pull-requests script
(the repo for which I've moved into our GitHub org). Because it was
making so many GitHub API requests, it only needed one to fail causing
the entire run to error out, meaning another 15 mins until the next
attempt. Sometimes this could mean it was out for a couple of hours.
The other was to add hooks to our GitHub repos which trigger the PR
scanner script for an individual project when a PR event happens.
Thanks to Eric for the idea. This means when a PR's opened, the scanner
script runs pretty much instantly, so builds will usually trigger within
a minute or two of the PR being opened.
I've started on some general Jenkins setup docs here too:
http://projects.theforeman.org/projects/foreman/wiki/Jenkins