I've changed the PR test jobs in Jenkins for the main Foreman repo
(theforeman/foreman) to try and improve performance. You'll see that PRs
now receive "foreman", "upgrade" and "katello" test statuses, replacing
"default".
The old [test] phrase changes to the following commands:
"[test foreman]" or "[test upgrade]" to re-test a PR that has failed
tests (as always, use this sparingly please)
"ok to test" to run tests for an unknown user, if the patch is not
malicious (the plugin will unfortunately add three comments to ask, one
per test)
"add to PR whitelist" to always run tests from a trusted user
Aside from these cosmetic changes, the tests should work identically to
before.
Please note that plugins and other repo PR tests remain completely
unchanged for now, I'll look at migrating those if this works well.
This uses the GHPRB Jenkins plugin, replacing our test-pull-requests
script. It has a few advantages, such as running the jobs separately to
reduce the number of slave slots used, and aborting running jobs when a
PR is updated. I hope it will improve the general throughput of PR
tests, particularly under very high load by ensuring more slave slots
are actively running tests.
PR tests will no longer be suspended when test_develop is failing, so
please expect to rebase or re-test PRs after an issue is fixed on
develop instead of relying on it to automatically clear. I don't think
this original behaviour is possible with the plugin.
Commenting on an existing PR that was tested on the old system for
the first time will cause the new PR tests to run too, so expect to see
a few more builds over the next few days. You'll see both new and old
statuses reported on a PR commit if this happens, simply ignore the old
one, it's nothing to worry about.
···
On 03/09/16 14:02, Dominic Cleal wrote:
> I've changed the PR test jobs in Jenkins for the main Foreman repo
> (theforeman/foreman) to try and improve performance. You'll see that PRs
> now receive "foreman", "upgrade" and "katello" test statuses, replacing
> "default".
>
> The old [test] phrase changes to the following commands:
>
> 1. "[test foreman]" or "[test upgrade]" to re-test a PR that has failed
> tests (as always, use this sparingly please)
> 2. "ok to test" to run tests for an unknown user, if the patch is not
> malicious (the plugin will unfortunately add three comments to ask, one
> per test)
> 3. "add to PR whitelist" to always run tests from a trusted user
>
> If you forget, they're on the wiki at
> http://projects.theforeman.org/projects/foreman/wiki/Jenkins#Quick-reference-for-maintainers.
>
> Aside from these cosmetic changes, the tests should work identically to
> before.
> script. It has a few advantages, such as running the jobs separately to
> reduce the number of slave slots used, and aborting running jobs when a
> PR is updated. I hope it will improve the general throughput of PR
There have been some test failures occurring when the base branch
(develop) has been updated while the tests are queued in Jenkins, you'll
see these as git checkout failures. This happens because the PR merge
commit changes and original SHA is no longer available from GitHub.
I've changed the job configurations now to test the PR commit only and
not the merge commit, which should ensure they always work.
If there are changes in develop that you want to test a PR with, please
ask the author to rebase it. Note that this also matches how other PR
tests (such as Travis CI, Hound) already work.
I will probably try and restore the original merge behaviour in future.
···
On 03/09/16 14:02, Dominic Cleal wrote:
> I've changed the PR test jobs in Jenkins for the main Foreman repo
> (theforeman/foreman) to try and improve performance. You'll see that PRs
> now receive "foreman", "upgrade" and "katello" test statuses, replacing
> "default".
>
> The old [test] phrase changes to the following commands:
>
> 1. "[test foreman]" or "[test upgrade]" to re-test a PR that has failed
> tests (as always, use this sparingly please)
> 2. "ok to test" to run tests for an unknown user, if the patch is not
> malicious (the plugin will unfortunately add three comments to ask, one
> per test)
> 3. "add to PR whitelist" to always run tests from a trusted user
>
> If you forget, they're on the wiki at
> http://projects.theforeman.org/projects/foreman/wiki/Jenkins#Quick-reference-for-maintainers.
>
> Aside from these cosmetic changes, the tests should work identically to
> before.
···
On Mon, Sep 5, 2016 at 4:32 AM, Lukas Zapletal wrote:
script. It has a few advantages, such as running the jobs separately to
reduce the number of slave slots used, and aborting running jobs when a
PR is updated. I hope it will improve the general throughput of PR
···
On Mon, Sep 5, 2016 at 12:21 PM, David Davis wrote:
David
On Mon, Sep 5, 2016 at 4:32 AM, Lukas Zapletal lzap@redhat.com wrote:
script. It has a few advantages, such as running the jobs separately to
reduce the number of slave slots used, and aborting running jobs when a
PR is updated. I hope it will improve the general throughput of PR
