[Infra] Proposal to Move Jenkins Job Configurations

Howdy,

Some quick background, for those that don't know a majority of our Jenkins
job configuration are stored in git. I find our Jenkins job configuration
to be important for developers to be able to understand and contribute
changes. However, today, these configurations are stored deep inside the
foreman-infra [1] repository. I am proposing one of two ideas to bring
these to the forefront:

  1. Move them to the top of foreman-infra under a "jenkins" or "jobs" folder
  2. Move them to their own repository (e.g. foreman-ci)

[1]
https://github.com/theforeman/foreman-infra/tree/master/puppet/modules/jenkins_job_builder/files/theforeman.org

··· -- Eric D. Helms Red Hat Engineering

so 2. 9. 2017 v 1:27 odesílatel Eric D Helms <ericdhelms@gmail.com> napsal:

> Howdy,
>
> Some quick background, for those that don't know a majority of our Jenkins
> job configuration are stored in git. I find our Jenkins job configuration
> to be important for developers to be able to understand and contribute
> changes. However, today, these configurations are stored deep inside the
> foreman-infra [1] repository. I am proposing one of two ideas to bring
> these to the forefront:
>
> 1) Move them to the top of foreman-infra under a "jenkins" or "jobs"
> folder
> 2) Move them to their own repository (e.g. foreman-ci)
>

:+1: for both. I'm leaning towards 2)

– Ivan

··· > > [1] > https://github.com/theforeman/foreman-infra/tree/master/puppet/modules/jenkins_job_builder/files/theforeman.org > > -- > Eric D. Helms > Red Hat Engineering > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >

+1 on bringing them to the forefront. The question where to put them is,
for better or worse, tightly coupled to the way we deploy them.
Currently it's puppet. If you move them to the top it can still be
deployed easily with a symlink or other scripting magic.

Splitting to their own repository makes it harder to re-use the puppet
deployment pipeline and we'd need to build a new one. Where would we put
this? Usually you think of Jenkins when building a pipeline but having
Jenkins manage itself might not be the best idea.

Given that background right now I'm leaning to 1) but I can easily be
convinced of 2) if a more suitable deployment method is found.

··· On Fri, Sep 01, 2017 at 07:27:23PM -0400, Eric D Helms wrote: >Howdy, > >Some quick background, for those that don't know a majority of our Jenkins >job configuration are stored in git. I find our Jenkins job configuration >to be important for developers to be able to understand and contribute >changes. However, today, these configurations are stored deep inside the >foreman-infra [1] repository. I am proposing one of two ideas to bring >these to the forefront: > > 1) Move them to the top of foreman-infra under a "jenkins" or "jobs" folder > 2) Move them to their own repository (e.g. foreman-ci) > > >[1] >https://github.com/theforeman/foreman-infra/tree/master/puppet/modules/jenkins_job_builder/files/theforeman.org

+1 to whatever new location. Since I already know they live in foreman-infra,
I'm fine with keeping them there.

··· On sobota 2. září 2017 1:27:23 CEST Eric D Helms wrote: > Howdy, > > Some quick background, for those that don't know a majority of our Jenkins > job configuration are stored in git. I find our Jenkins job configuration > to be important for developers to be able to understand and contribute > changes. However, today, these configurations are stored deep inside the > foreman-infra [1] repository. I am proposing one of two ideas to bring > these to the forefront: > > 1) Move them to the top of foreman-infra under a "jenkins" or "jobs" folder > 2) Move them to their own repository (e.g. foreman-ci) > > > [1] > https://github.com/theforeman/foreman-infra/tree/master/puppet/modules/jenki > ns_job_builder/files/theforeman.org


Marek

+1 on bringing them to the forefront. The question where to put them is,
for better or worse, tightly coupled to the way we deploy them. Currently
it's puppet. If you move them to the top it can still be deployed easily
with a symlink or other scripting magic.

Splitting to their own repository makes it harder to re-use the puppet
deployment pipeline and we'd need to build a new one. Where would we put
this? Usually you think of Jenkins when building a pipeline but having
Jenkins manage itself might not be the best idea.

Could the puppet just clone the repository if using option #2 and work like
before?

Given that background right now I'm leaning to 1) but I can easily be
convinced of 2) if a more suitable deployment method is found.

··· On Sep 2, 2017 6:27 AM, "Ewoud Kohl van Wijngaarden" < ewoud@kohlvanwijngaarden.nl> wrote:

On Fri, Sep 01, 2017 at 07:27:23PM -0400, Eric D Helms wrote:

Howdy,

Some quick background, for those that don’t know a majority of our Jenkins
job configuration are stored in git. I find our Jenkins job configuration
to be important for developers to be able to understand and contribute
changes. However, today, these configurations are stored deep inside the
foreman-infra [1] repository. I am proposing one of two ideas to bring
these to the forefront:

  1. Move them to the top of foreman-infra under a “jenkins” or “jobs” folder
  2. Move them to their own repository (e.g. foreman-ci)

[1]
https://github.com/theforeman/foreman-infra/tree/master/pupp
et/modules/jenkins_job_builder/files/theforeman.org


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

>
>>
>>> Howdy,
>>>
>>> Some quick background, for those that don't know a majority of our Jenkins
>>> job configuration are stored in git. I find our Jenkins job configuration
>>> to be important for developers to be able to understand and contribute
>>> changes. However, today, these configurations are stored deep inside the
>>> foreman-infra [1] repository. I am proposing one of two ideas to bring
>>> these to the forefront:
>>>
>>> 1) Move them to the top of foreman-infra under a "jenkins" or "jobs" folder
>>> 2) Move them to their own repository (e.g. foreman-ci)
>>>
>>>
>>> [1]
>>> https://github.com/theforeman/foreman-infra/tree/master/puppet/modules/jenkins_job_builder/files/theforeman.org
>>>
>>
>>+1 on bringing them to the forefront. The question where to put them is,
>>for better or worse, tightly coupled to the way we deploy them. Currently
>>it's puppet. If you move them to the top it can still be deployed easily
>>with a symlink or other scripting magic.
>>
>>Splitting to their own repository makes it harder to re-use the puppet
>>deployment pipeline and we'd need to build a new one. Where would we put
>>this? Usually you think of Jenkins when building a pipeline but having
>>Jenkins manage itself might not be the best idea.
>
>Could the puppet just clone the repository if using option #2 and work like
>before?

This biggest challenge I see here is the correct triggers, but I must
admit that I don't know how it's triggered now. AFAIK foreman-infra
currently pretty much contains /etc/puppet and is checked out there.
This should also be moved to a proper control repository (and there's a
TODO item for that).

··· On Sat, Sep 02, 2017 at 08:02:32AM -0400, Eric D Helms wrote: >On Sep 2, 2017 6:27 AM, "Ewoud Kohl van Wijngaarden" < >ewoud@kohlvanwijngaarden.nl> wrote: >>On Fri, Sep 01, 2017 at 07:27:23PM -0400, Eric D Helms wrote:

Given that background right now I’m leaning to 1) but I can easily be
convinced of 2) if a more suitable deployment method is found.

So currently we make use of the fact that it's in the files/ subdir of
the jjb module to simply do a recursive deploy to /etc/jenkins_jobs -
see [1]. So long as (a) we can maintain the deployment to
/etc/jenkins_jobs, and (b) correctly trigger JJB itself, then it should
be fine. There are plenty of VCS puppet modules out there, I'm sure we
can use one to clone from git and notify the exec.

[1] https://github.com/theforeman/foreman-infra/blob/master/puppet/modules/jenkins_job_builder/manifests/config.pp#L15-L25

Greg

··· On Mon, 2017-09-04 at 18:31 +0200, Ewoud Kohl van Wijngaarden wrote: > > This biggest challenge I see here is the correct triggers, but I > must admit that I don't know how it's triggered now. AFAIK foreman- > infra currently pretty much contains /etc/puppet and is checked out > there. This should also be moved to a proper control repository (and > there's a TODO item for that).