Pulling Plugins into Core?

Development
1

There has been some sporadic discussions around the idea of pulling certain
plugins into core at some point. I am sending this email along to spur a
more centralized discussion around this issue. For example, the idea has
been floated around to pull foreman-tasks into core or that the new remote
execution work might be pulled into the core at some point. I think there
are potentially three interpretations of 'pulled into core':

  1. Making a plugin a hard dependency of Foreman
  2. Literally merging the plugin's code into Foreman's as a singular code
    base
  3. Keeping the plugin as an engine, but maintaining it as a single code
    base (e.g. an engines/ directory)

The initial questions that come to my mind around this are:

  1. What do we actually mean by pulling a plugin into core?
  2. When does a plugin get pulled into Foreman core? e.g. What is the
    criteria for such?
  3. What are the expectations (e.g. maturity, infrastructure testing setup)
    for a plugin that gets 'pulled into core' ?
··· -- Eric D. Helms Red Hat Engineering Ph.D. Student - North Carolina State University
2

> There has been some sporadic discussions around the idea of pulling certain
> plugins into core at some point. I am sending this email along to spur a
> more centralized discussion around this issue. For example, the idea has
> been floated around to pull foreman-tasks into core or that the new remote
> execution work might be pulled into the core at some point. I think there
> are potentially three interpretations of 'pulled into core':

> 1) Making a plugin a hard dependency of Foreman
> 2) Literally merging the plugin's code into Foreman's as a singular code
> base
> 3) Keeping the plugin as an engine, but maintaining it as a single code
> base (e.g. an engines/ directory)

With the foreman-tasks, as well as the remote execution, my assumption is
the evolution: from the hard-dependency to merging it directly to the core.
The benefit of not having the plugin in the core is possibility
to have faster release cycles: rolling the versions in the plugin, and keeping
the plugin compatible with multiple versions is somehow easier, than having
that in the core and backporting to multiple versions.

The ultimate goal, however, is to get the code merged into the core, to have
more maintenance power over it.

>
> The initial questions that come to my mind around this are:
>
> 1) What do we actually mean by pulling a plugin into core?

we above

> 2) When does a plugin get pulled into Foreman core? e.g. What is the
> criteria for such?

The criteria for foreman-takss (and remote execution) are one of these:

a) multiple other plugins use it for their work, so it's extending
the platform itself - this is the foreman-tasks case

b) the core is using the functionality of the plugin: this should be the
case for both foreman-tasks (using that for async operations) and
remote_execution (using that for the puppet run and finish scripts).

> 3) What are the expectations (e.g. maturity, infrastructure testing setup)
> for a plugin that gets 'pulled into core' ?

The unit test coverage is one of the biggest requirements I'm aware of.

– Ivan

··· ----- Original Message -----

–
Eric D. Helms
Red Hat Engineering
Ph.D. Student - North Carolina State University

–
You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

> There has been some sporadic discussions around the idea of pulling
> certain plugins into core at some point. I am sending this email along
> to spur a more centralized discussion around this issue. For example,
> the idea has been floated around to pull foreman-tasks into core or that
> the new remote execution work might be pulled into the core at some
> point. I think there are potentially three interpretations of 'pulled
> into core':
>
> 1) Making a plugin a hard dependency of Foreman

I think this would be a bad path to take, as we'd end up with a two-way
dependency between two projects which is hard to maintain, as any
breaking change in one requires updates to the other. That's doubly
hard when we typically only use released versions of plugins as gems.

> 2) Literally merging the plugin's code into Foreman's as a singular code
> base
> 3) Keeping the plugin as an engine, but maintaining it as a single code
> base (e.g. an engines/ directory)

Yeah, either?

> The initial questions that come to my mind around this are:
>
> 1) What do we actually mean by pulling a plugin into core?
> 2) When does a plugin get pulled into Foreman core? e.g. What is the
> criteria for such?

I think it should only happen when there's a necessity to use that
functionality in Foreman core.

If it's only used by other plugins and that seems to be working, then we
should probably keep it as a plugin itself and continue to ensure that
works well through tooling etc. There's still more to do to make
plugins easy to consume, to improve our APIs to developers etc, but
we're always making progress.

If there's no necessity in core itself, then we ought to keep it outside
so as to keep the default Foreman installation small, fast and minimal.

··· On 02/07/15 19:36, Eric D Helms wrote:

–
Dominic Cleal
Red Hat Engineering

4

Is there a 4th which is setting up a plugin so that:

  1. Every commit against core is tested against core and then against
    core + that plugin
  2. Every commit against the plugin tests the plugin and some core versions

That way, there is the guarantee that the plugins work, but they can
evolve indepdently.

– bk

··· On 07/02/2015 02:36 PM, Eric D Helms wrote: > There has been some sporadic discussions around the idea of pulling > certain plugins into core at some point. I am sending this email along > to spur a more centralized discussion around this issue. For example, > the idea has been floated around to pull foreman-tasks into core or that > the new remote execution work might be pulled into the core at some > point. I think there are potentially three interpretations of 'pulled > into core': > > 1) Making a plugin a hard dependency of Foreman > 2) Literally merging the plugin's code into Foreman's as a singular code > base > 3) Keeping the plugin as an engine, but maintaining it as a single code > base (e.g. an engines/ directory)
5

> > There has been some sporadic discussions around the idea of pulling
> > certain plugins into core at some point. I am sending this email along
> > to spur a more centralized discussion around this issue. For example,
> > the idea has been floated around to pull foreman-tasks into core or that
> > the new remote execution work might be pulled into the core at some
> > point. I think there are potentially three interpretations of 'pulled
> > into core':
> >
> > 1) Making a plugin a hard dependency of Foreman
>
> I think this would be a bad path to take, as we'd end up with a two-way
> dependency between two projects which is hard to maintain, as any
> breaking change in one requires updates to the other. That's doubly
> hard when we typically only use released versions of plugins as gems.
>
> > 2) Literally merging the plugin's code into Foreman's as a singular code
> > base
> > 3) Keeping the plugin as an engine, but maintaining it as a single code
> > base (e.g. an engines/ directory)
>
> Yeah, either?
>
> > The initial questions that come to my mind around this are:
> >
> > 1) What do we actually mean by pulling a plugin into core?
> > 2) When does a plugin get pulled into Foreman core? e.g. What is the
> > criteria for such?
>
> I think it should only happen when there's a necessity to use that
> functionality in Foreman core.
>
> If it's only used by other plugins and that seems to be working, then we
> should probably keep it as a plugin itself and continue to ensure that
> works well through tooling etc. There's still more to do to make
> plugins easy to consume, to improve our APIs to developers etc, but
> we're always making progress.
>
> If there's no necessity in core itself, then we ought to keep it outside
> so as to keep the default Foreman installation small, fast and minimal.

What would you define as a necessity? If more than the majority of Foreman
installations out in the wild include plugin X or want plugin X, I would
consider that as reasonable grounds for inclusion in core (I prefer #2 but
would accept #3).

Thoughts?

··· ----- Original Message ----- > On 02/07/15 19:36, Eric D Helms wrote:

–
Dominic Cleal
Red Hat Engineering

–
You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–

  • adam price
6

We already test plugins constantly against Foreman's develop and stable
branches.

That's fine for almost all plugins, but doesn't allow usage of the
plugin from core, which is desired sometimes. Those need to be merged
in when the time comes.

··· On 06/07/15 15:17, Bryan Kearney wrote: > > > On 07/02/2015 02:36 PM, Eric D Helms wrote: >> There has been some sporadic discussions around the idea of pulling >> certain plugins into core at some point. I am sending this email along >> to spur a more centralized discussion around this issue. For example, >> the idea has been floated around to pull foreman-tasks into core or that >> the new remote execution work might be pulled into the core at some >> point. I think there are potentially three interpretations of 'pulled >> into core': >> >> 1) Making a plugin a hard dependency of Foreman >> 2) Literally merging the plugin's code into Foreman's as a singular code >> base >> 3) Keeping the plugin as an engine, but maintaining it as a single code >> base (e.g. an engines/ directory) > > Is there a 4th which is setting up a plugin so that: > > 1) Every commit against core is tested against core and then against > core + that plugin > 2) Every commit against the plugin tests the plugin and some core versions > > That way, there is the guarantee that the plugins work, but they can > evolve indepdently.

–
Dominic Cleal
Red Hat Engineering

7

>>
>>
>>> There has been some sporadic discussions around the idea of pulling
>>> certain plugins into core at some point. I am sending this email along
>>> to spur a more centralized discussion around this issue. For example,
>>> the idea has been floated around to pull foreman-tasks into core or that
>>> the new remote execution work might be pulled into the core at some
>>> point. I think there are potentially three interpretations of 'pulled
>>> into core':
>>>
>>> 1) Making a plugin a hard dependency of Foreman
>>> 2) Literally merging the plugin's code into Foreman's as a singular code
>>> base
>>> 3) Keeping the plugin as an engine, but maintaining it as a single code
>>> base (e.g. an engines/ directory)
>>
>> Is there a 4th which is setting up a plugin so that:
>>
>> 1) Every commit against core is tested against core and then against
>> core + that plugin
>> 2) Every commit against the plugin tests the plugin and some core versions
>>
>> That way, there is the guarantee that the plugins work, but they can
>> evolve indepdently.
>
> We already test plugins constantly against Foreman's develop and stable
> branches.

are the core tests re-run with the plugin installed to make sure there
is no breakage?

>
> That's fine for almost all plugins, but doesn't allow usage of the
> plugin from core, which is desired sometimes. Those need to be merged
> in when the time comes.
>

Yeah… understood… i would assume plumbing items (e.g. tasks) would
eventually come in.

– bk

··· On 07/06/2015 10:24 AM, Dominic Cleal wrote: > On 06/07/15 15:17, Bryan Kearney wrote: >> On 07/02/2015 02:36 PM, Eric D Helms wrote:
8

>>> There has been some sporadic discussions around the idea of pulling
>>> certain plugins into core at some point. I am sending this email along
>>> to spur a more centralized discussion around this issue. For example,
>>> the idea has been floated around to pull foreman-tasks into core or that
>>> the new remote execution work might be pulled into the core at some
>>> point. I think there are potentially three interpretations of 'pulled
>>> into core':
>>>
>>> 1) Making a plugin a hard dependency of Foreman
>>
>> I think this would be a bad path to take, as we'd end up with a two-way
>> dependency between two projects which is hard to maintain, as any
>> breaking change in one requires updates to the other. That's doubly
>> hard when we typically only use released versions of plugins as gems.
>>
>>> 2) Literally merging the plugin's code into Foreman's as a singular code
>>> base
>>> 3) Keeping the plugin as an engine, but maintaining it as a single code
>>> base (e.g. an engines/ directory)
>>
>> Yeah, either?
>>
>>> The initial questions that come to my mind around this are:
>>>
>>> 1) What do we actually mean by pulling a plugin into core?
>>> 2) When does a plugin get pulled into Foreman core? e.g. What is the
>>> criteria for such?
>>
>> I think it should only happen when there's a necessity to use that
>> functionality in Foreman core.
>>
>> If it's only used by other plugins and that seems to be working, then we
>> should probably keep it as a plugin itself and continue to ensure that
>> works well through tooling etc. There's still more to do to make
>> plugins easy to consume, to improve our APIs to developers etc, but
>> we're always making progress.
>>
>> If there's no necessity in core itself, then we ought to keep it outside
>> so as to keep the default Foreman installation small, fast and minimal.
>
> What would you define as a necessity? If more than the majority of Foreman
> installations out in the wild include plugin X or want plugin X, I would
> consider that as reasonable grounds for inclusion in core (I prefer #2 but
> would accept #3).

I have no idea how you'd evaluate that.

Anyway, that's not what I mean by necessity - I mean, if it's required
to use the plugin's functionality from core then merge it in, not just
because a number of users have it.

> Thoughts?

Taking Ohad's example, if there was a use for the foreman-tasks plugin
in core code then adopt, review and merge it in.

··· On 06/07/15 14:41, Adam Price wrote: > ----- Original Message ----- >> On 02/07/15 19:36, Eric D Helms wrote:

–
Dominic Cleal
Red Hat Engineering

9

Yes, since you're just running the regular test rake task and the plugin
appends any extra tests of its own
(How to Create a Plugin - Foreman).
Katello's the only plugin that does its own thing at the moment (as per
the previous thread).

··· On 06/07/15 15:55, Bryan Kearney wrote: > > > On 07/06/2015 10:24 AM, Dominic Cleal wrote: >> On 06/07/15 15:17, Bryan Kearney wrote: >>> >>> >>> On 07/02/2015 02:36 PM, Eric D Helms wrote: >>>> There has been some sporadic discussions around the idea of pulling >>>> certain plugins into core at some point. I am sending this email along >>>> to spur a more centralized discussion around this issue. For example, >>>> the idea has been floated around to pull foreman-tasks into core or that >>>> the new remote execution work might be pulled into the core at some >>>> point. I think there are potentially three interpretations of 'pulled >>>> into core': >>>> >>>> 1) Making a plugin a hard dependency of Foreman >>>> 2) Literally merging the plugin's code into Foreman's as a singular code >>>> base >>>> 3) Keeping the plugin as an engine, but maintaining it as a single code >>>> base (e.g. an engines/ directory) >>> >>> Is there a 4th which is setting up a plugin so that: >>> >>> 1) Every commit against core is tested against core and then against >>> core + that plugin >>> 2) Every commit against the plugin tests the plugin and some core versions >>> >>> That way, there is the guarantee that the plugins work, but they can >>> evolve indepdently. >> >> We already test plugins constantly against Foreman's develop and stable >> branches. > > are the core tests re-run with the plugin installed to make sure there > is no breakage?

–
Dominic Cleal
Red Hat Engineering

10

> > > There has been some sporadic discussions around the idea of pulling
> > > certain plugins into core at some point. I am sending this email along
> > > to spur a more centralized discussion around this issue. For example,
> > > the idea has been floated around to pull foreman-tasks into core or
> that
> > > the new remote execution work might be pulled into the core at some
> > > point. I think there are potentially three interpretations of 'pulled
> > > into core':
> > >
> > > 1) Making a plugin a hard dependency of Foreman
> >
> > I think this would be a bad path to take, as we'd end up with a two-way
> > dependency between two projects which is hard to maintain, as any
> > breaking change in one requires updates to the other. That's doubly
> > hard when we typically only use released versions of plugins as gems.
> >
> > > 2) Literally merging the plugin's code into Foreman's as a singular
> code
> > > base
> > > 3) Keeping the plugin as an engine, but maintaining it as a single code
> > > base (e.g. an engines/ directory)
> >
> > Yeah, either?
> >
> > > The initial questions that come to my mind around this are:
> > >
> > > 1) What do we actually mean by pulling a plugin into core?
> > > 2) When does a plugin get pulled into Foreman core? e.g. What is the
> > > criteria for such?
> >
> > I think it should only happen when there's a necessity to use that
> > functionality in Foreman core.
> >
> > If it's only used by other plugins and that seems to be working, then we
> > should probably keep it as a plugin itself and continue to ensure that
> > works well through tooling etc. There's still more to do to make
> > plugins easy to consume, to improve our APIs to developers etc, but
> > we're always making progress.
> >
> > If there's no necessity in core itself, then we ought to keep it outside
> > so as to keep the default Foreman installation small, fast and minimal.
>
> What would you define as a necessity? If more than the majority of Foreman
> installations out in the wild include plugin X or want plugin X, I would
> consider that as reasonable grounds for inclusion in core (I prefer #2 but
> would accept #3).
>
> Thoughts?
>
> IMHO the main problem for core is chicken and egg, I want to use
foreman-tasks, but I don't because its not in core.

Ohad

··· On Mon, Jul 6, 2015 at 4:41 PM, Adam Price wrote: > ----- Original Message ----- > > On 02/07/15 19:36, Eric D Helms wrote:

–
Dominic Cleal
Red Hat Engineering

–
You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–

  • adam price

–
You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.