Smart-proxy plugins POC

Development
1

Repositories (you'll need both) are located at [1] and [2].

This POC follows Sinatra's approach to modular applications: each area
of functionality is now encapsulated by a dedicated Sinatra
application. The main application is assembled and started using rack
(pls. see [3] and [4] correspondingly) via rackup fragments defined in
each of the plugins, e.g. [5].

The main application can be quite minimal, its main responsibility
being assembling and launching of the whole application based on the
config settings. We could, however, include the most commonly used
modules in the main application code-base (as opposed to extracting
each area of functionality into a dedicated gem).

The benefits of such a break of the current codebase include ability
to test modules in complete isolation from each other, no
customization of Sinatra, and ability to use rack (which may come
handy if we want to use other webservers).

I have not tested this approach on Windows.

Please let me know if you have comments, questions, suggestions, etc,
-d

[1] https://github.com/witlessbird/sinatra-with-plugins.git
[2] https://github.com/witlessbird/sinatra-plugin-one.git
[3] https://github.com/witlessbird/sinatra-with-plugins/blob/master/bin/launch_me.rb#L17
[4] https://github.com/witlessbird/sinatra-with-plugins/blob/master/bin/launch_me.rb#L22
[5] https://github.com/witlessbird/sinatra-plugin-one/blob/master/lib/sinatra_plugin_one/config.ru

2

That looks quite straightforward.

I think we need to be able to draw a distinction between the APIs and
"providers" too, so we could ship the API in core (easy enough), but
then have a gem add a provider to say, the DNS API. We might need
something like the ModuleConfig or Foreman's plugin registration for
that backend piece.

Many providers add additional dependencies (e.g. GSSAPI, rubyipmi) which
would be really valuable to separate into independent plugins. I'd like
to be able to install the DNS API (which might be in core) with the
virsh provider without needing the nsupdate_gss provider and its
Kerberos dependencies.

We should also consider making versions a first class bit of each API,
as versioning of a core plus plugins is going to get more complex after
this change. Would it make sense for each API to expose its own version
route, which might contain various data, or for a single proxy-wide
route (like /features)? I'm coming back to the idea of an explicit
plugin registration again I think.

There will be some packaging work to complete this, which should just be
a case of releasing gems and packaging like we do with other plugins.

I can imagine us making some APIs and providers plugins in order to
remove dependencies from the smart proxy core (i.e. Puppet), but still
shipping them in our main repos. Either that, or splitting them up
inside the smart-proxy repo (like a Rails engine stored within the main
app repo) but then still mounting them as plugins, and shipping
foreman-proxy subpackages.

The installer's foreman_proxy module will also need updating to install
any additional packages depending on the options selected.

··· On 20/03/14 12:22, Dmitri Dolguikh wrote: > Repositories (you'll need both) are located at [1] and [2]. > > This POC follows Sinatra's approach to modular applications: each area > of functionality is now encapsulated by a dedicated Sinatra > application. The main application is assembled and started using rack > (pls. see [3] and [4] correspondingly) via rackup fragments defined in > each of the plugins, e.g. [5]. > > The main application can be quite minimal, its main responsibility > being assembling and launching of the whole application based on the > config settings. We could, however, include the most commonly used > modules in the main application code-base (as opposed to extracting > each area of functionality into a dedicated gem). > > The benefits of such a break of the current codebase include ability > to test modules in complete isolation from each other, no > customization of Sinatra, and ability to use rack (which may come > handy if we want to use other webservers). > > I have not tested this approach on Windows. > > Please let me know if you have comments, questions, suggestions, etc,

–
Dominic Cleal
Red Hat Engineering

3

This looks good at first glance, I'll try to play with it today. My
first thought (that hasn't already been covered by Dominic) is that
we'll need some changes on the Foreman side too - today, the list of
features a proxy can have is hard-coded in the db. I guess that won't
play nice with plugins adding their own featues…

Greg

4

Pls. see my comments below…

>> Repositories (you'll need both) are located at [1] and [2].
>>
>> This POC follows Sinatra's approach to modular applications: each area
>> of functionality is now encapsulated by a dedicated Sinatra
>> application. The main application is assembled and started using rack
>> (pls. see [3] and [4] correspondingly) via rackup fragments defined in
>> each of the plugins, e.g. [5].
>>
>> The main application can be quite minimal, its main responsibility
>> being assembling and launching of the whole application based on the
>> config settings. We could, however, include the most commonly used
>> modules in the main application code-base (as opposed to extracting
>> each area of functionality into a dedicated gem).
>>
>> The benefits of such a break of the current codebase include ability
>> to test modules in complete isolation from each other, no
>> customization of Sinatra, and ability to use rack (which may come
>> handy if we want to use other webservers).
>>
>> I have not tested this approach on Windows.
>>
>> Please let me know if you have comments, questions, suggestions, etc,
>
> That looks quite straightforward.
>
> I think we need to be able to draw a distinction between the APIs and
> "providers" too, so we could ship the API in core (easy enough), but
> then have a gem add a provider to say, the DNS API. We might need
> something like the ModuleConfig or Foreman's plugin registration for
> that backend piece.
>
> Many providers add additional dependencies (e.g. GSSAPI, rubyipmi) which
> would be really valuable to separate into independent plugins. I'd like
> to be able to install the DNS API (which might be in core) with the
> virsh provider without needing the nsupdate_gss provider and its
> Kerberos dependencies.

Where, besides DNS, do you think we need plugins such as above? Would
be useful to see a few various cases in order to derive a useful
abstraction for this. I think the responsibility for loading
providers, etc should reside with plugins themselves, for a variety of
reasons and we could keep the api for smart-proxy modules and
providers separate.

>
> We should also consider making versions a first class bit of each API,
> as versioning of a core plus plugins is going to get more complex after
> this change. Would it make sense for each API to expose its own version
> route, which might contain various data, or for a single proxy-wide
> route (like /features)? I'm coming back to the idea of an explicit
> plugin registration again I think.

Not sure about /features route. I think we can rely on REST discovery
to enumerate available features (and get their metadata, such as
versioning). This would require changes on both foreman and
smart-proxy sides, however. It would look something like:

  • Foreman, on startup (periodically? we can cache response data
    together with ETAG) queries the smart-proxy API entry point: GET
    http://smart-proxy/
  • Smart-proxy replies with something like:
    {
    "dns": { "href": "http://…/dns", "rel": "http://…/dns_api" },
    "dhcp": {},
    …
    }
  • Foreman then parser the reply and (re)configures itself appropriately.

Such a change would probably be better carried as a standalone step,
as it can be quite disruptive.

>
> There will be some packaging work to complete this, which should just be
> a case of releasing gems and packaging like we do with other plugins.

Yep.

>
> I can imagine us making some APIs and providers plugins in order to
> remove dependencies from the smart proxy core (i.e. Puppet), but still
> shipping them in our main repos. Either that, or splitting them up
> inside the smart-proxy repo (like a Rails engine stored within the main
> app repo) but then still mounting them as plugins, and shipping
> foreman-proxy subpackages.

It would probably make sense to include the most commonly used
features together with smart-proxy. They still can be implemented as
plugins, but will reside together with the proxy code (trying to
simplify packaging/install here).

>
> The installer's foreman_proxy module will also need updating to install
> any additional packages depending on the options selected.

yep.

··· On Thu, Mar 20, 2014 at 1:01 PM, Dominic Cleal wrote: > On 20/03/14 12:22, Dmitri Dolguikh wrote:

–
Dominic Cleal
Red Hat Engineering

–
You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

It certainly doesn't cope today if the feature advertised is unknown to
Foreman (so that needs fixing), but I'm expecting the "Feature" records
themselves to be added by seed scripts provided by Foreman plugins.

··· On 20/03/14 13:22, Greg Sutcliffe wrote: > This looks good at first glance, I'll try to play with it today. My > first thought (that hasn't already been covered by Dominic) is that > we'll need some changes on the Foreman side too - today, the list of > features a proxy can have is hard-coded in the db. I guess that won't > play nice with plugins adding their own featues...

–
Dominic Cleal
Red Hat Engineering

6

> Pls. see my comments below…
>
>>> Repositories (you'll need both) are located at [1] and [2].
>>>
>>> This POC follows Sinatra's approach to modular applications: each area
>>> of functionality is now encapsulated by a dedicated Sinatra
>>> application. The main application is assembled and started using rack
>>> (pls. see [3] and [4] correspondingly) via rackup fragments defined in
>>> each of the plugins, e.g. [5].
>>>
>>> The main application can be quite minimal, its main responsibility
>>> being assembling and launching of the whole application based on the
>>> config settings. We could, however, include the most commonly used
>>> modules in the main application code-base (as opposed to extracting
>>> each area of functionality into a dedicated gem).
>>>
>>> The benefits of such a break of the current codebase include ability
>>> to test modules in complete isolation from each other, no
>>> customization of Sinatra, and ability to use rack (which may come
>>> handy if we want to use other webservers).
>>>
>>> I have not tested this approach on Windows.
>>>
>>> Please let me know if you have comments, questions, suggestions, etc,
>>
>> That looks quite straightforward.
>>
>> I think we need to be able to draw a distinction between the APIs and
>> "providers" too, so we could ship the API in core (easy enough), but
>> then have a gem add a provider to say, the DNS API. We might need
>> something like the ModuleConfig or Foreman's plugin registration for
>> that backend piece.
>>
>> Many providers add additional dependencies (e.g. GSSAPI, rubyipmi) which
>> would be really valuable to separate into independent plugins. I'd like
>> to be able to install the DNS API (which might be in core) with the
>> virsh provider without needing the nsupdate_gss provider and its
>> Kerberos dependencies.
>
> Where, besides DNS, do you think we need plugins such as above? Would
> be useful to see a few various cases in order to derive a useful
> abstraction for this. I think the responsibility for loading
> providers, etc should reside with plugins themselves, for a variety of
> reasons and we could keep the api for smart-proxy modules and
> providers separate.

The Puppet run API is currently part of the "puppet" feature (which also
does class/env imports) and has a number of providers that depend on
external tools like MCollective, Salt and SSH. These make sense as
separate plugins to me, with packages that depend on the appropriate
package/binary.

Potentially the BMC API, which is mostly the freeipmi/ipmitool
implementation and a second shell one. I think this could be made a
sub-package, but I can foresee non-IPMI based implementations.

Potentially the DHCP API, which might get providers to other DHCP
implementations like Bluecat devices or something.

>> We should also consider making versions a first class bit of each API,
>> as versioning of a core plus plugins is going to get more complex after
>> this change. Would it make sense for each API to expose its own version
>> route, which might contain various data, or for a single proxy-wide
>> route (like /features)? I'm coming back to the idea of an explicit
>> plugin registration again I think.
>
> Not sure about /features route. I think we can rely on REST discovery
> to enumerate available features (and get their metadata, such as
> versioning). This would require changes on both foreman and
> smart-proxy sides, however. It would look something like:
>
> - Foreman, on startup (periodically? we can cache response data
> together with ETAG) queries the smart-proxy API entry point: GET
> http://smart-proxy/
> - Smart-proxy replies with something like:
> {
> "dns": { "href": "http://…/dns", "rel": "http://…/dns_api" },
> "dhcp": {},
> …
> }
> - Foreman then parser the reply and (re)configures itself appropriately.
>
> Such a change would probably be better carried as a standalone step,
> as it can be quite disruptive.

Sounds good to me.

>> There will be some packaging work to complete this, which should just be
>> a case of releasing gems and packaging like we do with other plugins.
>
> Yep.
>
>>
>> I can imagine us making some APIs and providers plugins in order to
>> remove dependencies from the smart proxy core (i.e. Puppet), but still
>> shipping them in our main repos. Either that, or splitting them up
>> inside the smart-proxy repo (like a Rails engine stored within the main
>> app repo) but then still mounting them as plugins, and shipping
>> foreman-proxy subpackages.
>
> It would probably make sense to include the most commonly used
> features together with smart-proxy. They still can be implemented as
> plugins, but will reside together with the proxy code (trying to
> simplify packaging/install here).

Indeed, I think retaining the primary APIs for things like DHCP, DNS,
Puppet in core makes sense, since they would form a bare minimum
implementation that Foreman requires from the proxy anyway.

··· On 20/03/14 15:09, Dmitri Dolguikh wrote: > On Thu, Mar 20, 2014 at 1:01 PM, Dominic Cleal wrote: >> On 20/03/14 12:22, Dmitri Dolguikh wrote:

–
Dominic Cleal
Red Hat Engineering

7

> Pls. see my comments below…
>
> >> Repositories (you'll need both) are located at [1] and [2].
> >>
> >> This POC follows Sinatra's approach to modular applications: each area
> >> of functionality is now encapsulated by a dedicated Sinatra
> >> application. The main application is assembled and started using rack
> >> (pls. see [3] and [4] correspondingly) via rackup fragments defined in
> >> each of the plugins, e.g. [5].
> >>
> >> The main application can be quite minimal, its main responsibility
> >> being assembling and launching of the whole application based on the
> >> config settings. We could, however, include the most commonly used
> >> modules in the main application code-base (as opposed to extracting
> >> each area of functionality into a dedicated gem).
> >>
> >> The benefits of such a break of the current codebase include ability
> >> to test modules in complete isolation from each other, no
> >> customization of Sinatra, and ability to use rack (which may come
> >> handy if we want to use other webservers).
> >>
> >> I have not tested this approach on Windows.
> >>
> >> Please let me know if you have comments, questions, suggestions, etc,
> >
> > That looks quite straightforward.
> >
> > I think we need to be able to draw a distinction between the APIs and
> > "providers" too, so we could ship the API in core (easy enough), but
> > then have a gem add a provider to say, the DNS API. We might need
> > something like the ModuleConfig or Foreman's plugin registration for
> > that backend piece.
> >
> > Many providers add additional dependencies (e.g. GSSAPI, rubyipmi) which
> > would be really valuable to separate into independent plugins. I'd like
> > to be able to install the DNS API (which might be in core) with the
> > virsh provider without needing the nsupdate_gss provider and its
> > Kerberos dependencies.
>
> Where, besides DNS, do you think we need plugins such as above? Would
> be useful to see a few various cases in order to derive a useful
> abstraction for this. I think the responsibility for loading
> providers, etc should reside with plugins themselves, for a variety of
> reasons and we could keep the api for smart-proxy modules and
> providers separate.

DHCP could also have multiple providers (ISC DHCP vs MS-DHCP). Would it
make sense to allow "interface"-plugins which just describe the API and
have a fairly standard mechanism to load a provider? I wouldn't mind per
se that this configuration is static and done by the admin. An interface
can include a default load order to auto support common implementations.

> > We should also consider making versions a first class bit of each API,
> > as versioning of a core plus plugins is going to get more complex after
> > this change. Would it make sense for each API to expose its own version
> > route, which might contain various data, or for a single proxy-wide
> > route (like /features)? I'm coming back to the idea of an explicit
> > plugin registration again I think.
>
> Not sure about /features route. I think we can rely on REST discovery
> to enumerate available features (and get their metadata, such as
> versioning). This would require changes on both foreman and
> smart-proxy sides, however. It would look something like:
>
> - Foreman, on startup (periodically? we can cache response data
> together with ETAG) queries the smart-proxy API entry point: GET
> http://smart-proxy/
> - Smart-proxy replies with something like:
> {
> "dns": { "href": "http://…/dns", "rel": "http://…/dns_api" },
> "dhcp": {},
> …
> }

I'd vote for a version parameter per feature as well.

> - Foreman then parser the reply and (re)configures itself appropriately.
>
> Such a change would probably be better carried as a standalone step,
> as it can be quite disruptive.

We already have the problem that smartproxy features need to be
refreshed. Foreman just needs to learn the new format.

> > There will be some packaging work to complete this, which should just be
> > a case of releasing gems and packaging like we do with other plugins.
>
> Yep.
>
> >
> > I can imagine us making some APIs and providers plugins in order to
> > remove dependencies from the smart proxy core (i.e. Puppet), but still
> > shipping them in our main repos. Either that, or splitting them up
> > inside the smart-proxy repo (like a Rails engine stored within the main
> > app repo) but then still mounting them as plugins, and shipping
> > foreman-proxy subpackages.
>
> It would probably make sense to include the most commonly used
> features together with smart-proxy. They still can be implemented as
> plugins, but will reside together with the proxy code (trying to
> simplify packaging/install here).

Big +1 to shipping common plugins as separate packages, but from the
same code base. Especially if that means we can set proper dependencies.

> > The installer's foreman_proxy module will also need updating to install
> > any additional packages depending on the options selected.
>
> yep.

I'd also say we need something like /etc/foreman-proxy/settings.d/ like
foreman has.

··· On Thu, Mar 20, 2014 at 03:09:04PM +0000, Dmitri Dolguikh wrote: > On Thu, Mar 20, 2014 at 1:01 PM, Dominic Cleal wrote: > > On 20/03/14 12:22, Dmitri Dolguikh wrote:
8

Whilst I can agree most plugins in the proxy will be consumed by
Foreman, and thus it's fine… I suspect people will write plugins to
allow other 3rd party stuff to interact with the proxy too. We should
at least silently ignore features we don't recognise, instead of
bombing out.

··· On 20 March 2014 13:28, Dominic Cleal wrote: > It certainly doesn't cope today if the feature advertised is unknown to > Foreman (so that needs fixing), but I'm expecting the "Feature" records > themselves to be added by seed scripts provided by Foreman plugins.
9

This works, at least on develop. My proxy advertises “realm” and no problems adding the proxy and being able to use the other features Foreman knows about.

··· On 20 Mar 2014, at 14:28, Dominic Cleal wrote:

On 20/03/14 13:22, Greg Sutcliffe wrote:

This looks good at first glance, I’ll try to play with it today. My
first thought (that hasn’t already been covered by Dominic) is that
we’ll need some changes on the Foreman side too - today, the list of
features a proxy can have is hard-coded in the db. I guess that won’t
play nice with plugins adding their own featues…

It certainly doesn’t cope today if the feature advertised is unknown to
Foreman (so that needs fixing),

–
Stephen Benjamin
stephen@bitbin.de

10

>> Pls. see my comments below…
>>
>> >> Repositories (you'll need both) are located at [1] and [2].
>> >>
>> >> This POC follows Sinatra's approach to modular applications: each area
>> >> of functionality is now encapsulated by a dedicated Sinatra
>> >> application. The main application is assembled and started using rack
>> >> (pls. see [3] and [4] correspondingly) via rackup fragments defined in
>> >> each of the plugins, e.g. [5].
>> >>
>> >> The main application can be quite minimal, its main responsibility
>> >> being assembling and launching of the whole application based on the
>> >> config settings. We could, however, include the most commonly used
>> >> modules in the main application code-base (as opposed to extracting
>> >> each area of functionality into a dedicated gem).
>> >>
>> >> The benefits of such a break of the current codebase include ability
>> >> to test modules in complete isolation from each other, no
>> >> customization of Sinatra, and ability to use rack (which may come
>> >> handy if we want to use other webservers).
>> >>
>> >> I have not tested this approach on Windows.
>> >>
>> >> Please let me know if you have comments, questions, suggestions, etc,
>> >
>> > That looks quite straightforward.
>> >
>> > I think we need to be able to draw a distinction between the APIs and
>> > "providers" too, so we could ship the API in core (easy enough), but
>> > then have a gem add a provider to say, the DNS API. We might need
>> > something like the ModuleConfig or Foreman's plugin registration for
>> > that backend piece.
>> >
>> > Many providers add additional dependencies (e.g. GSSAPI, rubyipmi) which
>> > would be really valuable to separate into independent plugins. I'd like
>> > to be able to install the DNS API (which might be in core) with the
>> > virsh provider without needing the nsupdate_gss provider and its
>> > Kerberos dependencies.
>>
>> Where, besides DNS, do you think we need plugins such as above? Would
>> be useful to see a few various cases in order to derive a useful
>> abstraction for this. I think the responsibility for loading
>> providers, etc should reside with plugins themselves, for a variety of
>> reasons and we could keep the api for smart-proxy modules and
>> providers separate.
>
> DHCP could also have multiple providers (ISC DHCP vs MS-DHCP). Would it
> make sense to allow "interface"-plugins which just describe the API and
> have a fairly standard mechanism to load a provider? I wouldn't mind per
> se that this configuration is static and done by the admin. An interface
> can include a default load order to auto support common implementations.
>

Not sure we need plugin API for this – a given module can attempt to
dynamically load a specific provider on startup. Hmm. I suppose some
basic api to verify compatibility and provide dynamic loading might be
useful after all…

··· On Thu, Mar 20, 2014 at 3:24 PM, Ewoud Kohl van Wijngaarden wrote: > On Thu, Mar 20, 2014 at 03:09:04PM +0000, Dmitri Dolguikh wrote: >> On Thu, Mar 20, 2014 at 1:01 PM, Dominic Cleal wrote: >> > On 20/03/14 12:22, Dmitri Dolguikh wrote:

We should also consider making versions a first class bit of each API,
as versioning of a core plus plugins is going to get more complex after
this change. Would it make sense for each API to expose its own version
route, which might contain various data, or for a single proxy-wide
route (like /features)? I’m coming back to the idea of an explicit
plugin registration again I think.

Not sure about /features route. I think we can rely on REST discovery
to enumerate available features (and get their metadata, such as
versioning). This would require changes on both foreman and
smart-proxy sides, however. It would look something like:

  • Foreman, on startup (periodically? we can cache response data
    together with ETAG) queries the smart-proxy API entry point: GET
    http://smart-proxy/
  • Smart-proxy replies with something like:
    {
    “dns”: { “href”: “http://…/dns”, “rel”: “http://…/dns_api” },
    “dhcp”: {},
    …
    }

I’d vote for a version parameter per feature as well.

  • Foreman then parser the reply and (re)configures itself appropriately.

Such a change would probably be better carried as a standalone step,
as it can be quite disruptive.

We already have the problem that smartproxy features need to be
refreshed. Foreman just needs to learn the new format.

There will be some packaging work to complete this, which should just be
a case of releasing gems and packaging like we do with other plugins.

Yep.

I can imagine us making some APIs and providers plugins in order to
remove dependencies from the smart proxy core (i.e. Puppet), but still
shipping them in our main repos. Either that, or splitting them up
inside the smart-proxy repo (like a Rails engine stored within the main
app repo) but then still mounting them as plugins, and shipping
foreman-proxy subpackages.

It would probably make sense to include the most commonly used
features together with smart-proxy. They still can be implemented as
plugins, but will reside together with the proxy code (trying to
simplify packaging/install here).

Big +1 to shipping common plugins as separate packages, but from the
same code base. Especially if that means we can set proper dependencies.

The installer’s foreman_proxy module will also need updating to install
any additional packages depending on the options selected.

yep.

I’d also say we need something like /etc/foreman-proxy/settings.d/ like
foreman has.

–
You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

11

To give some example from outside the current codebase - I can see
space for Discovery implementing custom proxy routes for interacting
with Discovered hosts. Right now we (ab)use the BMC provider to send
the reboot command to the host when it's provisioned, but I'd love
to be able to eliminate the direct Foreman->Host communcation that's
currently required.

My templates-via-proxy PR(s) are another example where we might
benefit from a plugin - Dmitri, can this this proposal listen on
multiple ports, or are will still single-threaded here?

Greg

··· On 20 March 2014 15:21, Dominic Cleal wrote: >> Where, besides DNS, do you think we need plugins such as above? Would >> be useful to see a few various cases in order to derive a useful >> abstraction for this. I think the responsibility for loading >> providers, etc should reside with plugins themselves, for a variety of >> reasons and we could keep the api for smart-proxy modules and >> providers separate.
12

Ah thanks, I must be muddled. Thought it was still broken.

Dominic Cleal
Red Hat Engineering

··· On 20/03/14 13:31, Stephen Benjamin wrote: > On 20 Mar 2014, at 14:28, Dominic Cleal wrote: > >> On 20/03/14 13:22, Greg Sutcliffe wrote: >>> This looks good at first glance, I'll try to play with it >>> today. My first thought (that hasn't already been covered by >>> Dominic) is that we'll need some changes on the Foreman side >>> too - today, the list of features a proxy can have is >>> hard-coded in the db. I guess that won't play nice with plugins >>> adding their own featues... >> >> It certainly doesn't cope today if the feature advertised is >> unknown to Foreman (so that needs fixing), > > This works, at least on develop. My proxy advertises “realm” and > no problems adding the proxy and being able to use the other > features Foreman knows about.
13

>>> Where, besides DNS, do you think we need plugins such as above? Would
>>> be useful to see a few various cases in order to derive a useful
>>> abstraction for this. I think the responsibility for loading
>>> providers, etc should reside with plugins themselves, for a variety of
>>> reasons and we could keep the api for smart-proxy modules and
>>> providers separate.
>
> To give some example from outside the current codebase - I can see
> space for Discovery implementing custom proxy routes for interacting
> with Discovered hosts. Right now we (ab)use the BMC provider to send
> the reboot command to the host when it's provisioned, but I'd love
> to be able to eliminate the direct Foreman->Host communcation that's
> currently required.
>
> My templates-via-proxy PR(s) are another example where we might
> benefit from a plugin - Dmitri, can this this proposal listen on
> multiple ports, or are will still single-threaded here?

It's going to listen on a single port. It should be possible to
build and launch several rack apps, each listening on a different
port, however.
-d

··· On Thu, Mar 20, 2014 at 4:44 PM, Greg Sutcliffe wrote: > On 20 March 2014 15:21, Dominic Cleal wrote:

Greg

–
You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

14

> > It certainly doesn't cope today if the feature advertised is unknown to
> > Foreman (so that needs fixing), but I'm expecting the "Feature" records
> > themselves to be added by seed scripts provided by Foreman plugins.
>
> Whilst I can agree most plugins in the proxy will be consumed by
> Foreman, and thus it's fine… I suspect people will write plugins to
> allow other 3rd party stuff to interact with the proxy too. We should
> at least silently ignore features we don't recognise, instead of
> bombing out.
>

I fixed that a while ago at 1ad4782a

Ohad

··· On Thu, Mar 20, 2014 at 3:29 PM, Greg Sutcliffe wrote: > On 20 March 2014 13:28, Dominic Cleal wrote:

–
You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

15

Based on the feedback in this thread, below is a course of action/user
stories I propose:

  • As a developer I'd like to break up smart-proxy into multiple components
  • based on the POC. With modules residing in the smart-proxy codebase
    for the time being
  • As a developer I'd like to have api to selectively load
    sub-components of modules
  • Introduce api to help with versioning, conflict detection, and
    loading of sub-components
  • As a developer I'd like to be able to dynamically (and periodically)
    discover available smart-proxy features
  • modify smart-proxy: introduce API entry-point; return a list of
    features when accessing the entry point
  • update Foreman to periodically poll smart-proxy for available features
  • As a user I'd like to have ability to selectively install
    smart-proxy components
  • move component code into dedicated repositories and package
    (sub-)components into individual gems
  • update the installer

-d

··· On Thu, Mar 20, 2014 at 4:53 PM, Dmitri Dolguikh wrote: > On Thu, Mar 20, 2014 at 4:44 PM, Greg Sutcliffe > wrote: >> On 20 March 2014 15:21, Dominic Cleal wrote: >>>> Where, besides DNS, do you think we need plugins such as above? Would >>>> be useful to see a few various cases in order to derive a useful >>>> abstraction for this. I think the responsibility for loading >>>> providers, etc should reside with plugins themselves, for a variety of >>>> reasons and we could keep the api for smart-proxy modules and >>>> providers separate. >> >> To give some example from outside the current codebase - I can see >> space for Discovery implementing custom proxy routes for interacting >> with Discovered hosts. Right now we (ab)use the BMC provider to send >> the reboot command to the host when it's provisioned, but I'd *love* >> to be able to eliminate the direct Foreman->Host communcation that's >> currently required. >> >> My templates-via-proxy PR(s) are another example where we might >> benefit from a plugin - Dmitri, can this this proposal listen on >> multiple ports, or are will still single-threaded here? > > It's going to listen on a single port. It *should* be possible to > build and launch several rack apps, each listening on a different > port, however. > -d > >> >> Greg >> >> -- >> You received this message because you are subscribed to the Google Groups "foreman-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout.
16

If you have time…

  • [Optional] As a developer, I'd like to be able to listen on multiple ports
  • [Optional] As a developer, I'd like to be able to assign certain
    features to certain ports
17

After a quick chat with Greg I propose the re-phrase the stories
suggested by Greg:

  • As a developer, I'd like to be able to configure a module (or
    perhaps even a single route) to listen on http, https, or both ports.

-d

··· On Fri, Mar 21, 2014 at 10:52 AM, Greg Sutcliffe wrote: > If you have time... > > - [Optional] As a developer, I'd like to be able to listen on multiple ports > - [Optional] As a developer, I'd like to be able to assign certain > features to certain ports > > -- > You received this message because you are subscribed to the Google Groups "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout.
18

+1

··· On 21 March 2014 11:25, Dmitri Dolguikh wrote: > After a quick chat with Greg I propose the re-phrase the stories > suggested by Greg: > - As a developer, I'd like to be able to configure a module (or > perhaps even a single route) to listen on http, https, or both ports.