Proxy determination + HA

Hi,

I'm working on how we determine the Remote Execution Proxy to use for a
particular host. There's currently a bit of magic involved that wouldn't work
in multi-proxy/complex environments well.

Full description with screenshots are in this PR, with a new approach:
https://github.com/theforeman/foreman_remote_execution/pull/33

Your feedback would be very valuable, particularly:

  1. How does this setup feel from a usability perspective? Any suggestions?

  2. Should you also be able to assign remote execution proxies directly to
    hosts? The default action would always be inherit from subnet, but a user could
    want in some cases some one-off overrides.

  3. Are there environments where this setup will not work well?

  4. Any concerns about integration with Katello?

··· -- Best Regards,

Stephen Benjamin
Red Hat Engineering

> Hi,
>
> I'm working on how we determine the Remote Execution Proxy to use for a
> particular host. There's currently a bit of magic involved that wouldn't
> work
> in multi-proxy/complex environments well.
>
> Full description with screenshots are in this PR, with a new approach:
> https://github.com/theforeman/foreman_remote_execution/pull/33
>
> Your feedback would be very valuable, particularly:
>
> 1. How does this setup feel from a usability perspective? Any suggestions?
>

This keeps on coming up in multiple contexts, for example:

  1. the proxy to use for provisioning/templates
  2. the puppet master to use (could use multple, not just one)
  3. the pulp server to fetch content from
  4. the scap server to send reports to

I feel the location attribute needs to be extended, to resolve per proxy
feature.

>
> 2. Should you also be able to assign remote execution proxies directly to
> hosts? The default action would always be inherit from subnet, but a user
> could
> want in some cases some one-off overrides.
>

no, but you could let the user select a proxy (as in some advance hidden
place)?

>
> 3. Are there environments where this setup will not work well?
>

> 4. Any concerns about integration with Katello?
>

what exactly do you mean?

Ohad

··· On Wed, Sep 2, 2015 at 8:46 PM, Stephen Benjamin wrote:


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

> Hi,
>
> I'm working on how we determine the Remote Execution Proxy to use for a
> particular host. There's currently a bit of magic involved that wouldn't work
> in multi-proxy/complex environments well.
>
> Full description with screenshots are in this PR, with a new approach:
> https://github.com/theforeman/foreman_remote_execution/pull/33
>
> Your feedback would be very valuable, particularly:
>
> 1. How does this setup feel from a usability perspective? Any suggestions?
>
> 2. Should you also be able to assign remote execution proxies directly to
> hosts? The default action would always be inherit from subnet, but a user could
> want in some cases some one-off overrides.
>
> 3. Are there environments where this setup will not work well?
>
> 4. Any concerns about integration with Katello?

Thought about this a tad bit, and I think the following workflow is very
important for katello:

  • run some command on an existing system (such as subscription-manager
    register, yum install, whatever to bootstrap the client)
  • be able to execute remote commands against the new host

in most cases the smart-proxy/capsule the host registered to will be the
same as the one hosting the remote_execution proxy.

At registration time, we should know what client interface (ip address)
the registration request came in on and what smart-proxy it used. The
tricky part in your scenario is handling all the subnets, especially
when considering orgs and locations. I'm not sure we want to try to
create a subnet or fix mismatches during registration time. For this
case a richer location feature would be useful, although i agree that
would be a feature in and of itself.

Possibly we could fallback to content source if no interfaces have the
execution flagged checked and/or a subnet is not properly configured.

There is also the crazy thought of using this for things like roaming
laptops where the system's ip address may change from day to day (or
hour to hour). In that case the ip address is really not important (and
likely we would be using a different remote execution technology).

-Justin

··· On 09/02/2015 01:46 PM, Stephen Benjamin wrote:


Best Regards,

Stephen Benjamin
Red Hat Engineering

>
> > Hi,
> >
> > I'm working on how we determine the Remote Execution Proxy to use for a
> > particular host. There's currently a bit of magic involved that wouldn't
> > work
> > in multi-proxy/complex environments well.
> >
> > Full description with screenshots are in this PR, with a new approach:
> > https://github.com/theforeman/foreman_remote_execution/pull/33
> >
> > Your feedback would be very valuable, particularly:
> >
> > 1. How does this setup feel from a usability perspective? Any suggestions?
> >
>
> This keeps on coming up in multiple contexts, for example:
> 1. the proxy to use for provisioning/templates
> 2. the puppet master to use (could use multple, not just one)
> 3. the pulp server to fetch content from
> 4. the scap server to send reports to
> …
>
> I feel the location attribute needs to be extended, to resolve per proxy
> feature.

Or just use the subnet?

I think subnet applies to any of those, as the network segment is
probably the most relevant thing when asking the question "What proxy
should this host talk to?"

I can see why using locations as an additional layer of abstraction
might be useful - but I think this makes the average user's life too
difficult.

>
> >
> > 2. Should you also be able to assign remote execution proxies directly to
> > hosts? The default action would always be inherit from subnet, but a user
> > could
> > want in some cases some one-off overrides.
> >
>
> no, but you could let the user select a proxy (as in some advance hidden
> place)?

Ok, I'll look for a good place to hide something like that, any ideas?
Maybe an 'Advanced' button inside the hosts form? Or a tab hidden by a
setting?

>
> >
> > 3. Are there environments where this setup will not work well?
> >
>
> > 4. Any concerns about integration with Katello?
> >
>
> what exactly do you mean?

Anything here that might make it difficult for Katello to integrate. We
haven't discussed plugins in detail yet, but it was worth asking now in
case someone saw something obvious.

One thing was Katello has this "Content Source" selection which is
basically "this host's capsule" - is that relevant for REX? I would say
no and perhaps Katello should use subnets someday, but maybe others have
a different perspective.

··· On Wed, Sep 02, 2015 at 09:11:18PM +0300, Ohad Levy wrote: > On Wed, Sep 2, 2015 at 8:46 PM, Stephen Benjamin wrote:

Ohad


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering

> >
> > > Hi,
> > >
> > > I'm working on how we determine the Remote Execution Proxy to use for a
> > > particular host. There's currently a bit of magic involved that
> wouldn't
> > > work
> > > in multi-proxy/complex environments well.
> > >
> > > Full description with screenshots are in this PR, with a new approach:
> > > https://github.com/theforeman/foreman_remote_execution/pull/33
> > >
> > > Your feedback would be very valuable, particularly:
> > >
> > > 1. How does this setup feel from a usability perspective? Any
> suggestions?
> > >
> >
> > This keeps on coming up in multiple contexts, for example:
> > 1. the proxy to use for provisioning/templates
> > 2. the puppet master to use (could use multple, not just one)
> > 3. the pulp server to fetch content from
> > 4. the scap server to send reports to
> > …
> >
> > I feel the location attribute needs to be extended, to resolve per proxy
> > feature.
>
> Or just use the subnet?
>
> I think subnet applies to any of those, as the network segment is
> probably the most relevant thing when asking the question "What proxy
> should this host talk to?"
>
> I can see why using locations as an additional layer of abstraction
> might be useful - but I think this makes the average user's life too
> difficult.
>

I would probably want to think about this a bit more, somehow the Location
"feels" more generic object for this kind of usage cases, and since we have
inheritance, it could come in handy (instead of defining it per subnet), it
also allows multple proxies vs one.

>
> >
> > >
> > > 2. Should you also be able to assign remote execution proxies directly
> to
> > > hosts? The default action would always be inherit from subnet, but a
> user
> > > could
> > > want in some cases some one-off overrides.
> > >
> >
> > no, but you could let the user select a proxy (as in some advance hidden
> > place)?
>
> Ok, I'll look for a good place to hide something like that, any ideas?
> Maybe an 'Advanced' button inside the hosts form? Or a tab hidden by a
> setting?
>

I was thinking per job execution vs yet another host property.

>
> >
> > >
> > > 3. Are there environments where this setup will not work well?
> > >
> >
> > > 4. Any concerns about integration with Katello?
> > >
> >
> > what exactly do you mean?
>
> Anything here that might make it difficult for Katello to integrate. We
> haven't discussed plugins in detail yet, but it was worth asking now in
> case someone saw something obvious.
>
> One thing was Katello has this "Content Source" selection which is
> basically "this host's capsule" - is that relevant for REX? I would say
> no and perhaps Katello should use subnets someday, but maybe others have
> a different perspective.
>

Yes, i agree, this is some how the same conversation, I do feel that up
until now the flow of lets add another proxy to host/hostgroup (puppet,
puppetca, realm, salt, capsule…) was good enough, but i belive its
time to build something more generic, maybe with some sort of general
purpose abstraction ? for sure we can do better then yet another dropdown
in the host creation/edit.

··· On Wed, Sep 2, 2015 at 9:30 PM, Stephen Benjamin wrote: > On Wed, Sep 02, 2015 at 09:11:18PM +0300, Ohad Levy wrote: > > On Wed, Sep 2, 2015 at 8:46 PM, Stephen Benjamin > wrote:

Ohad


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google
Groups

“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send
an

email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

> >Hi,
> >
> >I'm working on how we determine the Remote Execution Proxy to use for a
> >particular host. There's currently a bit of magic involved that wouldn't work
> >in multi-proxy/complex environments well.
> >
> >Full description with screenshots are in this PR, with a new approach:
> > https://github.com/theforeman/foreman_remote_execution/pull/33
> >
> >Your feedback would be very valuable, particularly:
> >
> >1. How does this setup feel from a usability perspective? Any suggestions?
> >
> >2. Should you also be able to assign remote execution proxies directly to
> >hosts? The default action would always be inherit from subnet, but a user could
> >want in some cases some one-off overrides.
> >
> >3. Are there environments where this setup will not work well?
> >
> >4. Any concerns about integration with Katello?
>
> Thought about this a tad bit, and I think the following workflow is very
> important for katello:
>
> * run some command on an existing system (such as subscription-manager
> register, yum install, whatever to bootstrap the client)
> * be able to execute remote commands against the new host
>
>
> in most cases the smart-proxy/capsule the host registered to will be the
> same as the one hosting the remote_execution proxy.
>
>
> At registration time, we should know what client interface (ip address) the
> registration request came in on and what smart-proxy it used. The tricky
> part in your scenario is handling all the subnets, especially when
> considering orgs and locations.
>
>
> I'm not sure we want to try to create a subnet or fix mismatches
> during registration time. For this case a richer
> location feature would be useful, although i agree that would be a
> feature in and of itself.
>
>
> Possibly we could fallback to content source if no interfaces have the
> execution flagged checked and/or a subnet is not properly configured.

I think that's OK, for the Katello use case we look at the Content
Source. We can also make a guess about the subnet (not create - just
check if it exists already), but Content Source gives us a reliable
fallback.

>
>
> There is also the crazy thought of using this for things like roaming
> laptops where the system's ip address may change from day to day (or hour to
> hour). In that case the ip address is really not important (and likely we
> would be using a different remote execution technology).

Yea, that's a more complicated case - you really need a better topology
like that provided by Salt Syndic or our existing Dispatch Router setup
where it doesn't much matter where the client connects, or where the
execution is initiated.

··· On Tue, Sep 08, 2015 at 11:05:24AM -0400, Justin Sherrill wrote: > On 09/02/2015 01:46 PM, Stephen Benjamin wrote:


Best Regards,

Stephen Benjamin
Red Hat Engineering

>
> > >
> > > > Hi,
> > > >
> > > > I'm working on how we determine the Remote Execution Proxy to use for a
> > > > particular host. There's currently a bit of magic involved that
> > wouldn't
> > > > work
> > > > in multi-proxy/complex environments well.
> > > >
> > > > Full description with screenshots are in this PR, with a new approach:
> > > > https://github.com/theforeman/foreman_remote_execution/pull/33
> > > >
> > > > Your feedback would be very valuable, particularly:
> > > >
> > > > 1. How does this setup feel from a usability perspective? Any
> > suggestions?
> > > >
> > >
> > > This keeps on coming up in multiple contexts, for example:
> > > 1. the proxy to use for provisioning/templates
> > > 2. the puppet master to use (could use multple, not just one)
> > > 3. the pulp server to fetch content from
> > > 4. the scap server to send reports to
> > > …
> > >
> > > I feel the location attribute needs to be extended, to resolve per proxy
> > > feature.
> >
> > Or just use the subnet?
> >
> > I think subnet applies to any of those, as the network segment is
> > probably the most relevant thing when asking the question "What proxy
> > should this host talk to?"
> >
> > I can see why using locations as an additional layer of abstraction
> > might be useful - but I think this makes the average user's life too
> > difficult.
> >
>
> I would probably want to think about this a bit more, somehow the Location
> "feels" more generic object for this kind of usage cases, and since we have
> inheritance, it could come in handy (instead of defining it per subnet), it
> also allows multple proxies vs one.

The PR allows many execution proxies to one subnet, that's the
foundation of how I pictured us doing HA. It's very simple.
Taxonomies are not simple, by any means.

··· On Wed, Sep 02, 2015 at 09:37:04PM +0300, Ohad Levy wrote: > On Wed, Sep 2, 2015 at 9:30 PM, Stephen Benjamin wrote: > > On Wed, Sep 02, 2015 at 09:11:18PM +0300, Ohad Levy wrote: > > > On Wed, Sep 2, 2015 at 8:46 PM, Stephen Benjamin > > wrote:
  1. Should you also be able to assign remote execution proxies directly
    to

hosts? The default action would always be inherit from subnet, but a
user

could
want in some cases some one-off overrides.

no, but you could let the user select a proxy (as in some advance hidden
place)?

Ok, I’ll look for a good place to hide something like that, any ideas?
Maybe an ‘Advanced’ button inside the hosts form? Or a tab hidden by a
setting?

I was thinking per job execution vs yet another host property.

  1. Are there environments where this setup will not work well?
  1. Any concerns about integration with Katello?

what exactly do you mean?

Anything here that might make it difficult for Katello to integrate. We
haven’t discussed plugins in detail yet, but it was worth asking now in
case someone saw something obvious.

One thing was Katello has this “Content Source” selection which is
basically “this host’s capsule” - is that relevant for REX? I would say
no and perhaps Katello should use subnets someday, but maybe others have
a different perspective.

Yes, i agree, this is some how the same conversation, I do feel that up
until now the flow of lets add another proxy to host/hostgroup (puppet,
puppetca, realm, salt, capsule…) was good enough, but i belive its
time to build something more generic, maybe with some sort of general
purpose abstraction ? for sure we can do better then yet another dropdown
in the host creation/edit.

Ohad


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google
Groups

“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send
an

email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering

> >
> > > >
> > > > > Hi,
> > > > >
> > > > > I'm working on how we determine the Remote Execution Proxy to use for
> > > > > a
> > > > > particular host. There's currently a bit of magic involved that
> > > wouldn't
> > > > > work
> > > > > in multi-proxy/complex environments well.
> > > > >
> > > > > Full description with screenshots are in this PR, with a new
> > > > > approach:
> > > > > https://github.com/theforeman/foreman_remote_execution/pull/33
> > > > >
> > > > > Your feedback would be very valuable, particularly:
> > > > >
> > > > > 1. How does this setup feel from a usability perspective? Any
> > > suggestions?
> > > > >
> > > >
> > > > This keeps on coming up in multiple contexts, for example:
> > > > 1. the proxy to use for provisioning/templates
> > > > 2. the puppet master to use (could use multple, not just one)
> > > > 3. the pulp server to fetch content from
> > > > 4. the scap server to send reports to
> > > > …
> > > >
> > > > I feel the location attribute needs to be extended, to resolve per
> > > > proxy
> > > > feature.
> > >
> > > Or just use the subnet?
> > >
> > > I think subnet applies to any of those, as the network segment is
> > > probably the most relevant thing when asking the question "What proxy
> > > should this host talk to?"
> > >
> > > I can see why using locations as an additional layer of abstraction
> > > might be useful - but I think this makes the average user's life too
> > > difficult.
> > >
> >
> > I would probably want to think about this a bit more, somehow the Location
> > "feels" more generic object for this kind of usage cases, and since we have
> > inheritance, it could come in handy (instead of defining it per subnet), it
> > also allows multple proxies vs one.
>
> The PR allows many execution proxies to one subnet, that's the
> foundation of how I pictured us doing HA. It's very simple.
> Taxonomies are not simple, by any means.
>
> >
> > >
> > > >
> > > > >
> > > > > 2. Should you also be able to assign remote execution proxies
> > > > > directly
> > > to
> > > > > hosts? The default action would always be inherit from subnet, but a
> > > user
> > > > > could
> > > > > want in some cases some one-off overrides.
> > > > >
> > > >
> > > > no, but you could let the user select a proxy (as in some advance
> > > > hidden
> > > > place)?
> > >
> > > Ok, I'll look for a good place to hide something like that, any ideas?
> > > Maybe an 'Advanced' button inside the hosts form? Or a tab hidden by a
> > > setting?
> > >
> >
> > I was thinking per job execution vs yet another host property.
> >
> > >
> > > >
> > > > >
> > > > > 3. Are there environments where this setup will not work well?
> > > > >
> > > >
> > > > > 4. Any concerns about integration with Katello?
> > > > >
> > > >
> > > > what exactly do you mean?
> > >
> > > Anything here that might make it difficult for Katello to integrate. We
> > > haven't discussed plugins in detail yet, but it was worth asking now in
> > > case someone saw something obvious.
> > >
> > > One thing was Katello has this "Content Source" selection which is
> > > basically "this host's capsule" - is that relevant for REX? I would say
> > > no and perhaps Katello should use subnets someday, but maybe others have
> > > a different perspective.
> > >
> >
> > Yes, i agree, this is some how the same conversation, I do feel that up
> > until now the flow of lets add another proxy to host/hostgroup (puppet,
> > puppetca, realm, salt, capsule…) was good enough, but i belive its
> > time to build something more generic, maybe with some sort of general
> > purpose abstraction ? for sure we can do better then yet another dropdown
> > in the host creation/edit.

I there a plan on tackling this from all the different angles?
The proposed solution seems to work pretty well for the remote execution
(with question mark on the Katello use-case: that's why we ask the folks
with opinions on the behavior from this side).

Would it make sense to go the simpler solution first (taking the things
that we already have), and looking at the bigger picture, involving
the more generic solution afterwards?

I've felt for some time, that assigning proxies to the hosts directly
doesn't scale. Using subnets seems to work, although one might question,
if the relation between subnets and proxies isn't just accidental?
Maybe a silly question, but couldn't there be even more subnets,
just to achieve the HA on the networking side? Also, we assigning puppet master
or content source to hosts directly. Would it make sense to do so via network,
or would it be a misuse of the object to something that it's not reflecting the reality?

The most important questions for me are:

  • wouldn't introducing additional layer make it more complicated for users to understand
  • would the new concept be added to the foreman in timely fashion (Foreman 1.10)

From our experience, it's hard to make this kind of changes, that affect the whole core,
without proper justification, corresponding design, discussion and general acceptance.
Things like "we need it for our plugin" are usually not enough.

Therefore, I tent to approve the proposed solution and in the meantime, getting
more folks together from all the areas that touches this topic, to make the effort of
unifying the proxy selection (and maybe they will just agree, that using subnets
makes most sense). From remote execution point of view, it's "just" a question
of one migration, once the new concept is there.

– Ivan

··· ----- Original Message ----- > On Wed, Sep 02, 2015 at 09:37:04PM +0300, Ohad Levy wrote: > > On Wed, Sep 2, 2015 at 9:30 PM, Stephen Benjamin > > wrote: > > > On Wed, Sep 02, 2015 at 09:11:18PM +0300, Ohad Levy wrote: > > > > On Wed, Sep 2, 2015 at 8:46 PM, Stephen Benjamin > > > wrote:

Ohad


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google
Groups

“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it,
send
an

email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

This might be covered somewhere else and is possibly too much of an edge case to be considered at the moment. What about portable systems that can relocate around a network with the user. The majority of systems will be static in their location, but my Linux Workstation roams around the network with me. At the moment under Spacewalk/Sat5 thankfully I a) don't often move to a VLAN that can't contact my normal Proxy and b) can manually rejig things if and when I do.
Is there anything to cope with movable systems roaming around a network, connecting to VLANs that cannot see the regular Proxy etc?

D

>
>
> > >
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm working on how we determine the Remote Execution Proxy to use for
> > > > > > a
> > > > > > particular host. There's currently a bit of magic involved that
> > > > wouldn't
> > > > > > work
> > > > > > in multi-proxy/complex environments well.
> > > > > >
> > > > > > Full description with screenshots are in this PR, with a new
> > > > > > approach:
> > > > > > https://github.com/theforeman/foreman_remote_execution/pull/33
> > > > > >
> > > > > > Your feedback would be very valuable, particularly:
> > > > > >
> > > > > > 1. How does this setup feel from a usability perspective? Any
> > > > suggestions?
> > > > > >
> > > > >
> > > > > This keeps on coming up in multiple contexts, for example:
> > > > > 1. the proxy to use for provisioning/templates
> > > > > 2. the puppet master to use (could use multple, not just one)
> > > > > 3. the pulp server to fetch content from
> > > > > 4. the scap server to send reports to
> > > > > …
> > > > >
> > > > > I feel the location attribute needs to be extended, to resolve per
> > > > > proxy
> > > > > feature.
> > > >
> > > > Or just use the subnet?
> > > >
> > > > I think subnet applies to any of those, as the network segment is
> > > > probably the most relevant thing when asking the question "What proxy
> > > > should this host talk to?"
> > > >
> > > > I can see why using locations as an additional layer of abstraction
> > > > might be useful - but I think this makes the average user's life too
> > > > difficult.
> > > >
> > >
> > > I would probably want to think about this a bit more, somehow the Location
> > > "feels" more generic object for this kind of usage cases, and since we have
> > > inheritance, it could come in handy (instead of defining it per subnet), it
> > > also allows multple proxies vs one.
> >
> > The PR allows many execution proxies to one subnet, that's the
> > foundation of how I pictured us doing HA. It's very simple.
> > Taxonomies are not simple, by any means.
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > 2. Should you also be able to assign remote execution proxies
> > > > > > directly
> > > > to
> > > > > > hosts? The default action would always be inherit from subnet, but a
> > > > user
> > > > > > could
> > > > > > want in some cases some one-off overrides.
> > > > > >
> > > > >
> > > > > no, but you could let the user select a proxy (as in some advance
> > > > > hidden
> > > > > place)?
> > > >
> > > > Ok, I'll look for a good place to hide something like that, any ideas?
> > > > Maybe an 'Advanced' button inside the hosts form? Or a tab hidden by a
> > > > setting?
> > > >
> > >
> > > I was thinking per job execution vs yet another host property.
> > >
> > > >
> > > > >
> > > > > >
> > > > > > 3. Are there environments where this setup will not work well?
> > > > > >
> > > > >
> > > > > > 4. Any concerns about integration with Katello?
> > > > > >
> > > > >
> > > > > what exactly do you mean?
> > > >
> > > > Anything here that might make it difficult for Katello to integrate. We
> > > > haven't discussed plugins in detail yet, but it was worth asking now in
> > > > case someone saw something obvious.
> > > >
> > > > One thing was Katello has this "Content Source" selection which is
> > > > basically "this host's capsule" - is that relevant for REX? I would say
> > > > no and perhaps Katello should use subnets someday, but maybe others have
> > > > a different perspective.
> > > >
> > >
> > > Yes, i agree, this is some how the same conversation, I do feel that up
> > > until now the flow of lets add another proxy to host/hostgroup (puppet,
> > > puppetca, realm, salt, capsule…) was good enough, but i belive its
> > > time to build something more generic, maybe with some sort of general
> > > purpose abstraction ? for sure we can do better then yet another dropdown
> > > in the host creation/edit.
>
>
> I there a plan on tackling this from all the different angles?
> The proposed solution seems to work pretty well for the remote execution
> (with question mark on the Katello use-case: that's why we ask the folks
> with opinions on the behavior from this side).
>
> Would it make sense to go the simpler solution first (taking the things
> that we already have), and looking at the bigger picture, involving
> the more generic solution afterwards?
>
> I've felt for some time, that assigning proxies to the hosts directly
> doesn't scale. Using subnets seems to work, although one might question,
> if the relation between subnets and proxies isn't just accidental?
> Maybe a silly question, but couldn't there be even more subnets,
> just to achieve the HA on the networking side?

HA would normally be done by using bonded/teamed interfaces and
redundancy at the router and switch level. From Foreman's perspective
that just means marking that bond as the Execution interface.

For hosts in multiple networks, each network would be dedicated to a
particular purpose like storage, management, etc. In this scenario a
"management" network is what we're most interested in.

It would be atypical for one host to talk to Foreman across two
different networks. If that needed to happen - say someone spilled
their ramen noodles on the management switches, you can change the
execution interface to use a different network path.

> Also, we assigning puppet master or content source to hosts directly.
> Would it make sense to do so via network, or would it be a misuse of
> the object to something that it's not reflecting the reality?

It makes sense to me to use the network for these types of things, and
from a network perspective the smallest unit above host is going to be a
subnet.

You could use Locations to somehow implement this as well. I'd rather if
effort was going to be invested in taxonomies it would be improving the
usability of what they already are supposed to do, instead of adding
more. For Remote Execution, we also have to solve the problem with what
we have available today (or available very very soon).

··· On Thu, Sep 03, 2015 at 07:48:58AM -0400, Ivan Necas wrote: > ----- Original Message ----- > > On Wed, Sep 02, 2015 at 09:37:04PM +0300, Ohad Levy wrote: > > > On Wed, Sep 2, 2015 at 9:30 PM, Stephen Benjamin > > > wrote: > > > > On Wed, Sep 02, 2015 at 09:11:18PM +0300, Ohad Levy wrote: > > > > > On Wed, Sep 2, 2015 at 8:46 PM, Stephen Benjamin > > > > wrote:

The most important questions for me are:

  • wouldn’t introducing additional layer make it more complicated for users to understand
  • would the new concept be added to the foreman in timely fashion (Foreman 1.10)

From our experience, it’s hard to make this kind of changes, that affect the whole core,
without proper justification, corresponding design, discussion and general acceptance.
Things like “we need it for our plugin” are usually not enough.

Therefore, I tent to approve the proposed solution and in the meantime, getting
more folks together from all the areas that touches this topic, to make the effort of
unifying the proxy selection (and maybe they will just agree, that using subnets
makes most sense). From remote execution point of view, it’s “just” a question
of one migration, once the new concept is there.

– Ivan

Ohad


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google
Groups

“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it,
send
an

email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Best Regards,

Stephen Benjamin
Red Hat Engineering

>
>
> > >
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm working on how we determine the Remote Execution Proxy to use for
> > > > > > a
> > > > > > particular host. There's currently a bit of magic involved that
> > > > wouldn't
> > > > > > work
> > > > > > in multi-proxy/complex environments well.
> > > > > >
> > > > > > Full description with screenshots are in this PR, with a new
> > > > > > approach:
> > > > > > https://github.com/theforeman/foreman_remote_execution/pull/33
> > > > > >
> > > > > > Your feedback would be very valuable, particularly:
> > > > > >
> > > > > > 1. How does this setup feel from a usability perspective? Any
> > > > suggestions?
> > > > > >
> > > > >
> > > > > This keeps on coming up in multiple contexts, for example:
> > > > > 1. the proxy to use for provisioning/templates
> > > > > 2. the puppet master to use (could use multple, not just one)
> > > > > 3. the pulp server to fetch content from
> > > > > 4. the scap server to send reports to
> > > > > …
> > > > >
> > > > > I feel the location attribute needs to be extended, to resolve per
> > > > > proxy
> > > > > feature.
> > > >
> > > > Or just use the subnet?
> > > >
> > > > I think subnet applies to any of those, as the network segment is
> > > > probably the most relevant thing when asking the question "What proxy
> > > > should this host talk to?"
> > > >
> > > > I can see why using locations as an additional layer of abstraction
> > > > might be useful - but I think this makes the average user's life too
> > > > difficult.
> > > >
> > >
> > > I would probably want to think about this a bit more, somehow the Location
> > > "feels" more generic object for this kind of usage cases, and since we have
> > > inheritance, it could come in handy (instead of defining it per subnet), it
> > > also allows multple proxies vs one.
> >
> > The PR allows many execution proxies to one subnet, that's the
> > foundation of how I pictured us doing HA. It's very simple.
> > Taxonomies are not simple, by any means.
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > > 2. Should you also be able to assign remote execution proxies
> > > > > > directly
> > > > to
> > > > > > hosts? The default action would always be inherit from subnet, but a
> > > > user
> > > > > > could
> > > > > > want in some cases some one-off overrides.
> > > > > >
> > > > >
> > > > > no, but you could let the user select a proxy (as in some advance
> > > > > hidden
> > > > > place)?
> > > >
> > > > Ok, I'll look for a good place to hide something like that, any ideas?
> > > > Maybe an 'Advanced' button inside the hosts form? Or a tab hidden by a
> > > > setting?
> > > >
> > >
> > > I was thinking per job execution vs yet another host property.
> > >
> > > >
> > > > >
> > > > > >
> > > > > > 3. Are there environments where this setup will not work well?
> > > > > >
> > > > >
> > > > > > 4. Any concerns about integration with Katello?
> > > > > >
> > > > >
> > > > > what exactly do you mean?
> > > >
> > > > Anything here that might make it difficult for Katello to integrate. We
> > > > haven't discussed plugins in detail yet, but it was worth asking now in
> > > > case someone saw something obvious.
> > > >
> > > > One thing was Katello has this "Content Source" selection which is
> > > > basically "this host's capsule" - is that relevant for REX? I would say
> > > > no and perhaps Katello should use subnets someday, but maybe others have
> > > > a different perspective.
> > > >
> > >
> > > Yes, i agree, this is some how the same conversation, I do feel that up
> > > until now the flow of lets add another proxy to host/hostgroup (puppet,
> > > puppetca, realm, salt, capsule…) was good enough, but i belive its
> > > time to build something more generic, maybe with some sort of general
> > > purpose abstraction ? for sure we can do better then yet another dropdown
> > > in the host creation/edit.
>
>
> I there a plan on tackling this from all the different angles?
> The proposed solution seems to work pretty well for the remote execution
> (with question mark on the Katello use-case: that's why we ask the folks
> with opinions on the behavior from this side).
>
> Would it make sense to go the simpler solution first (taking the things
> that we already have), and looking at the bigger picture, involving
> the more generic solution afterwards?
>
> I've felt for some time, that assigning proxies to the hosts directly
> doesn't scale. Using subnets seems to work, although one might question,
> if the relation between subnets and proxies isn't just accidental?
> Maybe a silly question, but couldn't there be even more subnets,
> just to achieve the HA on the networking side? Also, we assigning puppet master
> or content source to hosts directly. Would it make sense to do so via network,
> or would it be a misuse of the object to something that it's not reflecting the reality?

How would you deal with hosts that have multiple subnets?

··· On Thu, Sep 03, 2015 at 07:48:58AM -0400, Ivan Necas wrote: > ----- Original Message ----- > > On Wed, Sep 02, 2015 at 09:37:04PM +0300, Ohad Levy wrote: > > > On Wed, Sep 2, 2015 at 9:30 PM, Stephen Benjamin > > > wrote: > > > > On Wed, Sep 02, 2015 at 09:11:18PM +0300, Ohad Levy wrote: > > > > > On Wed, Sep 2, 2015 at 8:46 PM, Stephen Benjamin > > > > wrote:

The most important questions for me are:

  • wouldn’t introducing additional layer make it more complicated for users to understand
  • would the new concept be added to the foreman in timely fashion (Foreman 1.10)

From our experience, it’s hard to make this kind of changes, that affect the whole core,
without proper justification, corresponding design, discussion and general acceptance.
Things like “we need it for our plugin” are usually not enough.

Therefore, I tent to approve the proposed solution and in the meantime, getting
more folks together from all the areas that touches this topic, to make the effort of
unifying the proxy selection (and maybe they will just agree, that using subnets
makes most sense). From remote execution point of view, it’s “just” a question
of one migration, once the new concept is there.

Yes, this is considered, but not currently being implemented http://theforeman.github.io/foreman_remote_execution/design/#SshSingleHostCheck-in, this case is a bit more specific, as it would require for the host to choose what proxy to use at the moment.

··· ----- Original Message ----- > This might be covered somewhere else and is possibly too much of an edge case > to be considered at the moment. What about portable systems that can > relocate around a network with the user. The majority of systems will be > static in their location, but my Linux Workstation roams around the network > with me. At the moment under Spacewalk/Sat5 thankfully I a) don't often move > to a VLAN that can't contact my normal Proxy and b) can manually rejig > things if and when I do. > Is there anything to cope with movable systems roaming around a network, > connecting to VLANs that cannot see the regular Proxy etc? > > D > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >