SCL, puppet and proxies

In the plan to use Software Collections (SCL) for the 1.2 release with
our Ruby 1.9.3 stack, I think we need to examine how we integrate with
Puppet. I'm concerned that we might force people to use an SCL-enabled
Puppet RPM supplied by us, which means they can't use Puppet as supplied
by EPEL, Fedora or Puppet Labs. I think this would be a huge
disappointment and problem for many people who can't use the latest version.

We require Puppet in three places: Foreman core, the proxy and the
puppetmaster. The core usage of Puppet is minimal (it loads settings
etc, see #2414), but since we definitely want Foreman core on SCL I
think it's unavoidable that we ship ruby193-puppet-3.1.1 to provide this
small function. Currently we recommend using Passenger under Apache to
run Foreman (I'm assuming our installer setup for the remainder of this
e-mail).

The proxy requires Puppet to parse modules etc, so it's very helpful for
this to match the puppetmaster version. The puppetmaster lastly also
runs in Passenger under the same Apache instance as Foreman.

If we want to allow the puppetmaster to be non-SCL and Foreman to be
SCL, I'm not sure if these can co-exist within the same Apache instance,
since LD_LIBRARY_PATH and others need setting for SCL enabled Passenger.

What I'm proposing is a different default setup:

  1. Ship Foreman core as SCL enabled with ruby193-puppet
  2. Run Foreman under Thin with a variable number of workers (Katello
    does this today), again SCL-enabled.
  3. Ship the proxy as non-SCL, so it can use normal versions of Puppet
  4. Run the puppetmaster as non-SCL under Passenger in Apache
  5. Configure a reverse proxy in Apache for the Thin instances

This allows us to use the puppetmaster and proxy with normal Puppet
packages, but lets Foreman run in SCL mode. Moving from Passenger to
Thin isn't ideal, but I think it's the best option.

Thoughts?

··· -- Dominic Cleal Red Hat Engineering

> In the plan to use Software Collections (SCL) for the 1.2 release with
> our Ruby 1.9.3 stack, I think we need to examine how we integrate with
> Puppet. I'm concerned that we might force people to use an SCL-enabled
> Puppet RPM supplied by us, which means they can't use Puppet as supplied
> by EPEL, Fedora or Puppet Labs. I think this would be a huge
> disappointment and problem for many people who can't use the latest
> version.

> We require Puppet in three places: Foreman core, the proxy and the
> puppetmaster. The core usage of Puppet is minimal (it loads settings
> etc, see #2414), but since we definitely want Foreman core on SCL I
> think it's unavoidable that we ship ruby193-puppet-3.1.1 to provide this
> small function. Currently we recommend using Passenger under Apache to
> run Foreman (I'm assuming our installer setup for the remainder of this
> e-mail).
>
> The proxy requires Puppet to parse modules etc, so it's very helpful for
> this to match the puppetmaster version. The puppetmaster lastly also
> runs in Passenger under the same Apache instance as Foreman.
>
> If we want to allow the puppetmaster to be non-SCL and Foreman to be
> SCL, I'm not sure if these can co-exist within the same Apache instance,
> since LD_LIBRARY_PATH and others need setting for SCL enabled Passenger.
>
> What I'm proposing is a different default setup:
>
> 1. Ship Foreman core as SCL enabled with ruby193-puppet
>
+1

> 2. Run Foreman under Thin with a variable number of workers (Katello
> does this today), again SCL-enabled.
>
±0… I think I would prefer passenger standalone similar to [1], as we
don't need to handle the GC issues thin brings.

> 3. Ship the proxy as non-SCL, so it can use normal versions of Puppet
>
+1

> 4. Run the puppetmaster as non-SCL under Passenger in Apache
>
+1

Ohad
[1]
http://blog.phusion.nl/2010/09/21/phusion-passenger-running-multiple-ruby-versions/

··· On Tue, May 21, 2013 at 1:23 PM, Dominic Cleal wrote:

I agree with all 4 of Ohad's points.

> From: "Dominic Cleal" <dcleal@redhat.com>
> To: foreman-dev@googlegroups.com
> Sent: Tuesday, May 21, 2013 8:59:49 AM
> Subject: Re: [foreman-dev] SCL, puppet and proxies
>
> > What I'm proposing is a different default setup:
> >
> > 1. Ship Foreman core as SCL enabled with ruby193-puppet
> >
> > +1
> >
> > 2. Run Foreman under Thin with a variable number of workers (Katello
> > does this today), again SCL-enabled.
> >
> > ±0… I think I would prefer passenger standalone similar to [1], as we
> > don't need to handle the GC issues thin brings.
> > [1]
> > http://blog.phusion.nl/2010/09/21/phusion-passenger-running-multiple-ruby-versions/
>
> I'm ignorant of this, so thanks for the suggestion. That looks quite
> usable and we can just adapt the init script which currently supports thin.
>
> The tasks then to implement this are:
>
> * update foreman.init in core to support launching passenger
> * update passenger module to install passenger-standalone
> * replicate foreman::config::passenger for standalone usage
> * configure the foreman module to use the standalone class on SCL systems
> * build proxy and its dependencies as non-SCL
>
> Martin has worked out how to build the proxy as non-SCL via tito and
> koji, so we can document this process and make that part of the release.
>
> I'll file some issues to get this done, unless there are any more
> suggestions or objections.

Assign some of those to me :slight_smile:

··· ----- Original Message ----- > On 21/05/13 12:40, Ohad Levy wrote: > > On Tue, May 21, 2013 at 1:23 PM, Dominic Cleal > > wrote:

Thanks Ohad.


Dominic Cleal
Red Hat Engineering


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I'm ignorant of this, so thanks for the suggestion. That looks quite
usable and we can just adapt the init script which currently supports thin.

The tasks then to implement this are:

  • update foreman.init in core to support launching passenger
  • update passenger module to install passenger-standalone
  • replicate foreman::config::passenger for standalone usage
  • configure the foreman module to use the standalone class on SCL systems
  • build proxy and its dependencies as non-SCL

Martin has worked out how to build the proxy as non-SCL via tito and
koji, so we can document this process and make that part of the release.

I'll file some issues to get this done, unless there are any more
suggestions or objections.

Thanks Ohad.

··· On 21/05/13 12:40, Ohad Levy wrote: > On Tue, May 21, 2013 at 1:23 PM, Dominic Cleal > wrote: > What I'm proposing is a different default setup: > > 1. Ship Foreman core as SCL enabled with ruby193-puppet > > +1 > > 2. Run Foreman under Thin with a variable number of workers (Katello > does this today), again SCL-enabled. > > +-0.. I think I would prefer passenger standalone similar to [1], as we > don't need to handle the GC issues thin brings. > [1] http://blog.phusion.nl/2010/09/21/phusion-passenger-running-multiple-ruby-versions/


Dominic Cleal
Red Hat Engineering

> The tasks then to implement this are:
>
> * update foreman.init in core to support launching passenger

http://projects.theforeman.org/issues/2529

> * update passenger module to install passenger-standalone
> * replicate foreman::config::passenger for standalone usage
> * configure the foreman module to use the standalone class on SCL systems

http://projects.theforeman.org/issues/2530

> * build proxy and its dependencies as non-SCL

http://projects.theforeman.org/issues/2531

··· On 21/05/13 13:59, Dominic Cleal wrote:


Dominic Cleal
Red Hat Engineering

Just asking - are we sure we will be able to run these two guys without
SCL? I mean, aren't there dragons?

LZ

··· On Tue, May 21, 2013 at 02:40:15PM +0300, Ohad Levy wrote: > > 3. Ship the proxy as non-SCL, so it can use normal versions of Puppet > > > +1 > > > 4. Run the puppetmaster as non-SCL under Passenger in Apache > > > +1


Later,

Lukas “lzap” Zapletal
irc: lzap #theforeman

A downside that's become apparent is that passenger-standalone isn't
available in Fedora or EPEL, so we would go back to using the
stealthymonkeys.com repository. It's also not a wonderful package,
shipping a copy of nginx internally.

Another interesting development is that Passenger 4 can support multiple
Ruby versions, so we could have the puppetmaster vhost running under the
system Ruby, while the Foreman vhost runs under SCL. This would be an
ideal solution I think.

http://blog.phusion.nl/2012/10/24/phusion-passenger-4-0-beta-1-is-here/

Unfortunately neither stealthymonkeys or EPEL have published Passenger 4
RPMs yet, so we would need to package this new major release and try to
upstream the specs back to Fedora.

··· On 21/05/13 13:59, Dominic Cleal wrote: > On 21/05/13 12:40, Ohad Levy wrote: >> > On Tue, May 21, 2013 at 1:23 PM, Dominic Cleal > > > wrote: >> > What I'm proposing is a different default setup: >> > >> > 1. Ship Foreman core as SCL enabled with ruby193-puppet >> > >> > +1 >> > >> > 2. Run Foreman under Thin with a variable number of workers (Katello >> > does this today), again SCL-enabled. >> > >> > +-0.. I think I would prefer passenger standalone similar to [1], as we >> > don't need to handle the GC issues thin brings. >> > [1] http://blog.phusion.nl/2010/09/21/phusion-passenger-running-multiple-ruby-versions/ > > I'm ignorant of this, so thanks for the suggestion. That looks quite > usable and we can just adapt the init script which currently supports thin.


Dominic Cleal
Red Hat Engineering

Very few, the requirements for the proxy are extremely minimal so it'll
be easier to keep running on Ruby 1.8. It requires sinatra, json and
optionally rubyipmi and rkerberos.

The puppetmaster continues to be supported on Ruby 1.8.7 or higher and
won't be an issue.

··· On 21/05/13 20:04, Lukas Zapletal wrote: > On Tue, May 21, 2013 at 02:40:15PM +0300, Ohad Levy wrote: >>> 3. Ship the proxy as non-SCL, so it can use normal versions of Puppet >>> >> +1 >> >>> 4. Run the puppetmaster as non-SCL under Passenger in Apache >>> >> +1 > > Just asking - are we sure we will be able to run these two guys without > SCL? I mean, aren't there dragons?


Dominic Cleal
Red Hat Engineering