Final touches to merge the foreman and katello installers

Hi all,

I've been asked to look into making sure we don't have two
independant, separate installers. As of right now, the
katello-installer is still a separate entity, but it re-uses all the
upstream foreman-installer modukes. This mail is to talk about the
last few things needed to make the katello-installer an extension of
the foreman-installer.

I spoke with Eric about this last week, and it seemed to us that the
current katello-installer simply lays down some extra puppet modules
on top of the foreman-installer ones, and uses a separate script to
call the install (enabling the extra modules and passing config to
them). Therefore if we:

  • Change the RPM build to only include the additional modules, and
    depend on foreman-installer for the base ones
  • Lay down a new script which merely calls the foreman-installer
    script with a different config file and answers file

I think that's all we actually need to achieve a complete integration,
where the katello-installer is an extension of the foreman-installer,
in much the same the way as Katello itself is an extension of Foreman.
Then we're maintaining one installer, and Katello gets the benefit of
changes to the installation script in foreman-installer without any
patch cherry-picking.

Does that sound feasible? Did I miss something?

Cheers,
Greg

>
> Hi all,
>
> I've been asked to look into making sure we don't have two
> independant, separate installers. As of right now, the
> katello-installer is still a separate entity, but it re-uses all the
> upstream foreman-installer modukes. This mail is to talk about the
> last few things needed to make the katello-installer an extension of
> the foreman-installer.
>
> I spoke with Eric about this last week, and it seemed to us that the
> current katello-installer simply lays down some extra puppet modules
> on top of the foreman-installer ones, and uses a separate script to
> call the install (enabling the extra modules and passing config to
> them). Therefore if we:
>
> * Change the RPM build to only include the additional modules, and
> depend on foreman-installer for the base ones
> * Lay down a new script which merely calls the foreman-installer
> script with a different config file and answers file
>
> I think that's all we actually need to achieve a complete integration,
> where the katello-installer is an extension of the foreman-installer,
> in much the same the way as Katello itself is an extension of Foreman.
> Then we're maintaining one installer, and Katello gets the benefit of
> changes to the installation script in foreman-installer without any
> patch cherry-picking.
>
> Does that sound feasible? Did I miss something?
>

Does that mean there are two puppet runs? one for foreman and another for
other puppet modules?

··· On Monday, March 10, 2014 1:56:10 PM UTC+2, Greg Sutcliffe wrote:

Cheers,
Greg

No. The foreman-installer already supports being told to read a
different config file, so arguably you could just do

foreman-installer -c katello-config.yaml # where katello-config.yaml
specifies katello-answers.yaml too

However, to make it nice to package and nice for users, we can easily
have a wrapper:

/usr/bin/katello-installer:
#!/bin/bash

foreman-installer -c /etc/foreman/katello-config.yaml
EOF

That way we pass the full config of all modules (via the specified
config/answers file) to Kafo, and get a single run.

··· On 10 March 2014 12:11, ohadlevy wrote: > Does that mean there are two puppet runs? one for foreman and another for > other puppet modules?

That seems decent. If that is all that it is, seems great.

– bk

··· On 03/10/2014 08:27 AM, Greg Sutcliffe wrote: > On 10 March 2014 12:11, ohadlevy wrote: >> Does that mean there are two puppet runs? one for foreman and another for >> other puppet modules? > > No. The foreman-installer already supports being told to read a > different config file, so arguably you could just do > > foreman-installer -c katello-config.yaml # where katello-config.yaml > specifies katello-answers.yaml too > > However, to make it nice to package and nice for users, we can easily > have a wrapper: > > /usr/bin/katello-installer: > #!/bin/bash > > foreman-installer -c /etc/foreman/katello-config.yaml > EOF > > That way we pass the full config of all modules (via the specified > config/answers file) to Kafo, and get a single run. >

We already have code to read the config file based on file name1 so
why not have katello-installer as a symlink to foreman-installer?

··· On Mon, Mar 10, 2014 at 12:27:22PM +0000, Greg Sutcliffe wrote: > On 10 March 2014 12:11, ohadlevy wrote: > > Does that mean there are two puppet runs? one for foreman and another for > > other puppet modules? > > No. The foreman-installer already supports being told to read a > different config file, so arguably you could just do > > foreman-installer -c katello-config.yaml # where katello-config.yaml > specifies katello-answers.yaml too > > However, to make it nice to package and nice for users, we can easily > have a wrapper: > > /usr/bin/katello-installer: > #!/bin/bash > > foreman-installer -c /etc/foreman/katello-config.yaml > EOF > > That way we pass the full config of all modules (via the specified > config/answers file) to Kafo, and get a single run.

Ha, I'd forgotten about that - even easier to package! :slight_smile:

··· On 10 March 2014 12:47, Ewoud Kohl van Wijngaarden wrote: > We already have code to read the config file based on file name[1] so > why not have katello-installer as a symlink to foreman-installer? > > [1]: https://github.com/theforeman/foreman-installer/blob/develop/bin/foreman-installer#L7-L17

> > We already have code to read the config file based on file name[1] so
> > why not have katello-installer as a symlink to foreman-installer?
> >
> > [1]:
> https://github.com/theforeman/foreman-installer/blob/develop/bin/foreman-installer#L7-L17
>

I think we may have to update the Katello installer answer files to fit
this format (e.g.
https://github.com/Katello/katello-installer/tree/master/config) but that
should be trivial. By and large I think this strategy should work, but I'd
like to test it first in a development like setup as well which we often do
for testing the Katello installer (e.g. doing a git checkout of the
katello-installer repo) since this strategy (I believe) assumes that
modules all live in the same directory.

And just to be clear, as we should document this a strategy for users, we
are saying to users the recommended way to provide your own config and
modules (i.e. extend or multi-plex installation types) to the
foreman-installer is to:

  1. Deploy your modules to the same directory as the Foreman modules
  2. Create your answers files
  3. Create a thin script with a name of your choosing to call the
    foreman-installer

Eric

··· On Mon, Mar 10, 2014 at 8:52 AM, Greg Sutcliffe wrote: > On 10 March 2014 12:47, Ewoud Kohl van Wijngaarden > wrote:

Ha, I’d forgotten about that - even easier to package! :slight_smile:


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

> I think we may have to update the Katello installer answer files to fit this
> format (e.g.
> https://github.com/Katello/katello-installer/tree/master/config) but that
> should be trivial. By and large I think this strategy should work, but I'd
> like to test it first in a development like setup as well which we often do
> for testing the Katello installer (e.g. doing a git checkout of the
> katello-installer repo) since this strategy (I believe) assumes that modules
> all live in the same directory.

It does, yes.

> And just to be clear, as we should document this a strategy for users, we
> are saying to users the recommended way to provide your own config and
> modules (i.e. extend or multi-plex installation types) to the
> foreman-installer is to:
>
> 1. Deploy your modules to the same directory as the Foreman modules
> 2. Create your answers files
> 3. Create a thin script with a name of your choosing to call the
> foreman-installer

Pretty much, although (2) should read "s/answers/config and answers/",
and as Ewoud points out, we can make (3) the thinnest of scripts - a
symlink :slight_smile:

Greg

··· On 10 March 2014 19:31, Eric D Helms wrote:

So after some investigation and some discussion, it seems like it's
not worth the effort to do this. Let me justify that…

We had 2 action items, I'll explain each in turn:

  1. Overlaying katello's extra modules on top of the foreman installer

This is kind-of already done. The katello-installer package uses a
Puppetfile to fetch the modules at build time, and so is using the
same sources as the foreman-installer package. Additionally, if we
make katello-installer depend upon foreman-installer we have to solve
(a) the deduction problem in which we need to determine what modules
are already present in foreman-installer when building
katello-installer (to avoid duplicate files), and (b) ensuring that
the underlying foreman-installer rpm supplies perfectly compatible
modules to the katello modules.

(a) is quite solvable, but (b) is not so easy - simply assuming a
specific version of the foreman-installer rpm is considerably less
flexible than simply continuing to use the Puppetfile in the
katello-installer repo today. This is because the Puppetfile can lock
individual modules (consider where puppet-foreman 2.2 release breaks
katello-installer, so the Puppetfile in katello-installer can be
changed to require version 2.1).

So, given the katello-installer rpm is already consuming the upstream
puppet modules in the exact same way as foreman-installer, and retains
more flexibility and less problems by not being an overlay, I vote we
leave it at the status quo.

  1. Answers files, config files, and executables

Building on the conclusion of (1), it now seems silly to depend upon
the foreman-installer rpm just to be able to symlink to
/usr/sbin/foreman-installer. The differences in the two scripts as
they exist today are minimal (the foreman-installer script has a lot
of foreman-only things that don't make sense to port- although a
modifed version of --reset-db would be good). Additionally, we always
said we would need to ship the current answers and config files, so
again, it seems the current state of things is fine.

So basically, I feel we're already as integrated as we need to be, for
now. Both installers consume the same upstream modules, so there
should be no duplication of effort within the Puppet code itself. That
accounts for >95% of the codebase. There is a small item for
discussion about we sync things in the exectuable script, and
optionally the checks/ directory, between the two repos, but this
seems like a small overhead, and should probably be done manually.

For the record, I did an install using the method outlined in my
original post (using the foreman-installer rpm, plus the extra
modules, plus a symlink to foreman-installer) and it worked fine. So
did just installing katello-installer and running it - the two things
are effectively identical. We're really just discussing packaging and
future flexibility.

I hope that's fairly clear - it's been tough to put into words because
I've see-sawed back and forth on the right way forward myself :slight_smile:

Eric, Dominic, Ivan, you guys are fairly close to the respective
installers - what do you think?

Greg