Call for assistance - Debian packaging

Support
1

Hi all,

For the last eighteen months I've been running the Debian packaging effort
for Foreman, and with contributions from around our community, we've got a
reasonable stable set of Foreman (and Proxy) packages that seem to work.

They're not perfect though, and certainly wouldn't be accepted by any
upsteam distribution, as they vendor gems within the packages. This was a
decision taken due to the amount of work necessary to build (and maintain)
every gem dependency of Foreman across 6 OSes (Wheezy, Squeeze, Precise, 32
and 64bit).

The vendoring solution works well for Foreman as it uses Bundler anyway,
but we have an increasing list of things which don't use Bundler, should
be packaged, and aren't:

  • Foreman-installer (properly packaged, not using fpm)
  • hammer (new foreman cli)
  • kafo (the very very new foreman-installer)
  • a variety of plugins

These packages all have their own gem dependencies which need to be checked
on each OS, and built if the they don't exist (or do exist, at the wrong
version). They need to be available at the system level, as we cannot
vendor things which are intended to be libraries in the first place.
Plugins particularly complicate the Foreman package's own vendoring
strategy, since they need to be in the same space as Foreman's own
dependant gems.

The RPM packaging effort has had some great efforts recently to automate
building and pushing of gem dependencies, and I'd love to get to a similar
state with the debs. But I cannot do it by myself - I simply do not have
the time. I've no intention to stop working on the Debian packages, but I
can recognise when the quantity of things that need doing is greater than
my capacity.

So, I'm calling out to all you lovely Debian users out there - if any of
you want to get involved in (re)building the Foreman packages with better
automation, now is the time to step up. Send me an email (or ping me on
IRC) to express your interest, and we'll try to get a meeting together
sometime soon to discuss what our options are and how we'll get it done.

In the meantime, I'll do some research on how upstream Debian is handling
mass gem build problems. We can't be the only ones in this situation.

Cheers,
Greg

2

> From: "Greg Sutcliffe" <greg.sutcliffe@gmail.com>
> To: "Foreman ." <foreman-users@googlegroups.com>, "Foreman ." <foreman-dev@googlegroups.com>
> Sent: Thursday, August 29, 2013 12:36:16 PM
> Subject: [foreman-dev] Call for assistance - Debian packaging
>
> Hi all,
>
> For the last eighteen months I've been running the Debian packaging effort
> for Foreman, and with contributions from around our community, we've got a
> reasonable stable set of Foreman (and Proxy) packages that seem to work.
>
> They're not perfect though, and certainly wouldn't be accepted by any
> upsteam distribution, as they vendor gems within the packages. This was a
> decision taken due to the amount of work necessary to build (and maintain)
> every gem dependency of Foreman across 6 OSes (Wheezy, Squeeze, Precise, 32
> and 64bit).

This is slightly OT, but have you thought at all about dropping 32-bit packages? I've heard maybe 2 people complain about the fact that we only support 64-bit for the RPM's. Cutting the matrix in half is a pretty nice thing to be able to do :slight_smile:

>
> The vendoring solution works well for Foreman as it uses Bundler anyway,
> but we have an increasing list of things which don't use Bundler, should
> be packaged, and aren't:
>
> * Foreman-installer (properly packaged, not using fpm)
> * hammer (new foreman cli)
> * kafo (the very very new foreman-installer)
> * a variety of plugins

I'm happy to help out in my free time. I worked recently on packaging hammer so I can pick that up as an initial task since it's pretty straightforward.

>
> These packages all have their own gem dependencies which need to be checked
> on each OS, and built if the they don't exist (or do exist, at the wrong
> version). They need to be available at the system level, as we cannot
> vendor things which are intended to be libraries in the first place.
> Plugins particularly complicate the Foreman package's own vendoring
> strategy, since they need to be in the same space as Foreman's own
> dependant gems.
>
> The RPM packaging effort has had some great efforts recently to automate
> building and pushing of gem dependencies, and I'd love to get to a similar
> state with the debs. But I cannot do it by myself - I simply do not have
> the time. I've no intention to stop working on the Debian packages, but I
> can recognise when the quantity of things that need doing is greater than
> my capacity.
>
> So, I'm calling out to all you lovely Debian users out there - if any of
> you want to get involved in (re)building the Foreman packages with better
> automation, now is the time to step up. Send me an email (or ping me on
> IRC) to express your interest, and we'll try to get a meeting together
> sometime soon to discuss what our options are and how we'll get it done.
>
> In the meantime, I'll do some research on how upstream Debian is handling
> mass gem build problems. We can't be the only ones in this situation.

They're in a state of transition over the using gem2deb for almost everything right now. Basically their whole build system (buildd.debian.org) is based around dput.

··· ----- Original Message -----

Cheers,
Greg

–
You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

> Hi all,
>
> For the last eighteen months I've been running the Debian packaging
> effort for Foreman, and with contributions from around our community,
> we've got a reasonable stable set of Foreman (and Proxy) packages that
> seem to work.
>
> They're not perfect though, and certainly wouldn't be accepted by any
> upsteam distribution, as they vendor gems within the packages. This was
> a decision taken due to the amount of work necessary to build (and
> maintain) every gem dependency of Foreman across 6 OSes (Wheezy,
> Squeeze, Precise, 32 and 64bit).
>
> The vendoring solution works well for Foreman as it uses Bundler anyway,
> but we have an increasing list of things which don't use Bundler,
> should be packaged, and aren't:
>
> * Foreman-installer (properly packaged, not using fpm)
> * hammer (new foreman cli)
> * kafo (the very very new foreman-installer)
> * a variety of plugins
>
> These packages all have their own gem dependencies which need to be
> checked on each OS, and built if the they don't exist (or do exist, at
> the wrong version). They need to be available at the system level, as we
> cannot vendor things which are intended to be libraries in the first
> place. Plugins particularly complicate the Foreman package's own
> vendoring strategy, since they need to be in the same space as Foreman's
> own dependant gems.

I would suggest using bundler for foreman-installer and hammer in the
short/medium term, as it doesn't appear that we have the capacity for
packaging dependencies fully.

These projects already use bundler for development, so with a few lines
(i.e. set BUNDLER_GEMFILE and require "bundler/setup") added to the bin/
script in the two projects we could support bundler as an optional tool
for production deployment.

Using standalone rather than deployment mode would help, as it avoids
using Gemfile.lock or bundle install/update, so would be more reliable.

This doesn't help with plugins for core or hammer (we'd need to vendor
hammer_cli_foreman specifically), but at least we'd get these two key
tools packaged for 1.3.

> The RPM packaging effort has had some great efforts recently to automate
> building and pushing of gem dependencies, and I'd love to get to a
> similar state with the debs. But I cannot do it by myself - I simply do
> not have the time. I've no intention to stop working on the Debian
> packages, but I can recognise when the quantity of things that need
> doing is greater than my capacity.

We get a lot of help with the RPMs due to two factors:

a) Rails and its dependencies are provided as part of the ruby193
software collection, so we don't repackage it ourselves.
b) We share a build system with Katello, so dependencies are often
shared too, saving some work.

It's quite an undertaking to package the whole lot again, as I'm sure
Sam or Jason could tell you.

> So, I'm calling out to all you lovely Debian users out there - if any of
> you want to get involved in (re)building the Foreman packages with
> better automation, now is the time to step up. Send me an email (or ping
> me on IRC) to express your interest, and we'll try to get a meeting
> together sometime soon to discuss what our options are and how we'll get
> it done.
>
> In the meantime, I'll do some research on how upstream Debian is
> handling mass gem build problems. We can't be the only ones in this
> situation.

I've got a few ideas about how to improve the Jenkins jobs so they're
more generic and reusable, and possibly the Freight usage to make it
easier to manage in progress builds versus published builds.

··· On 29/08/13 17:36, Greg Sutcliffe wrote:

–
Dominic Cleal
Red Hat Engineering

4

> I would suggest using bundler for foreman-installer and hammer in the
> short/medium term, as it doesn't appear that we have the capacity for
> packaging dependencies fully

+1 to that, but it's just a stop-gap, we still need to figure out an
automated gem build system.

> It's quite an undertaking to package the whole lot again, as I'm sure
> Sam or Jason could tell you.

I already know - I did it by hand for 1.0 on Squeeze, and when those
packages needed redoing for Precise, I gave up and vendored it.

> I've got a few ideas about how to improve the Jenkins jobs so they're
> more generic and reusable, and possibly the Freight usage to make it
> easier to manage in progress builds versus published builds.
>

I'd love to hear them. I'm think we should publish a hangout event (maybe
for early next week, or as a DD?) and brainstorm in person.

Greg

··· On 2 September 2013 08:30, Dominic Cleal wrote:
5

Hi all,

We had a Google Hangout yesterday to discuss some of the issues around the
way we currently build and store the Debian packages. You can find the
recording here[1] if you're interested, but I thought I'd summarize our
discussion for the list.

  • 3 components (stable/rc/nightly) doesn't provide for enough separation
    between stable releases. Starting from 1.3, we'll use versioned components
    (so wheezy/1.3). Current stable will be renamed to '1.2' when 1.3 is
    released, and 'stable' will be made a symlink to the latest release so
    existing installs work. 'nightly' remains as it is today. 'rc' disappears,
    with RC packages going into the new version's repo (but 'stable' link not
    updated until final release)
  • The foreman-packaging git repo will mirror the branching strategy used in
    core - one develop tree with long-lived branches cut for each release. This
    also mirrors the component names above.
  • A staging repo (probably stagingdeb.theforeman.org) will be provided for
    packages to build and push debs to for testing to avoid messing with the
    main repos. This will have the form "deb
    http://stagingdeb.theforeman.org/<distro> <jenkins-username>" e.g
    wheezy/gsutcliffe. These will be
    shortlived (maybe ~ a few days) and regularly cleaned.
  • Once the above is in place, we will begin building the gem dependencies
    for the foreman (and it's extras). The target is to package the
    foreman-installer with all dependencies for 1.3, and then use the
    experience gained from this to have the proxy and core split out into
    proper gem packages for 1.4.
  • Look into finding/writing a tool similar to repoclosure (for yum) which
    can identify missing dependencies in a repo. This will help in identifying
    missing gem packages. A list of packages currently being (list still being
    compiled) built can be found at [2]

One undiscussed item which may become relevant for 1.4 is that we may have
to drop support for 32bit, given the size of the build matrix. It's not
been decided yet though.

These actions should make it considerably easier to scratch-build, test,
build, release, and maintain debian packages going forward. If there is
anyone still wishing to get involved with the debian packaging effort, get
in touch and we can show you where to start.

Cheers,
Greg

[1]http://www.youtube.com/watch?v=nY18x2m18hI&feature=youtu.be
[2]
https://docs.google.com/spreadsheet/ccc?key=0AhcK94BNB5PedG1KT3pvY3ZVbXRybUwyRkNLeldNcWc#gid=0