Foreman 1.2 packaging tasks

Development
1

For the first RC of Foreman 1.2, I'd like to ensure we have a basic RPM
installation that works using software collections (SCL), preferably
using foreman-installer. Debian and Ubuntu builds are unchanged and
will work as they do today.

The EL6 x86_64 build is mostly ready but the EL5 and 32-bit platforms
aren't and SCL-enabled Passenger is also missing. I'm going to consider
these lower priority (add them to RC2) and in the meantime get RC1
packages out with this configuration:

EL6 x86_64, Debian Squeeze and Wheezy (amd64 and i386), Ubuntu 12.04
(amd64 and i386)
SCL on EL6
Webrick on EL6, Passenger on Debian and Ubuntu

I've put together a few lists of things that need doing. If you can
chip in, please do.

High priority (before RC1 release)

··· ============= 1. foreman RPM spec needs a refresh * https://github.com/theforeman/foreman/pull/565 + fixes noted in my comment + https://github.com/domcleal/foreman/commit/1d7cffa4 + Sam's recent fixes * ensure SCL init.d enhancements are in our RPM
  1. proxy RPM spec
  • needs a refresh for 1.1
  • move into smart-proxy repo for use with tito?
  • confirm proxy remains non-SCL?
  1. rkerberos needs building in koji (proxy dependency)
  1. dbmigrate needs fixes
  • using bundle exec, skip for bundler_ext
  • ignores SCL (we could perhaps do this in the installer)

Medium priority (before RC2 release)

  1. ruby193-rubygem-mysql{,2} need building in koji

  2. Passenger needs building

  • LD_LIBRARY_PATH tweaks from jomara and/or OpenShift
  1. installer
  • report foreman.rb location changes if puppetmaster is in SCL
  • passenger package name will change
  • change generate_answers.rb shebang: /usr/bin/env ruby
  • change foreman-installer package dependencies for ruby193
  1. SELinux subpackage

Lower priority (before 1.2 release?)

  1. EL5 32/64-bit, EL6 32-bit packages need building

  2. sign RPMs

  3. update foreman-release version

  4. nightly RPMs built from develop

  • run tito on CI to trigger build jobs in koji
  • publish resulting repo to yum.tf.org/nightly

–
Dominic Cleal
Red Hat Engineering

2

Looks good. 1 additional deb task - the security token rake task is still
commented out in the deb constructs, due to a bug we found at 1.1 release.
I assume this is now fixed?

Greg

3

Completed, thanks Sam!

··· On 13/05/13 23:13, Dominic Cleal wrote: > High priority (before RC1 release) > ============= > 1. foreman RPM spec needs a refresh > * https://github.com/theforeman/foreman/pull/565 > + fixes noted in my comment > + https://github.com/domcleal/foreman/commit/1d7cffa4 > + Sam's recent fixes > * ensure SCL init.d enhancements are in our RPM

–
Dominic Cleal
Red Hat Engineering

4

Added to apt/<os>/nightly for all 6 os/arch combos. We'll need to remember
to promote these to rc/stable as required. Sinatra 1.3.6 has been placed in
the wheezy nightly repo to counter the 1.3.2 duplicate networks bug - this
will also need promoting.

··· On 13 May 2013 23:13, Dominic Cleal wrote:
  1. rkerberos needs building in koji (proxy dependency)
5

I've been working this week on rebuilding the RPMs in the Katello koji
(foreman-nightly-rhel6 target) for the latest develop, which will then
apply equally to 1.2. If somebody gets time to carry on and make
progress, that'd be appreciated!

There were a few changes I had to make to develop, namely moving rel-eng
to a top level dir (tito can't handle it), adding gettext_i18n_rails_js
(a recently added dep) and a couple small spec file fixes. This branch
is usable with tito for scratch builds:
https://github.com/domcleal/foreman/compare/develop-builds

(spec and comps changes can be merged in develop once we've got it
building.)

I haven't had a successful build of this yet due to the
gettext_i18n_rails_js dependencies not being there for assets
precompilation. I've been working through its dependencies and building
them into Koji's foreman-nightly-rhel6 tag. The one I'm stuck on is
ruby193-rubygem-gettext, which is failing while executing find-lang.sh
during build as it can't find .mo files in the source tree:
http://koji.katello.org/koji/getfile?taskID=35642&name=build.log

Though with some debug, you can see the .mo files are present, so I'm
not sure what's going on yet:
http://koji.katello.org/koji/getfile?taskID=35644&name=build.log

My current patch to the rubygem-gettext.spec is here:
http://pastie.org/private/nc5hmtdeg6k5kkve90qsg

It disables testing since we're not building the gems required for that
(and we're not really using this lib much), but fails on the %find_lang.

Cheers,

··· On 13/05/13 23:13, Dominic Cleal wrote: > 1. foreman RPM spec needs a refresh > * https://github.com/theforeman/foreman/pull/565 > + fixes noted in my comment > + https://github.com/domcleal/foreman/commit/1d7cffa4 > + Sam's recent fixes > * ensure SCL init.d enhancements are in our RPM

–
Dominic Cleal
Red Hat Engineering

6

> For the first RC of Foreman 1.2, I'd like to ensure we have a basic RPM
> installation that works using software collections (SCL), preferably
> using foreman-installer. Debian and Ubuntu builds are unchanged and
> will work as they do today.
>
> The EL6 x86_64 build is mostly ready but the EL5 and 32-bit platforms
> aren't and SCL-enabled Passenger is also missing. I'm going to consider
> these lower priority (add them to RC2) and in the meantime get RC1
> packages out with this configuration:
>
> EL6 x86_64, Debian Squeeze and Wheezy (amd64 and i386), Ubuntu 12.04
> (amd64 and i386)
> SCL on EL6
> Webrick on EL6, Passenger on Debian and Ubuntu
>
> I've put together a few lists of things that need doing. If you can
> chip in, please do.
>
>
> High priority (before RC1 release)
> =============
> 1. foreman RPM spec needs a refresh
> * Updated the spec for packaging Foreman on top of a Ruby 1.9.3 SCL by skottler · Pull Request #565 · theforeman/foreman · GitHub
> + fixes noted in my comment
> + Add missing comps entries · domcleal/foreman@1d7cffa · GitHub
> + Sam's recent fixes
> * ensure SCL init.d enhancements are in our RPM
>
> 2. proxy RPM spec
> * needs a refresh for 1.1
> * move into smart-proxy repo for use with tito?
> * confirm proxy remains non-SCL?
I started with smart-proxy.

My plan is to take related spec files from foreman-rpms repo, merge them
to have one spec for all Fedora/RHEL releases and add supoprt for tito.
There are some issues I'd like to discuss:

  • the rpm is called foreman-proxy while the git repo is called
    smart-proxy. Is there reason for this? Shouldn't that be unified?
  • who should decide / what it depands on whether the proxy should be
    packaged with or without SCL?

Cheers,
Martin

··· On 05/14/2013 12:13 AM, Dominic Cleal wrote:

  1. rkerberos needs building in koji (proxy dependency)

  2. dbmigrate needs fixes

    • using bundle exec, skip for bundler_ext
    • ignores SCL (we could perhaps do this in the installer)

Medium priority (before RC2 release)

  1. ruby193-rubygem-mysql{,2} need building in koji

  2. Passenger needs building

    • LD_LIBRARY_PATH tweaks from jomara and/or OpenShift

  3. installer

    • report foreman.rb location changes if puppetmaster is in SCL
    • passenger package name will change
    • change generate_answers.rb shebang: /usr/bin/env ruby
    • change foreman-installer package dependencies for ruby193

  4. SELinux subpackage

Lower priority (before 1.2 release?)

  1. EL5 32/64-bit, EL6 32-bit packages need building

  2. sign RPMs

  3. update foreman-release version

  4. nightly RPMs built from develop

    • run tito on CI to trigger build jobs in koji
    • publish resulting repo to yum.tf.org/nightly
7

> For the first RC of Foreman 1.2, I'd like to ensure we have a basic RPM
> installation that works using software collections (SCL), preferably
> using foreman-installer. Debian and Ubuntu builds are unchanged and
> will work as they do today.
>
> The EL6 x86_64 build is mostly ready but the EL5 and 32-bit platforms
> aren't and SCL-enabled Passenger is also missing. I'm going to consider
> these lower priority (add them to RC2) and in the meantime get RC1
> packages out with this configuration:
>
> EL6 x86_64, Debian Squeeze and Wheezy (amd64 and i386), Ubuntu 12.04
> (amd64 and i386)
> SCL on EL6
> Webrick on EL6, Passenger on Debian and Ubuntu
>
> I've put together a few lists of things that need doing. If you can
> chip in, please do.
>
>
> High priority (before RC1 release)
> =============
> 1. foreman RPM spec needs a refresh
> * https://github.com/theforeman/foreman/pull/565
> + fixes noted in my comment
> + https://github.com/domcleal/foreman/commit/1d7cffa4
> + Sam's recent fixes
> * ensure SCL init.d enhancements are in our RPM
>
> 2. proxy RPM spec
> * needs a refresh for 1.1
> * move into smart-proxy repo for use with tito?
> * confirm proxy remains non-SCL?
>
> 3. rkerberos needs building in koji (proxy dependency)
> * https://github.com/domcleal/foreman-rpms/tree/rkerberos for EL6
> * Greg has this in progress for Debian, Ubuntu
>
> 4. dbmigrate needs fixes
> * using bundle exec, skip for bundler_ext
> * ignores SCL (we could perhaps do this in the installer)
>
>
> Medium priority (before RC2 release)
> ===============
> 1. ruby193-rubygem-mysql{,2} need building in koji
Done. I added both to katello-thirdparty and built in koji.

What from the list bellow needs to be worked on next?

M.

··· On 05/14/2013 12:13 AM, Dominic Cleal wrote: > > 2. Passenger needs building > * LD_LIBRARY_PATH tweaks from jomara and/or OpenShift > > 3. installer > * report foreman.rb location changes if puppetmaster is in SCL > * passenger package name will change > * change generate_answers.rb shebang: /usr/bin/env ruby > * change foreman-installer package dependencies for ruby193 > > 4. SELinux subpackage > > > Lower priority (before 1.2 release?) > ============== > 1. EL5 32/64-bit, EL6 32-bit packages need building > > 2. sign RPMs > > 3. update foreman-release version > > 4. nightly RPMs built from develop > * run tito on CI to trigger build jobs in koji > * publish resulting repo to yum.tf.org/nightly >
8

What was the bug? :slight_smile:

··· On 13/05/13 23:34, Greg Sutcliffe wrote: > Looks good. 1 additional deb task - the security token rake task is > still commented out in the deb constructs, due to a bug we found at 1.1 > release. I assume this is now fixed?

–
Dominic Cleal
Red Hat Engineering

9

>> For the first RC of Foreman 1.2, I'd like to ensure we have a basic RPM
>> installation that works using software collections (SCL), preferably
>> using foreman-installer. Debian and Ubuntu builds are unchanged and
>> will work as they do today.
>>
>> The EL6 x86_64 build is mostly ready but the EL5 and 32-bit platforms
>> aren't and SCL-enabled Passenger is also missing. I'm going to consider
>> these lower priority (add them to RC2) and in the meantime get RC1
>> packages out with this configuration:
>>
>> EL6 x86_64, Debian Squeeze and Wheezy (amd64 and i386), Ubuntu 12.04
>> (amd64 and i386)
>> SCL on EL6
>> Webrick on EL6, Passenger on Debian and Ubuntu
>>
>> I've put together a few lists of things that need doing. If you can
>> chip in, please do.
>>
>>
>> High priority (before RC1 release)
>> =============
>> 1. foreman RPM spec needs a refresh
>> * https://github.com/theforeman/foreman/pull/565
>> + fixes noted in my comment
>> + https://github.com/domcleal/foreman/commit/1d7cffa4
>> + Sam's recent fixes
>> * ensure SCL init.d enhancements are in our RPM
>>
>> 2. proxy RPM spec
>> * needs a refresh for 1.1
>> * move into smart-proxy repo for use with tito?
>> * confirm proxy remains non-SCL?
> I started with smart-proxy.
>
> My plan is to take related spec files from foreman-rpms repo, merge
> them to have one spec for all Fedora/RHEL releases and add supoprt for
> tito. There are some issues I'd like to discuss:
>
> - the rpm is called foreman-proxy while the git repo is called
> smart-proxy. Is there reason for this? Shouldn't that be unified?
> - who should decide / what it depands on whether the proxy should be
> packaged with or without SCL?
>
>
Smart-proxy status:

  • naming kept as it was -> foreman-proxy package in smart-proxy repo.
  • spec is part of the smart-proxy repo
  • support for scl added
  • support for tito added

>
>>
>> 3. rkerberos needs building in koji (proxy dependency)
>> * https://github.com/domcleal/foreman-rpms/tree/rkerberos for EL6
>> * Greg has this in progress for Debian, Ubuntu
>>
I'll go on with rkerberos if it is not taken already

M

··· On 05/16/2013 11:09 AM, Martin Bačovský wrote: > On 05/14/2013 12:13 AM, Dominic Cleal wrote: >> 4. dbmigrate needs fixes >> * using bundle exec, skip for bundler_ext >> * ignores SCL (we could perhaps do this in the installer) >> >> >> Medium priority (before RC2 release) >> =============== >> 1. ruby193-rubygem-mysql{,2} need building in koji >> >> 2. Passenger needs building >> * LD_LIBRARY_PATH tweaks from jomara and/or OpenShift >> >> 3. installer >> * report foreman.rb location changes if puppetmaster is in SCL >> * passenger package name will change >> * change generate_answers.rb shebang: /usr/bin/env ruby >> * change foreman-installer package dependencies for ruby193 >> >> 4. SELinux subpackage >> >> >> Lower priority (before 1.2 release?) >> ============== >> 1. EL5 32/64-bit, EL6 32-bit packages need building >> >> 2. sign RPMs >> >> 3. update foreman-release version >> >> 4. nightly RPMs built from develop >> * run tito on CI to trigger build jobs in koji >> * publish resulting repo to yum.tf.org/nightly >> >
10

This is now finished, so the remaining spec changes got merged into
develop last night:

I'm now able to build Foreman on Koji from develop, with a few small
tweaks which you can carry out locally. I've documented the process
here for both scratch and release builds using tito and Katello's Koji:

http://projects.theforeman.org/projects/foreman/wiki/RPM_Packaging#foreman

Once we've published the SRPMs for 1.2.0, I hope we're able to document
a local build using only mock, so anybody can follow this without
needing Koji.

I've also been writing up the entire release process as I go,
suggestions are particularly welcome here:

http://projects.theforeman.org/projects/foreman/wiki/Release_Process

··· On 14/05/13 23:32, Dominic Cleal wrote: > On 13/05/13 23:13, Dominic Cleal wrote: >> 1. foreman RPM spec needs a refresh >> * https://github.com/theforeman/foreman/pull/565 >> + fixes noted in my comment >> + https://github.com/domcleal/foreman/commit/1d7cffa4 >> + Sam's recent fixes >> * ensure SCL init.d enhancements are in our RPM > > I've been working this week on rebuilding the RPMs in the Katello koji > (foreman-nightly-rhel6 target) for the latest develop, which will then > apply equally to 1.2. If somebody gets time to carry on and make > progress, that'd be appreciated!

–
Dominic Cleal
Red Hat Engineering

11

Can't recall. Something that came up at the eleventh hour during RC testing
that we disabled and then forgot to enable. I can test it some time this
week if you remind me what I'm looking for :wink:

··· On 13 May 2013 23:40, Dominic Cleal wrote:

On 13/05/13 23:34, Greg Sutcliffe wrote:

Looks good. 1 additional deb task - the security token rake task is
still commented out in the deb constructs, due to a bug we found at 1.1
release. I assume this is now fixed?

What was the bug? :slight_smile:

12

>>> For the first RC of Foreman 1.2, I'd like to ensure we have a basic RPM
>>> installation that works using software collections (SCL), preferably
>>> using foreman-installer. Debian and Ubuntu builds are unchanged and
>>> will work as they do today.
>>>
>>> The EL6 x86_64 build is mostly ready but the EL5 and 32-bit platforms
>>> aren't and SCL-enabled Passenger is also missing. I'm going to consider
>>> these lower priority (add them to RC2) and in the meantime get RC1
>>> packages out with this configuration:
>>>
>>> EL6 x86_64, Debian Squeeze and Wheezy (amd64 and i386), Ubuntu 12.04
>>> (amd64 and i386)
>>> SCL on EL6
>>> Webrick on EL6, Passenger on Debian and Ubuntu
>>>
>>> I've put together a few lists of things that need doing. If you can
>>> chip in, please do.
>>>
>>>
>>> High priority (before RC1 release)
>>> =============
>>> 1. foreman RPM spec needs a refresh
>>> * https://github.com/theforeman/foreman/pull/565
>>> + fixes noted in my comment
>>> + https://github.com/domcleal/foreman/commit/1d7cffa4
>>> + Sam's recent fixes
>>> * ensure SCL init.d enhancements are in our RPM
>>>
>>> 2. proxy RPM spec
>>> * needs a refresh for 1.1
>>> * move into smart-proxy repo for use with tito?
>>> * confirm proxy remains non-SCL?
>> I started with smart-proxy.
>>
>> My plan is to take related spec files from foreman-rpms repo, merge
>> them to have one spec for all Fedora/RHEL releases and add supoprt for
>> tito. There are some issues I'd like to discuss:
>>
>> - the rpm is called foreman-proxy while the git repo is called
>> smart-proxy. Is there reason for this? Shouldn't that be unified?
>> - who should decide / what it depands on whether the proxy should be
>> packaged with or without SCL?
>>
>>
> Smart-proxy status:
> - naming kept as it was -> foreman-proxy package in smart-proxy repo.
> - spec is part of the smart-proxy repo
> - support for scl added
> - support for tito added

Excellent, thanks Martin!

>>
>>>
>>> 3. rkerberos needs building in koji (proxy dependency)
>>> * https://github.com/domcleal/foreman-rpms/tree/rkerberos for EL6
>>> * Greg has this in progress for Debian, Ubuntu
>>>
> I'll go on with rkerberos if it is not taken already

Let me know if you hit any issues with it, it worked well on EL6 but
hasn't been SCLed yet.

··· On 17/05/13 11:52, Martin Bačovský wrote: > On 05/16/2013 11:09 AM, Martin Bačovský wrote: >> On 05/14/2013 12:13 AM, Dominic Cleal wrote:

–
Dominic Cleal
Red Hat Engineering