Katello Nightly Outage Report


TLDR - Katello nightlies are up and running.

For the past week the Katello nightlies have been broken due either to
broken builds or broken installation. The following is a summary of the
causes and the changes therein.

Problems Encountered:

  1. sass-rails moved to from foreman RPM to foreman-assets
  2. sass upgraded by Foreman which requires Katello to upgrade haml and
    haml-rails which forced an upgrade to compass and compass-rails
  3. Updating both sass-rails and less-rails to only be used in the
    production environment for asset pipe-ling. The subsequent breakage that
    occurred in development given less-rails/sass-rails are needed during asset
    compilation which occurs in the production environment and has slightly
    different requirements than in development.
  4. An update to mongodb for EL7 broke installation

Changes Made:

  1. Bastion updated to remove check for being in rake task and including
    less-rails during RPM build just for asset compilation. less-rails still
    required in development environment.
  2. Similar change made for Katello with respect to sass-rails.
  3. Removed the use of ui-alchemy-rails and compass from Katello and
    subsequently pulled in necessary jquery libraries still needed by legacy
  4. Upgraded haml and haml-rails as required by the newer version of sass.
  5. Updated puppet-pulp to account for the difference in mongodb –
  6. Katello release workflow in Jenkins updated to prevent broken nightlies
    from being pushed to the release repositories, see follow up email that
    discusses these changes in more depth.

Special thanks to jsherrill and inecas for providing help fixing these