The Katello nightlies, and release pipeline have been busted for almost a
week now and we are happy to report are back running successfully.
Normally, broken nightlies shouldn't occur due to the gated release
pipeline, however, since Katello relies on other plugins, their releases
can impact Katello. Below is a summary of the compounding issues that
contributed to the outage. For a historical view of pipeline failures and
issues, see [1].
Auto-attach on activation key broke custom products expectation during
registration
Modified bats test to account for this, scheduled deep dive to
discuss impact of this change
Katello Commit with scoped search broke bats, no test coverage present
fixed and tests added
foreman_discovery locking to 1.7 broke nightlies that use develop
removed foreman_discovery as a Katello dependency
Puppet 3.7.4 broke puppet-service_wait that wasn't using systemd
provider on EL7
puppet-service_wait updated to include a systemd based provider as
the default for EL7
Update to puppet-certs to add foreman_proxy client certs broke install
on el6
puppet-certs patched to account for similar cert names by checking
up to the version/release in the RPM name
Foreman CSS cleanup removed mixin Katello was relying on (broke RPM
builds)
Katello CSS patched and cleaned up to account for this change
>
> All,
>
> The Katello nightlies, and release pipeline have been busted for almost a
> week now and we are happy to report are back running successfully.
> Normally, broken nightlies shouldn't occur due to the gated release
> pipeline, however, since Katello relies on other plugins, their releases
> can impact Katello. Below is a summary of the compounding issues that
> contributed to the outage. For a historical view of pipeline failures and
> issues, see [1].
>
>
> - Auto-attach on activation key broke custom products expectation
> during registration
> - Modified bats test to account for this, scheduled deep dive to
> discuss impact of this change
>
>
> - Katello Commit with scoped search broke bats, no test coverage
> present
> - fixed and tests added
>
>
> - foreman_discovery locking to 1.7 broke nightlies that use develop
> - removed foreman_discovery as a Katello dependency
>
> is that on the stable branch or develop branch of discovery? if its not,
we should make sure we align branches?
>
> - Puppet 3.7.4 broke puppet-service_wait that wasn't using systemd
> provider on EL7
> - puppet-service_wait updated to include a systemd based provider
> as the default for EL7
>
>
> - Update to puppet-certs to add foreman_proxy client certs broke
> install on el6
> - puppet-certs patched to account for similar cert names by
> checking up to the version/release in the RPM name
>
>
> - Foreman CSS cleanup removed mixin Katello was relying on (broke RPM
> builds)
> - Katello CSS patched and cleaned up to account for this change
>
> How can we solve this in the future? also, does this limit your support
(e.g. works on 1.7 but no on nightly or the other way around?)
thanks,
Ohad
ยทยทยท
On Thursday, February 5, 2015 at 12:52:07 AM UTC+2, Eric Helms wrote:
