Migration to Copr POC

Development
1

Hello,

I would like to test Fedora Copr capabilities in serving us as the
main build system instead our Koji instance. With assistance from
Mirek Suchy, lead Copr developer, I would like to do proof-of-concept
of integrating Copr into our workflow as a secondary build system
together with Koji. The idea is to run both in paralel for some time
to excercise integration and stability, then doing one release from
Copr while keeping Koji up and running. After one successful release,
we will be able to deprovision the Koji instance and all its builders.

Since Tito supports Copr, integration should be relatively easy. There
are some limitations we need to workaround. Comp files are being
uploaded via Copr user interface, there is no API/CLI yet but the
question is if we want to use comps at all. Might be better idea to
filter out packages during downloading it from Copr to our file
server. Second known limitation is that Copr uses its own embedded
signing procedure (everything is signed by default) while we will want
to sign releases by ourselves. Since we do this manually today, we can
simply sign packages again before uploading them to the target server.

The plan:

Delete existing testing Copr repositories in @theforeman Copr group,
it's not shown in the UI who created them but I believe it was Dominic
or Eric: https://copr.fedorainfracloud.org/groups/g/theforeman/coprs/

Create new project structure and set necessary build roots and SCL dependencies.

Rebuild all nightly packages so they are all built in Copr
successfully for the first time via tito. This will be a PR in
foreman-packaging with all necessary configuration changes. We will
create Etherpad page to track the progress.

Merge the tito configuration change and update Jenkins job to build
nightly on both Koji and Copr in parallel.

This is the first stage, the second one will be doing a proper release
from Copr rather than Koji. Ideally, this would be 1.16 bit if we
don't hit this one, we can take our time.

If you want to help, please let me know. The first step is to set
things up. Please keep Mirek in CC, if you have comments, questions or
concerns.

Useful links:



https://github.com/theforeman/foreman-packaging/blob/rpm/develop/
https://github.com/theforeman/foreman-packaging/blob/rpm/develop/rel-eng/tito.props
https://github.com/theforeman/foreman-packaging/blob/rpm/develop/rel-eng/releasers.conf
https://github.com/theforeman/foreman-packaging/tree/rpm/develop/comps

··· -- Later, Lukas @lzap Zapletal
2

Those are mine, you can delete them if you need to. I can always
re-create them.

··· On 16/05/17 13:32, Lukas Zapletal wrote: > Delete existing testing Copr repositories in @theforeman Copr group, > it's not shown in the UI who created them but I believe it was Dominic > or Eric: https://copr.fedorainfracloud.org/groups/g/theforeman/coprs/

–
Dominic Cleal
dominic@cleal.org

3

As part of this, katello will need to move as well, which i think brings
up some questions for discussion:

  1. Currently we rebuild candlepin srpms in our koji, sign them, and
    publish the srpm and rpm

Should we continue doing this? (in which case we likely need to setup a
copr build root for it).

  1. Currently for pulp, we pull in the pulp projects build from our koji
    into our tags, and publish in our repositories (signed by our key).

I think we likely should start pointing to their major release repos
instead. There is a chance that there could be some breaking change,
but i feel like their z-stream builds over the past year have been
increasingly stable and more often our users are requesting that we
update to the latest pulp zstream before we are able to.

  1. can we start building katello in the foreman 'build root' in copr?

-Justin

··· On 05/16/2017 08:32 AM, Lukas Zapletal wrote: > Hello, > > I would like to test Fedora Copr capabilities in serving us as the > main build system instead our Koji instance. With assistance from > Mirek Suchy, lead Copr developer, I would like to do proof-of-concept > of integrating Copr into our workflow as a secondary build system > together with Koji. The idea is to run both in paralel for some time > to excercise integration and stability, then doing one release from > Copr while keeping Koji up and running. After one successful release, > we will be able to deprovision the Koji instance and all its builders. > > Since Tito supports Copr, integration should be relatively easy. There > are some limitations we need to workaround. Comp files are being > uploaded via Copr user interface, there is no API/CLI yet but the > question is if we want to use comps at all. Might be better idea to > filter out packages during downloading it from Copr to our file > server. Second known limitation is that Copr uses its own embedded > signing procedure (everything is signed by default) while we will want > to sign releases by ourselves. Since we do this manually today, we can > simply sign packages again before uploading them to the target server. > > The plan: > > Delete existing testing Copr repositories in @theforeman Copr group, > it's not shown in the UI who created them but I believe it was Dominic > or Eric: https://copr.fedorainfracloud.org/groups/g/theforeman/coprs/ > > Create new project structure and set necessary build roots and SCL dependencies. > > Rebuild all nightly packages so they are all built in Copr > successfully for the first time via tito. This will be a PR in > foreman-packaging with all necessary configuration changes. We will > create Etherpad page to track the progress. > > Merge the tito configuration change and update Jenkins job to build > nightly on both Koji and Copr in parallel. > > This is the first stage, the second one will be doing a proper release > from Copr rather than Koji. Ideally, this would be 1.16 bit if we > don't hit this one, we can take our time. > > If you want to help, please let me know. The first step is to set > things up. Please keep Mirek in CC, if you have comments, questions or > concerns. > > Useful links: > > https://copr.fedorainfracloud.org/groups/g/theforeman/coprs/ > http://projects.theforeman.org/projects/foreman/wiki/Release_Process > https://github.com/theforeman/foreman-packaging/blob/rpm/develop/ > https://github.com/theforeman/foreman-packaging/blob/rpm/develop/rel-eng/tito.props > https://github.com/theforeman/foreman-packaging/blob/rpm/develop/rel-eng/releasers.conf > https://github.com/theforeman/foreman-packaging/tree/rpm/develop/comps > >
4

Hello Justin!

> 1. Currently we rebuild candlepin srpms in our koji, sign them, and publish
> the srpm and rpm
>
> Should we continue doing this? (in which case we likely need to setup a copr
> build root for it).

So you want to create Copr repositories in @theforeman Copr group for
that, Candlepin package would perhaps be in katello repository, or you
can create a separate one if you prefer to.

> 2. Currently for pulp, we pull in the pulp projects build from our koji into
> our tags, and publish in our repositories (signed by our key).

I believe Copr does not support "retagging" releases, Mirek can you
suggest the best workflow in this case? You will need to import
existing packages there, rebuild in the worst case.

Copr automatically signs all builds by default with it's own internal
key (which cannot be changed or uploaded - it's built in). This is
acceptable for nightly builds, but for regular releases we will need
to download all packages and sign them by hand anyway. That's the
workflow we plan for Foreman core.

> I think we likely should start pointing to their major release repos
> instead. There is a chance that there could be some breaking change, but i
> feel like their z-stream builds over the past year have been increasingly
> stable and more often our users are requesting that we update to the latest
> pulp zstream before we are able to.

The plan is to keep Koji running for longer period of time, we are
thinking one year. So Pulp 2 would be kept in Koji as well as older
Katello (Candlepin) and Foreman versions. So we will only change
workflow in future upcoming release and keep the old one for released
packages.

> 3. can we start building katello in the foreman 'build root' in copr?

Mirek, please confirm, but the suggested setup would be a katello copr
repository with foreman build root, so your packages would end up in
your own repository

LZ