Testing PRs using RPM packages

Development RFCs
1

The goal of this RFC is to easily test changes in PRs using RPM packages. There are multiple levels of usability of this. The initial level is to get the infra there.

The plan

Currently we have testing capabilities in forklift using Ansible playbooks (pipelines). These do the setup of host(s) (including repositories), run the installer and run tests. These are run using:

ansible-playbook pipelines/pipeline_katello_nightly.yml -e forklift_state=up

Next up we also Koji which can do scratch builds. The results from these scratch builds can be downloaded and we already have an Ansible role in Forklift to download these results into a local yum repository.

The first step would be to include the koji role into every pipeline so you can change the command line to:

ansible-playbook pipelines/pipeline_katello_nightly.yml -e forklift_state=up -e koji_task_ids=xyz

Now we need Koji build IDs. Those with permission to Koji can create scratch builds. Another way is when you submit a RPM packaging PR.

My proposal is to create a third way. We have obal as a tool to do packaging for us. It knows how to build a scratch build using a local git repository. This can be combined into Jenkins jobs on git repositories (foreman, foreman-installer, smart-proxy, katello, katello-installer) as a manual [test rpm] trigger. That would attach a build ID to the PR which can be fed into the pipeline.

To neatly finish it up it’s also good to implement yum repository priorities in our Ansible roles. Otherwise the next nightly would have a higher version number than your koji build.

Lastly we need a place to document this workflow.

Limitations

Koji builds can’t change dependencies so if you need new dependencies, this won’t work. We can investigate using copr but that’s out of scope for the initial phase.

Note that you can combine foreman, foreman-proxy and installer changes into a pipeline since those don’t (build) depend on each other. Making a Plugin change that requires a Foreman PR can work, but not always so consider it out of scope (Ruby can work since it’s often a runtime dependency, webpack certainly wouldn’t since it’s a build time dependency).

5 Likes
2

I think this would be really nice addition. It would help to test, how the change affect production setups, which would be handy (not only) during stabilization period. Also it could make QE life easier. I suppose this also helps testing changes of assets compiled through asset pipeline in production. So big :+1: from me.

3

Thanks for bringing this up. This will be a huge win for testability and overall stability of the upstream, especially with ability to make it easier for QE folks and generic users community to provide feedback.

I think it can imagine also in future, with containerization efforts, to have similar workflow for containers, being able to build the image for a specific PR (out of scope for this discussion, just wanted to mention the connection)

If the resources were constrained to build the rpms on every PR change, I would be ok with ability to trigger the job via [test rpm].

4

I agree, this should probably not be the default.

1 Like
5

I’m also a fan, I think this would be a great way to use those tools all in concert.

6

I will consider this accepted and hope to have time to implement it soon.

7

@evgeni just posted Packit for Foreman - get production RPMs from PRs, which I think largely resolves some fundamental issues. This still mentions some forklift integration, so I’m not sure if we can call this resolved already but after 4 years there’s finally some major progress.

8

Yeah, I do plan to add easier consumption of those builds in Forklift!

9

And it’s now documented which means this can now be considered done.

1 Like