RFC: Remove react tests from Katello jenkins PR job and rely on GH actions job

John_Mitsch · July 31, 2020, 2:07pm

Hello all,

With the addition of functional tests to Katello’s react tests, we have seen quite a few timeout errors in Jenkins. We have expanded the timeouts, but will still see the occasional timeout issue. There is a non-blocking GH actions workflow that has been running the same tests and we haven’t seen those timeout issues. The GH actions test also allows for quicker feedback (it doesn’t have to wait on rails setup) and runs faster in general.

I’m proposing we remove Katello’s react-ui tests from the jenkins PR job and rely on GH actions. The tests would be kept in the release/nightly job

We can do this in the following way:

Move react-ui tests out of parallel steps so they aren’t competing for resources. https://github.com/theforeman/foreman-infra/pull/1400
Let the tests run for a few days and make sure we don’t see timeout issues
Remove the react-ui test and foreman npm install step from the PR job in Jenkins, but keep in release/nightly
Make the GH action running react tests in Katello a blocking test

This will also reduce the amount of time that the jenkins job will take on Katello PR’s, which can now take up to an hour and a half

Any objections to this change?

ekohl · July 31, 2020, 2:27pm

I’m concerned of the state of our CI - it needs some serious attention. However, it appears nobody is really stepping up.

You can run things in parallel on multiple agents. Right now that doesn’t happen, but here’s an example of how Kafo is tested on multiple nodes:

github.com

theforeman/foreman-infra/blob/3e379fb5d1ef376f0a8e06171de7cd58b4004389/puppet/modules/jenkins_job_builder/files/theforeman.org/pipelines/test/kafo.groovy#L10


pipeline {
    agent none
    options {
        timeout(time: 1, unit: 'HOURS')
    }


    stages {
        stage('Test') {
            matrix {
                agent any
                axes {
                    axis {
                        name 'ruby'
                        values '2.4', '2.5', '2.6'
                    }
                    axis {
                        name 'PUPPET_VERSION'
                        values '5.0', '6.0'
                    }
                }

I think you can also use it on the individual stage level, but be aware that you need to clone the git repo too then.

Don’t forget branching. https://github.com/Katello/katello/blob/master/.github/workflows/react_tests.yml currently doesn’t work at all after branching which means cherry picks aren’t tested.

John_Mitsch · July 31, 2020, 7:29pm

Is this not giving CI some attention? I’m hoping Jenkins can focus only on Rails tests for PRs and we can explore parallelizing them there as well. That is the long term goal.

Thanks, I’ve updated to that, but still would like to move to PRs only using the GH actions job.

Good point, I can modify the action