Using robottelo to replace bats

Development RFCs
,
1

Hello all,

A while back I investigated using robottelo to replace our bats testing.

What is Robottelo

It is a pytest-based test suite used by Satellite QE to test Foreman + Katello. It has CLI, UI and API tests with the needed utilities to easily use them. Since it’s written in Python, it’s a black box test.

Why switch?

Writing tests in bash is easy if the commands are easy. The advanced testing requires complex set up of objects which makes it hard to write complex tests. We’re also duplicating effort with the QE team.

What’s been done

There’s a robottelo role in forklift. I also wrote a Proof of concept replacement of a single test:

My branch contains a bit more complex rewrites.

What’s next?

First of all I’d like to know if others agree this is a good direction to take.

If we all agree, we’d need to rewrite all existing tests (my rewrite is very much a proof of concept) and see if we can also run any existing Robottelo tests.

Once that’s done, it needs to be integrated in the CI pipeline. Currently I’m rewriting the systest jobs to use forklift. When that’s done, it’s as simple as adding the role to the right playbooks with the right variables.

6 Likes
2

We’re talking about replacing bats tests for testing upstream with robottelo, right? +100

Also, +1 to s/robottelo/robotello/g

3

+1 on overall agreement and general path. To re-iterate, I like the idea of:

Migrate bats to robottelo framework 1:1 to achieve parity on nightly
Then explore bringing more testing online from Robottelo
Look into re-factoring 1:1 transition into existing test cases

4

Updated the title that this is indeed what I was thinking about

This is my biggest annoyance with the project. I always make this typo in the name.

5

a big +1 from me!

6

I would like to convert the “hammer csv” testing to robotello, if that is
an option. Now it is vcr based but since server api can change, these
breakages are not picked up actively.

7

We do have hammer tests:

They could benefit from a rewrite to robottelo which might already have quite a bit of coverage. I’m not an expert on robottelo.

8

+1 !