Katello Scenario tests and annotations merged

Justin_Sherrill · May 7, 2017, 1:41am

Hi All,

I have just merged https://github.com/Katello/katello/pull/6716 which
adds some end to end testing for a small set of scenarios (that will
likely be expanded over time). It also adds annotations to these
requests and the ability to generated documentation around them. The
end goal being to a) have better testing for full scenarios and b) be
able to look up all the interactions between katello and pulp and
candlepin easily. (https://github.com/theforeman/theforeman.org/pull/845
is the first iteration of that)

Most of the time you should notice no difference. However sometimes if
you add or change an api call to pulp or candlepin you may start seeing
test failures (especially as we add requests). Once you re-record the
annotations (see README below), you may see that there are missing
annotations which need to be added to your PR.

Feel free to read

github.com

Katello/katello/blob/master/test/scenarios/annotations/README.md

# Annotated Requests

## Purpose
To create documentation for our interactions with pulp and candlepin across different workflows.  This allows the pulp and candlepin team to better understand how Katello is utilizing their respective service.

## Components:

1. Scenario test file: ./test/scenarios/scenario_test.rb 
  * Currently houses all scenario tests.  Uses vcr to record reqeusts.
2. Scenario VCR cassettes: ./test/fixtures/vcr_cassettes/scenarios/*.yml
3. Scenario annotations: ./test/scenarios/annotations/*.yaml
  * This provides options for matching a particular vcr request as well as a description of the request. 
4. Scenario Rake tasks: ./lib/katello/tasks/annotate_scenarios.rake. Provides two rake tasks
  * rake katello:find_unannotated_requests
    * finds requests that are not annotated and suggests annotation entries
  * rake katello:create_annotated_output
    * creates the annotated request documentation
    
## Adding a new workflow

This file has been truncated. show original

for more information and let me know if you have questions.

-Justin

iNecas · May 10, 2017, 12:47pm

Justin Sherrill <jsherril@redhat.com> writes:

> Hi All,
>
> I have just merged https://github.com/Katello/katello/pull/6716 which
> adds some end to end testing for a small set of scenarios (that will
> likely be expanded over time). It also adds annotations to these
> requests and the ability to generated documentation around them. The
> end goal being to a) have better testing for full scenarios and b) be
> able to look up all the interactions between katello and pulp and
> candlepin easily. (https://github.com/theforeman/theforeman.org/pull/845
> is the first iteration of that)
>
> Most of the time you should notice no difference. However sometimes if
> you add or change an api call to pulp or candlepin you may start seeing
> test failures (especially as we add requests). Once you re-record the
> annotations (see README below), you may see that there are missing
> annotations which need to be added to your PR.
>
> Feel free to read
> https://github.com/Katello/katello/blob/master/test/scenarios/annotations/README.md
> for more information and let me know if you have questions.

Interesting,

BTW. for those of you lazy to actually run the rake, but still
interested into how the output looks like, have a look at

https://theforeman.org/plugins/katello/nightly/annotations/index.html

– Ivan

···

> > -Justin > > -- > You received this message because you are subscribed to the Google Groups "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout.