Fixed: Ability to efficiently run single test cases (not just a file)

Development
1

Up until now, running single tests has been pretty cumbersome with one having to pass in some regex of the test name. This would be fairly slow to run start-up and hard to remember the exact command, plus having to fish out the name of the test.

Rails 5 brought in the ability to Minitest for running single tests via the line number, however this has been broken due to a bug in minitest-spec-rails which can be seen here.

I fixed this in foreman with an in PR #5856. This fix allows us to be able to run single tests like so:
$ bundle exe rails test test/integration/about_test.rb:14
and files as:
$ bundle exe rails test test/integration/about_test.rb

There still lies the issue of running single file/tests of a plugiin, I am working on a solution to this and will post my findings :slight_smile:

6 Likes
2

Thanks so much. I wish I could do the same in a Foreman Plugin.

3

We have actually spent quite some time thinking of a solution. Rails 5 added a new way of running tests and I believe that’s not super compatible with our rake task stuff we do for plugins.
But we’ll keep looking into that.

4

No Problem.

I have something in the works for plugins as Timo mentioned. For now if you want to be able to run file you can change your rake task in the plugin to something like:

namespace :test do
  desc 'Test ForemanSpectrumPlus'
  Rake::TestTask.new(:foreman_spectrum_plus) do |t|
    ARGV.each { |a| task a.to_sym { puts 'jo jo'} }

    file_name = ARGV[1]
    root_dir = File.join(File.dirname(__FILE__), '../..')

    t.libs = ['test', File.join(root_dir, 'test')]

    if file_name
      t.pattern = "#{root_dir}#{file_name}"
    else
      t.pattern = "#{root_dir}/test/**/*_test.rb"
    end

    t.verbose = false
    t.warning = false
  end
end

Then you can do something like:
$ bundle exec rails test:my_plugin /test/unit/my_test.rb

NOTE: This is in its infancy so it might not work 100%

5

Ahh :smiley: :smiley: forget the puts 'jo jo'. This was just some debugging left over. Shame I can only edit my post once.

6

It’s time limited, not count limited. Edits are possible for 5 minutes after the post is made, so typos can be caught. I think people get the idea though :slight_smile:

7

Figured it out with together with @x9c4.

So in the plugin you want to use this for, you need to change the require for the test_plugin_helper to a require_relative with the correct relative path, otherwise the rails test command will fail saying it can’t find that file.

Once you have this you can run rails test <filename_with_full_path>:<line_number>. NOTE: You need to use the full file path, otherwise rails won’t find the specific tests, and just run the whole test suite. For example: bundle exec rails test ~/dev/plugins/foreman_plugin/test/models/your_model.rb:10. Omiting the line number runs all tests for the given file.

Im really not sure why yet that the full path needs to be provided and not a simple../your-plugin/......, but it works for now !!

See: https://github.com/dm-drogeriemarkt/foreman_dlm/pull/44 for the relevant changes needed.

3 Likes
8

Care to update Foreman :: Contribute with the information?

9

Sure!

1 Like
10

Thank you so much for tackling this!

I can confirm this also works for discovery now:

be rake test:discovery TEST=~/work/foreman_discovery/test/functional/discovered_hosts_controller_test.rb
11

@lzap: I believe the trick is not to use rake, but rails as the latter is a lot faster. The rake approach has worked for a long time.

12

@leewaa: As you know, I usually hate spending time with optimising the developer environment but rather fix the user experience, but I though: How hard can this be.

I came up with a gist that monkey-patches the rails test runner to do what you want to achive.

You can now run

bundle exec rails test --engine foreman_bootdisk ../foreman_bootdisk/test/unit/example_test.rb:20

And it will run a single test from foreman_bootdisk.
Care to take a look? What do you think?

13

Well, this enables running exactly one test case instead of only tests of a given file (which is all that is possible with my fix). So thumbs up from my side, thanks!