Simplecov: How to avoid running SimpleCov if running an individual rspec test?

[vemv]

This is probably a FAQ, I bet there's a solution but one cannot easily find it.

If I run e.g. rspec spec/controllers/my_controller (a single file, not the whole suite), the SimpleCov report will just take into account the subset of code used for running that single test, resulting in an incomplete report.

SimpleCov should only run when the whole suite is run (and without having to pass an explicit env var for that).

How to achieve it?

Thanks - Victor

[josephks]

I have a script that I use when I want to run tests with coverage that sets an environment variable that gets read in my test setup. I only use Simplecov if that environment variable is set. If I just run rspec on an individual file then coverage doesn't run.

[vemv]

The thing I'd want (and that I'd find correct), is that whenever I simply run rspec, coverage is run/updated (I don't have to remember to enable it), without a special script or env var.

[josephks]

You have to look in your spec_helper.rb or wherever you start Simplecov and modify it.

[vemv]

Sure, what's the specific code to insert? Surely someone has had this issue before...

[josephks]

It depends on what your existing code looks like. I wrap my whole SimpleCov setup in if ENV['RUN_COVERAGE'] which includes requiring simplecov, the SimpleCov config setup, and calling SimpleCov.start.

[vemv]

That approach doesn't satisfy the requirements I explained.

I guess that one has to programatically fetch Rspec's run options (which look similar to include {:ids=>{"./spec/features/foo_spec.rb"=>["1:2:3:17:1"]}}) and decide whether to launch SimpleCov depending on the options passed.

I don't know how to do that though...

[vemv]

Alrighty, I figured out a way. Dunno if this approach is too dirty like for putting it in the README as a FAQ?

if (RSpec.configuration.instance_variable_get :@files_or_directories_to_run) == ['spec']
  require 'simplecov'
  SimpleCov.start 'rails'
end

[josephks]

I prefer to explicitly control whether I'm running coverage rather than try to have it magically guess what I want. In my experience trying to add magic usually leads to confusion in the long run.

[vemv]

Well, it's a matter of perception. You perceive my feature as magic, but I perceive current behavior as totally undesirable - I don't want to ruin my coverage document (which maybe took 30 mins to generate) just because I ran a single test (which devs do all the time)

[threedaymonk]

@josephks If you're using RSpec, it can already tell you if you're running one file: the generated boilerplate spec_helper uses this to change the formatter to doc when running a single file. We've been using this for a while, and it's working well:

RSpec.configure do |config|
  if config.files_to_run.one?
    config.default_formatter = 'doc'
  else
    SimpleCov.start 'rails'
  end
end

[jaynetics]

@threedaymonk @vemv Sometimes you might run two or three specs, but something along these lines should cover that:

SimpleCov.start 'rails' if ARGV.grep(/spec\.rb/).empty?

[ConnorWGarvey]

Thanks, @janosch-x! That satisfies a requirement in SimpleCov's documentation, that it is the first thing to start, even before RSpec.

[bf4]

@ConnorWGarvey really simplecov needs to start before any app code is loaded.

Most people solve this problem by only running simplecov when ENV['COVERAGE'] == 'true' or some other toggle, like ENV['CI'] == 'true'.

If SimpleCov is configured to always run, then @threedaymonk may solve your problem.

# spec_helper.rb
require 'simplecov'
require 'rspec'
if RSpec.configuration.files_to_run > 1
    SimpleCov.start 'rails'
end

[ConorSheehan1]

@janosch-x what about rspec spec/some_folder?
It'll still run a portion of the test suite and it doesn't contain spec\.rb.

You could possibly handle that case using:

if ARGV.grep(/spec.\w+/).empty?
# rspec                     # no match (run full test suite)
# rspec spec                # no match (run full test suite)
# rspec spec/some_folder    # match    (run partial test suite)
# rspec spec/some_spec.rb   # match    (run single test)

SimpleCov.use_merging(true) might also solve the problem by merging results rather than settings coverage to 0 for tests that didn't run.