Simplecov: How to cover gem version.rb?

[steffenschmidt]

Hi,

when using simplecov I get 100 % code coverage except for the version.rb which is located in lib/<namespace>/<gemname>/version.rb:

module Namespace
  module Core
    VERSION = "X.X.X"
  end
end

My rspec test runs successful with the following but simplecov does not consider that:

module Namespace
  describe Core do
    it "has correct version number" do
      expect(Namespace::Core::VERSION).to eq "X.X.X"
    end
  end
end

How can I test that so that I also get a code coverage of 100 % for the complete gem? Any idea?

Thanks in advance
Steffen

Versions:
ruby 2.3.3
rails 5.0.1 (api only)
rspec-rails 3.5.2
simplecov 0.12.0

[PragTob]

Hi there!

Umm gotta take a look at that. I thought this works, e.g. you wouldn't even have to test it but it's just executed (the constant) and hence was covered. Need to check it out on one of my projects.

Worst case you could surround it with # :nocov: (as shown in the README).

is your lib open source to take a look?

[colszowka]

It should indeed be covered correctly just as expected. @steffenschmidt Could you try to upgrade to the latest simplecov 0.13 to see if that possibly works? I wouldn't expect so though. Also, possibly your 'version' file is required before simplecov is required and started? That would put it out of the view of the STDLIB Coverage library simplecov uses.

[steffenschmidt]

I updated to simplecov 0.13 and as expected by @colszowka it was not covered. The gem is currently not open source so I could not get access to you. But I will extract the relevant things this afternoon and make a public repo so you can take a look into it.

[bf4]

The entirety of version.rb will be evaluated when the file is loaded. If anything requires the file, it'll be fully covered.

I made a file foo.rb containing

module Namespace
  module Core
    VERSION = "X.X.X"
  end
end

and in irb ran

require 'coverage'
Coverage.start
load './foo.rb'
Coverage.result

and got {"foo.rb"=>[1, 1, 1, nil, nil]} which means the first three lines were hit once.

[PragTob]

@steffenschmidt any luck with that extraction? As long as it's required it should all be coverd as ruby just executes it

[steffenschmidt]

Here ist a link to the extraction:
https://github.com/steffenschmidt/coverage-extraction-gem

Let me know if I can help any further.

[steffenschmidt]

Just for my understanding as the solution of @bf4 works for me: Why do I have to load the version.rb separate while every other file in the lib directory is considered?

[PragTob]

@steffenschmidt when the file isn't loaded then it is not covered by the tests - there is no run time just parse code in there :) Leaving this open for now so one day when I got time could check out the test repo

[wjordan]

This occurs when bundle exec is used to run tests, where a gemspec declaration exists in the project's Gemfile and a require 'my_gem/version' statement is in the project's my_gem.gemspec file. Since the my_gem/version file is already required before the test helper is run, simplecov never tracks it.

This is the default setup for a new gem project as created by bundle gem, so it probably affects a large number of gem projects (though most probably don't care if their gem's VERSION constant isn't covered).

I can't think of any good workaround other than removing the require from the gemspec, and just duplicating/inlining the version directly.

[bf4]

Just surround the VERSION with # :nocov: comments or exclude it in your config. 100% and 0% or the version.rb are both kind of meaningless. Better to ignore it.

[PragTob]

Agreed with @bf4 - that's the best solution we'll ever get I think :)

[chase439]

I just had this same issue, but then realized my mistake. I was using SimpleCov.start 'rails'.... Once I removed 'rails' it got coverage for the version file.

require 'simplecov'
Coverage.start
require 'gem_name'

[jcouball]

I ran into this problem where SimpleCov didn't report coverage on version.rb. I found that the problem was that if the gemspec was loaded prior to me calling SimpleCov.start (as I do from my spec_helper.rb), then version.rb is not included in the coverage report.

This had two negative consequences:

• If I didn't have 100% coverage, the percent of coverage was reported inconsistently (by a small amount) depending on how I ran my spec suite. Coverage was slightly higher if version.rb was included in the coverage report. Running rspec omits version.rb while running rake rspec does not.
• When using SonarQube, SonarQube scans the code and matches up the coverage reported by SimpleCov to the files it scans. When version.rb is not included in the SimpleCov coverage report, it is impossible to get 100% coverage in SonarQube.

Reasonable people could say that the small discrepancy does not matter. If you care, here is why this happens and how to fix it.

SimpleCov only reports on files which were required after the call to SimpleCov.start. Most gemspecs include the line:

require_relative "lib/my_gem/version"

This is included in the gemspec that is generated when you run bundle gem my_gem.

This is the reason why SimpleCov does not show coverage on the version.rb file: the gemspec is loaded (say because you ran your test suite with rspec or bundle exec rspec) before SimpleCov.start is called in the spec_help.rb file.

Curiously, running rake spec does not load the gemspec first. This means that version.rb IS included in the coverage report when running rake spec.

The only way I have found around this is to get the version in the gemspec without requiring 'lib/my_gem/version' in the following way:

load 'lib/my_gem/version.rb'

This works because the file will be required by your test suite in the normal way after SimpleCov.start is called.

However, doing this results in the following warning when you run your test suite:

lib/my_gem/version.rb:4: warning: already initialized constant MyGem::VERSION

In order to get around this, I modified my gemspec as follows:

# Get the GEMFILE_VERSION without requiring my_gem/version
load 'lib/my_gem/version.rb'
GEMFILE_VERSION = MyGem::VERSION
MyGem.send(:remove_const, :VERSION)

Gem::Specification.new do |spec|
   # ...
   spec.version = GEMFILE_VERSION
   # ...
end

After this, lib/my_gem/version.rb is included in the coverage report and rspec and rake rspec both show identical coverage.