SimpleCov with multiple apps - or in short, how do

I'm trying to setup SimpleCov to generate reports for 3 applications that share most of their code(models, controllers) from a local gem but the specs for the code that each app uses are inside each ./spec and not on the gem itself.

For a clearer example. When i run bundle exec rspec spec inside app_1 that uses the shared models from the local gem I want to get(accurate) reports for all the specs that this app_1 has inside ./spec.

The local gem also has some models that belong exclusively for app_2, inside a namespace, so i want to skip the report for those files when i run the test suite inside app_1.

I'm trying to achieve this with something like the following code in app_1/spec/spec_helper.

# This couple of lines are needed to generate report for the models, etc. inside the local gem.
SimpleCov.adapters.delete(:root_filter)
SimpleCov.filters.clear

SimpleCov.adapters.define 'my_filter' do
  root = SimpleCov.root.split("/")
  root.pop
  add_filter do |src|
    !(src.filename =~ /^#{root.join("/")}/)
  end

  add_filter "/app_2_namespace/"
end

if ENV["COVERAGE"] == "true"
  SimpleCov.start 'rails'
end

This works, until some questions begin to arise.

Why i get a 85% coverage for a model that's inside the gem but the spec is inside app_2(I'm running the spec inside app_1).

The first time that was a problem, was when i tried to improve that model so i clicked on the report for it and saw which lines were uncovered and i tried to fix them writing tests for them on app_2/spec/namespace/my_model_spec.rb.

But that didn't make any difference, i tried a more aggressive test and i erased all the content on the spec file but somehow i still was getting the 85% of coverage, so the my_model_spec.rb is not related to the coverage results of my_model.rb. Kind of unexpected.

But since this file was on app_2 i decided to add a filter on the SimpleCov.start block on app_1 spec_helper, like:

add_filter "/app_2_name_space/"

I moved then to the app_2 folder and started setting up SimpleCov and see what results i would get here. And they turned out weirder.

For the same model i got 100% coverage, i did the same test of empty'ing the my_model_spec.rb file and still got the 100%. So this really f**ed up, or i don't understand something.

How does this work?(with the Ruby 1.9 Coverage module you say, well when i run locally the example on the official documentation i get different results, so i think there's a bug or outdated documentation there)

ruby-doc: {"foo.rb"=>[1, 1, 10, nil, nil, 1, 1, nil, 0, nil]} 
locally:  {"foo.rb"=>[1, 1, 10, nil, nil, 1, 0, nil, 1, nil]}

I hope the reports don't show positive results for lines that get evaluated somewhere on the app code, no matter where.

I think the expected behavior is that the results for a model for example are related to it's spec, same thing for controllers, etc.

Is this the case? If so, why am i getting this strange results.

Or do you think the structure of my apps could be messing up with SimpleCov and Coverage?

Thank you for taking the time to read this, if you need more detailed info, just ask.

Regarding your confusion with the model being 100% covered, as I'm not sure that I understand correctly: There's no way for Coverage (and therefore SimpleCov) to know whether your code has been executed from a spec or "somewhere else". Say I have a method "foo" and a method "bar" that calls foo. If I invoke bar in my specs, of course foo will also be shown as covered.

As to your general problem: I think it should be possible to have the coverage reported. Just because the source code is at some different point than your project root should not lead to the loss of coverage reporting.

Two things in your base config: Removing the base adapter (line 2) is unneccessary as adapters basically are glorified config chunks and at this point you'll have it already executed (since it gets invoked when Simplecov is loaded). Resetting the filters should be sufficient.

Also, the custom adapter you define is not used. Please refer to the README as to how to properly set up adapters, but I think you'd be fine with simply using this in the SimpleCov config block when you start the coverage run for now:

SimpleCov.start 'rails' do
  your_custom_config
end

What you'll probably want though is a merged coverage report for all your apps. For this, you'll have to define a command_name for each of your spec suites first, inside your config block, like this: command_name 'App1 Specs'.

You'll also have to define a central coverage_path, which will store away your coverage reports across your app suites. Say you have ~/projects/my_project/app[1-3], then putting that into my_project/coverage might make sense. This will lead to your different test suite results getting merged into one single report, just like when using SimpleCov with Cucumber & RSpec for example. Merging has a default timeout of ~10 minutes, so you might need to set this to a higher value using merge_timeout 3600 in your config (those are seconds). For the specifics of these configuration options please again check out the README and the SimpleCov::Configuration documentation. These things are outlined there in fair detail.

So, to sum it up, each of your apps should look somewhat like this:

require 'SimpleCov'
SimpleCov.start 'rails' do
  reset_filters!
  command_name 'App1 Spec'
  coverage_path File.dirname(__FILE__) + '../../coverage' # Assuming this is in my_project/app1/spec/spec_helper.rb
  merge_timeout 3600
end

Next thing you might want to add filters to reject all non-project gems by path, and you should be up & running.