Rspec, Cucumber: best speed database clean strateg

Question

I would like to increase the speed of my tests.

1. Should I use use_transactional_fixtures or go with the database_cleaner gem?
2. Which database_cleaner strategy is the best? I noticed that after migration from :truncation to :transaction my more than 800 examples run about 4 times faster!
3. Should I turn off use_transactional_fixtures when I use database_cleaner :transaction?
4. Is it true that the best strategy for rack_test is :transaction?
5. What is the best practices for changing strategy on the fly from :transaction to :truncation when using selenium or akephalos?

P.S. Mysql, Rails 3, Rspec2, Cucumber

P.P.S. I know about spork and parallel_test and using them. But they are offtopic. For example, Spork save about 15-20 sec on whole suite run, but changing from :transaction to :truncation dramatically increase running time from 3.5 to 13.5 minutes (10 minutes difference).

Answer 1

Have you used Spork ? It greatly enhances speed.

Answer 2

Using transactional fixtures will be faster since the DBMS doesn't commit changes (and therefore no heavy IO occurs resetting the database between tests) but as you know won't always work.

We have had some success using SQLite in-memory databases in the test environment so tests run super fast while leaving transactional fixtures off. This option is also available for MySQL (use :options to set "ENGINE=MEMORY") but I've never done it personally and if you search you'll find a few threads about caveats involved. Might be worth a look. Depending on your testing methodology it may not be acceptable to use a different DB engine though.

I suggest you enable transactional fixtures and use the DatabaseCleaner gem to selectively disable transactional fixtures per example group. I can't say that I've tried this but since you didn't have any answers I figured anything might potentially help you out.

before(:all) do
  DatabaseCleaner.strategy = :transaction
  DatabaseCleaner.clean_with(:truncation)
end

before(:each) do
  DatabaseCleaner.start
end

after(:each) do
  DatabaseCleaner.clean
end

If it were me I'd factor this out into a helper and call it as a one-line macro from each example group that needs transactional fixtures turned off.

Seems like there really should be a better way, though.... best of luck.

Answer 3

1., 2. & 4., You should use transactions (either with use_transactional_fixtures or transactions support from the database_cleaner gem) if you are using capybara's default engine, rack_test. As you noted, using transactions are substantially faster than using a truncation strategy. However, when database writes can go through different threads (as with selenium) transactions won't work. So you'll need to use truncation (or force everything to go through one db thread--another option).

3. Yes, you should turn off use_transactional_fixtures when using the database_cleaner gem since the gem natively support transactions. If you only need transactions then just use_transactional_fixtures and never load the database_cleaner gem.

5. The following code will switch between :transaction and :truncation on the fly. (Tested this with rspec, capybara, rails3.)

Features This should give you the best of both worlds. The speed of rack_test when you don't need to test javascript stuff and the flexibility of selenium when you do.

Also this code takes care of repopulating seed data in cases where it is needed (this method assumes you use seeds.rb to load your seed data--as is the current convention).

Add the following code to spec_helper.

config.use_transactional_fixtures = false
RSpec.configure do |config|
  config.before(:suite) do
    require "#{Rails.root}/db/seeds.rb"
  end

  config.before :each do
    if Capybara.current_driver == :rack_test
      DatabaseCleaner.strategy = :transaction
    else
      DatabaseCleaner.strategy = :truncation
    end
    DatabaseCleaner.start
  end
  config.after(:each) do
    if Capybara.current_driver == :rack_test
      DatabaseCleaner.clean
    else
      DatabaseCleaner.clean
      load "#{Rails.root}/db/seeds.rb"
    end
  end
end

Thanks Jo Liss for pointing the way.

PS: How to switch drivers on the fly

The above solution assumes you already know how to switch drivers on the fly. In case some who come here don't, here's how:

As above let's assume that you normally will use the default capybara driver rack_test, but need to use selenium to test some Ajaxy stuff. When you want to use the selenium driver use :js => true or @javascript for Rspec or cucumber respectively. For example:

Rspec example:

describe "something Ajaxy", :js => true do

Cucumber example:

@javascript
Scenario: do something Ajaxy

Answer 4

RSpec.configure do |config|

  config.before(:suite) do
    DatabaseCleaner.clean_with(:truncation)
  end

  config.before(:each) do
    DatabaseCleaner.strategy = :transaction
  end

  config.before(:each, :js => true) do
    DatabaseCleaner.strategy = :truncation
  end

  config.before(:each) do
    DatabaseCleaner.start
  end

  config.after(:each) do
    DatabaseCleaner.clean
  end

end

This is from Avdi Grimm's post about database cleaner and Rspec. Step-by-step analysis of the code is in the article.