Calling Block multiple times in Cucumber Around Ho

Question

I'm trying to run a scenario several (30) times in order to get a nice statistical sample. However the block is only executing once; each subsequent time results in the scenario being called and not executing (although it says that the scenario did successfully complete with a time of around 5 ms).

Around('@mass_benchmark') do |scenario, block|
  $seconds_taken = "SECONDS TAKEN NOT SET"
  @time_array = []
  30.times do
    before_hook(scenario)
    block.call
    after_hook(scenario)
    @time_array << $seconds_taken
  end
  write_time_array_to_file(@time_array, scenario_name)
end

The tag @mass_benchmark executes this block, as opposed to ~@mass_benchmark, which just executes the scenario normally. The methods before_hook and after_hook replicate the Before ('~@mass_benchmark') and After ('~@mass_benchmark') hooks (which actually just call the same method).

The variable $seconds_taken is set around the specific area for which I am timing. I am not timing the whole test there, just a critical portion of it; the remainder of the test is getting to that point, etc, which is not to be part of the timed portion, so I cannot just move the timing portion outside of this.

The issue may be with something I'm doing in those methods, but as far as I can tell, everything works normally (as indicated by well-placed puts statements). Any ideas are appreciated!

Answer 1

Currently Cucumber does not seem to support calling the block twice in an around hook. This can be demonstrated by the following feature file:

Feature: This scenario will print a line

  Scenario: Print a line
    When I print a line

And step definitions:

Around do |scenario, block|
  Kernel.puts "START AROUND, status=#{scenario.status}"
  block.call
  Kernel.puts "BETWEEN CALLS, status=#{scenario.status}"
  block.call
  Kernel.puts "END AROUND, status=#{scenario.status}"
end

When /^I print a line$/ do
  Kernel.puts "IN THE STEP DEFINITION"
end

When this is executed, Cucumber will print:

  Scenario: Print line  # features/test1.feature:3
START AROUND, status=skipped
IN THE STEP DEFINITION
    When I print a line # features/test.rb:9
BETWEEN CALLS, status=passed
    When I print a line # features/test.rb:9
END AROUND, status=passed

Evidently since the status of the scenario is already "passed", Cucumber does not re-execute it, though the output formatter receives the steps. I have not found any way to "reset" the status in the scenario API to get them to be re-run.

There are other problems with around hooks as well, for example you cannot set variables to the World in around hooks (like you can in before hooks). See also Cucumber issues 52 and 116 for more gory details.

Answer 2

One possibility might be to keep the passed-in block as it is, and call the ".call" method on a duplicate? Something like (untested):

Around do |scenario, block|
  30.times do
    duplicate = block.dup
    before_hook(scenario)
    duplicate.call
    after_hook(scenario)
  end
end

Just make sure not to use ".clone" on the block, since clone will create an object with the same Id, resulting in every change made to the duplicate also affecting the original.