Play! framework: customize which tests are run

Question

I have a Play! 2 for Scala application, and I am using Specs2 for tests. I can run all tests with the test command, or a particular specification with test-only MyParticularSpec.

What I would like to do is mark some particular specifications, or even single methods inside a specification, in order to do things like:

• running all tests that are not integration (that is, that do not access external resources)
• running all tests that do not access external resources in write mode (but still running the reading tests)
• running all tests but a given one

and so on.

I guess something like that should be doable, perhaps by adding some annotations, but I am not sure how to go for it.

Does there exist a mechanism to selectively run some tests and not other ones?

EDIT I have answered myself when using test-only. Still the command line option does not work for the test task. Following the sbt guide I have tried to create an additional sbt configuration, like

object ApplicationBuild extends Build {
  // more settings
  lazy val UnitTest = config("unit") extend(Test)
  lazy val specs = "org.scala-tools.testing" %% "specs" % "1.6.9" % "unit"

  val main = PlayProject(appName, appVersion, appDependencies, mainLang = SCALA)
    .configs(UnitTest)
    .settings(inConfig(UnitTest)(Defaults.testTasks) : _*)
    .settings(
      testOptions in UnitTest += Tests.Argument("exclude integration"),
      libraryDependencies += specs
    )
}

This works when I pass arguments without options, for instance when I put Test.Argument("plan"). But I was not able to find how to pass a more complex argument. I have tried

Tests.Argument("exclude integration")
Tests.Argument("exclude=integration")
Tests.Argument("-exclude integration")
Tests.Argument("-exclude=integration")
Tests.Argument("exclude", "integration")
Tests.Argument("exclude \"integration\"")

and probably more. Still not any clue what is the correct syntax.

Does anyone know how to pass arguments with options to specs2 from sbt?

Answer 1

I'm using Play2.2, and there are 2 ways to do this depending on whether or not you are in the play console.

1. From the console type: test-only full.namespace.to.TestSpec
2. From the terminal type: test-only "full.namespace.to.TestSpec"

Answer 2

If you want to pass several arguments you can add several strings to one Test.Argument

testOptions in Test += Tests.Argument("include", "unit")

There are examples of this in the specs2 User Guide here and in the Play documentation there.

Answer 3

I came across this question while trying to figure out how to do something similar for ScalaTest with Play. SBT has detailed documentation on how to configure additional test configurations but these could use a bit of tweaking for Play.

Apart from the subtly different Project configuration I found that I wanted to crib a bunch of the test settings from PlaySettings. The following is running and generating an Intellij project with integration test sources in the "/it" directory. I may still be missing reporting and lifecycle hooks,

object BuildSettings {
  def testSettings = {
    // required for ScalaTest. See http://stackoverflow.com/questions/10362388/using-scalatest-in-a-playframework-project
    testOptions in Test := Nil
  }

  def itSettings = {
    // variously cribbed from https://github.com/playframework/Play20/blob/master/framework/src/sbt-plugin/src/main/scala/PlaySettings.scala
    sourceDirectory in IntegrationTest <<= baseDirectory / "it"
    scalaSource in Test <<= baseDirectory / "it"
    libraryDependencies += "play" %% "play-test" % play.core.PlayVersion.current % "it"
  }
}

object ApplicationBuild extends Build {
  val main = play.Project(
    appName,
    appVersion,
    Dependencies.dependencies)
    .configs( IntegrationTest )
    .settings(Dependencies.resolutionRepos)
    .settings(BuildSettings.testSettings)
    .settings(Defaults.itSettings : _*)
    .settings(BuildSettings.itSettings)
}

Answer 4

You can also use without any change to your build

test-only * -- exclude integration

Tested in Play 2.1-RC3

Answer 5

First, following the specs2 guide one must add tags to the specifications, like this

class MySpec extends Specification with Tags {
  "My spec" should {
    "exclude this test" in {
      true
    } tag ("foo")

    "include this one" in {
      true
    }
  }
}

The command line arguments to include are documented here

Then one can selectively include or exclude test with

test-only MySpec -- exclude foo
test-only MySpec -- include foo