junit: impact of forkMode=“once” on test correctne

Question

I'd like to reduce the time which our build (using ant) takes for running the tests. Currently I am using the default forkMode, which forks a new vm on each test class (perTest).

I am thinking about to switch to forkMode="once" but I am unsure if this will couple the tests somehow and maybe give me false positive and/or false negatives results after running my tests.

Questions:

1. Will each test case get a new ClassLoader so that all static references from previous runs are not accessible/visible anymore?

2. Are there other things which lead to test dependency/coupling of test methods which may change the behavior (beside native library loading which I am not using)

3. What about garbage collection/finalization, are they run after each test? (I don't rely on them, but I just want to get a complete picture)

UPDATE

According to the current answers it seems that junit is always sharing a single classloader between all test cases per vm/fork when using forkMode. (so forkMode="once" indeed means there's one classloader for all tests)

This has many advantages (faster tests and may cause tests to fail because of static coupling) but also some disadvantages (static coupling which will only work if a shared classloader is used -> false positive)

Answer 1

1. The test runner will effectively make a single Suite of all of your tests and run them - so that only one classloader is involved.
2. Yes that means that static data will be shared between tests, which can occasionally be handy, but will force you to cut down on the static coupling between clauses, which is a good thing.
3. There isn't generally any explicit GC, but you can do your own.

Generally running all your tests in one VM is a good thing. It forces you to look at static coupling and is a lot quicker. Crucially, it's also the way that your IDE will be running them, and that really is the way that tests should be run - as close as possible to as often as you compile.

Answer 2

Note that the default mode forks a new VM for each test case (i.e. class) not for each test (i.e. method). In the application I am currently testing there are problems that arise when I reuse a VM for more than one test: objects and state are left over from earlier tests and stop later ones from working. This may not be a problem if your application is well structured and your tests are strictly self-contained. I doubt garbage collection runs automatically after each test: it is notoriously hard to ensure that it is called at any given time in any case.

Answer 3

Looking at Stefan's blog entry about this I would venture to guess:

1. you will only get a single class loader for forkMode="once"
2. you won't have access to the Ant environment anymore
3. the GC will be performed inside the spawned GC (if forceMode="once") and this means it's NOT after each test is run.