Running JUnit Test in parallel on Suite Level?

Question

I have a bunch of tests that are organized in JUnit test suites. These tests are greatly utilizing selenium to test a web application. So, naturaly for selenium, the runtime of these tests is quite long. Since the test classes in the suites can not run parallel due some overlaps in the test database, i would like to run the suites parallel.

The JUnit ParallelComputer can only execute tests on class or method level in parallel, are there any standard ways for JUnit to do that with suites?

If i just pass suite classes to the junit runner and configure the computer to parallelize on class level, it picks the test classes itself, not the suites.

br Frank

Answer 1

Here is some code that worked for me. I did not write this. If you use @RunWith(ConcurrentSuite.class) instead of @RunWith(Suite.class) it should work. There is an annotation that is also needed which is found below.

package utilities.runners;

import org.junit.internal.builders.AllDefaultPossibilitiesBuilder;
import org.junit.runner.Runner;
import org.junit.runners.Suite;
import org.junit.runners.model.InitializationError;
import org.junit.runners.model.RunnerBuilder;
import org.junit.runners.model.RunnerScheduler;

import utilities.annotations.Concurrent;

import java.util.Arrays;
import java.util.LinkedList;
import java.util.List;
import java.util.Queue;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.atomic.AtomicInteger;

/**
 * @author Mathieu Carbou ([e-mail])
 */
public final class ConcurrentSuite extends Suite {
    public ConcurrentSuite(final Class<?> klass) throws InitializationError {
        super(klass, new AllDefaultPossibilitiesBuilder(true) {
            @Override
            public Runner runnerForClass(Class<?> testClass) throws Throwable {
                List<RunnerBuilder> builders = Arrays.asList(
                        new RunnerBuilder() {
                            @Override
                            public Runner runnerForClass(Class<?> testClass) throws Throwable {
                                Concurrent annotation = testClass.getAnnotation(Concurrent.class);
                                if (annotation != null)
                                    return new ConcurrentJunitRunner(testClass);
                                return null;
                            }
                        },
                        ignoredBuilder(),
                        annotatedBuilder(),
                        suiteMethodBuilder(),
                        junit3Builder(),
                        junit4Builder());
                for (RunnerBuilder each : builders) {
                    Runner runner = each.safeRunnerForClass(testClass);
                    if (runner != null)
                        return runner;
                }
                return null;
            }
        });
        setScheduler(new RunnerScheduler() {
            ExecutorService executorService = Executors.newFixedThreadPool(
                    klass.isAnnotationPresent(Concurrent.class) ?
                            klass.getAnnotation(Concurrent.class).threads() :
                            (int) (Runtime.getRuntime().availableProcessors() * 1.5),
                    new NamedThreadFactory(klass.getSimpleName()));
            CompletionService<Void> completionService = new ExecutorCompletionService<Void>(executorService);
            Queue<Future<Void>> tasks = new LinkedList<Future<Void>>();

            @Override
            public void schedule(Runnable childStatement) {
                tasks.offer(completionService.submit(childStatement, null));
            }

            @Override
            public void finished() {
                try {
                    while (!tasks.isEmpty())
                        tasks.remove(completionService.take());
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                } finally {
                    while (!tasks.isEmpty())
                        tasks.poll().cancel(true);
                    executorService.shutdownNow();
                }
            }
        });
    }

    static final class NamedThreadFactory implements ThreadFactory {
        static final AtomicInteger poolNumber = new AtomicInteger(1);
        final AtomicInteger threadNumber = new AtomicInteger(1);
        final ThreadGroup group;

        NamedThreadFactory(String poolName) {
            group = new ThreadGroup(poolName + "-" + poolNumber.getAndIncrement());
        }

        @Override
        public Thread newThread(Runnable r) {
            return new Thread(group, r, group.getName() + "-thread-" + threadNumber.getAndIncrement(), 0);
        }
    }

}

And the annotation is as follows.

package utilities.annotations;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * @author Mathieu Carbou ([e-mail])
 */
@Retention(RetentionPolicy.RUNTIME)
@Target({ ElementType.TYPE })
public @interface Concurrent {
    int threads() default 5;
}

Answer 2

Since Suite is used to annotate a Class, so run the Suite-annotated class in JUnitCore.runClasses(ParallelComputer.classes(), cls) way. cls are Suite-annotated classes.

@RunWith(Suite.class)
@Suite.SuiteClasses({
Test1.class,
Test2.class})
public class Suite1 {
}

@RunWith(Suite.class)
@Suite.SuiteClasses({
Test3.class,
Test4.class})
public class Suite2 {
}
...
JUnitCore.runClasses(ParallelComputer.classes(), new Class[]{Suite1.class, Suite2.class})