TestNG iterate over test data instead of test meth

问题:

When using DataProvider with multiple TestNG methods, every method is run with all data sets in sequence. Instead I want to iterate over the data sets and execute all methods on each iteration. I do not care if the results show every single test method result or a summary of the runs per method.

I already tried the option

order-by-instances="true"

in a suite.xml with no success.

Sample code:

public class TestNGTest
{
    @DataProvider(name = "dp")
    public Object[][] createData(Method m) {
      return new Object[][] { new Object[] { "Cedric" }, new Object[] {"Martina"}};
    }

    @Test(dataProvider = "dp")    
    public void test1(String s) throws InterruptedException {
        System.out.println("test1 " + s);
        Thread.sleep(1000);
    }

    @Test(dataProvider = "dp")
    public void test2(String s) throws InterruptedException {
        System.out.println("test2 " + s);
        Thread.sleep(1000);
    }
}

Actual result:

test1 Cedric
test1 Martina
test2 Cedric
test2 Martina
PASSED: test1("Cedric")
PASSED: test1("Martina")
PASSED: test2("Cedric")
PASSED: test2("Martina")

Wanted result:

test1 Cedric
test2 Cedric
test1 Martina
test2 Martina
PASSED: test1("Cedric")
PASSED: test2("Cedric")
PASSED: test1("Martina")
PASSED: test2("Martina")

回答1:

Please try with following listener GroupByInstanceEnabler. You can put this listener in Listeners annotation in your test class (or test base class if have such) or just even simpler and better solution is to put it in META-INF to let TestNg load it using ServiceLoader (http://testng.org/doc/documentation-main.html#listeners-service-loader)

This will allow you to get rid of suite.xml and only what you will need to keep this META-INF and enabler on your classpath. Anytime you will run any test this will be loaded - not need to configure anything like IDE, create suites to run - it always will load your listener out-of-the-box.

import org.testng.ISuite;
import org.testng.ISuiteListener;

public class GroupByInstanceEnabler implements ISuiteListener {

    @Override
    public void onStart(ISuite suite) {
        suite.getXmlSuite().setGroupByInstances(true);
    }

    @Override
    public void onFinish(ISuite suite) {

    }
}

Pawel

回答2:

Found a solution using Factory, DataProvider and an injected XmlTest object.

Of course, the TestClass can be in another file. I prefer to keep it as an inner class however, because it can't be run without the factory anyways. The only downside of this solution is that you can't run single tests from the class.

package com.nhp.ts.test.business.qi;

import org.testng.annotations.BeforeTest;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
import org.testng.xml.XmlTest;

public class FooTest {

    @DataProvider(name = "dp")
    public Object[][] createData() {
        return new Object[][] {
            { "Cedric" },
            { "Martina" }
        };
    }

    @Factory(dataProvider = "dp")
    public TestClass[] testFactory(String name) {
        return new TestClass[] { new TestClass(name) };
    }

    public class TestClass {

        private String name;

        public TestClass(String name) {
            this.name = name;
        }

        @BeforeTest
        public void setOptions(XmlTest test)
        {
            test.setGroupByInstances(true);
        }

        @Test
        public void test1() throws InterruptedException {
            System.out.println("test1 " + name);
        }

        @Test
        public void test2() throws InterruptedException {
            System.out.println("test2 " + name);
        }
    }
}