Run test based on defined priority with multiple c

问题:

I have 3 classes for with 3 tests each.

Class 1

@Test( priority = 1 )
public void testA1() {
    System.out.println("testA1");
}

@Test( priority = 2 )
public void testA2() {
    System.out.println("testA2");
}

@Test( priority = 3 )
public void testA3() {
    System.out.println("testA3");
}

Class 2

@Test( priority = 1 )
public void testB1() {
    System.out.println("testB1");
}

@Test( priority = 2 )
public void testB2() {
    System.out.println("testB2");
}

@Test( priority = 3 )
public void testB3() {
    System.out.println("testB3");
}

Class 3

@Test( priority = 1 )
public void testC1() {
    System.out.println("testC1");
}

@Test( priority = 2 )
public void testC2() {
    System.out.println("testC2");
}

@Test( priority = 3 )
public void testC3() {
    System.out.println("testC3");
}

This is my XML file code.

<test verbose="2" name="hello" group-by-instances="true">
    <classes>
        <class name="Class1"></class>
        <class name="Class2"></class>
        <class name="Class3"></class>
    </classes>
</test>

Here is my Answer testA1 testB1 testC1 testA2 testB2 testC2 testA3 testB3 testC3

But my expected answer is testA1 testA2 testA3 testB1 testB2 testB3 testC1 testC2 testC3

Thanks in advance for any help on this.

回答1:

The seen behavior is the expected one.

In fact, priority is more important that group-by-instances ( https://github.com/cbeust/testng/blob/master/CHANGES.txt#L48) and that's why TestNG respect priority instead of group-by-instances.

To achieve your expected behavior, you have to replace priority by a more important order feature, like dependsOnMethods:

@Test
public void testA1() {
    System.out.println("testA1");
}

@Test( dependsOnMethods = "testA1" )
public void testA2() {
    System.out.println("testA2");
}

@Test( dependsOnMethods = "testA2" )
public void testA3() {
    System.out.println("testA3");
}

---

As asked in the comments, if you really want a "priority on a class without a strong dependency", you can make it yourself with a method interceptor where you can order methods as you want. In pseudo code, something like:

public class PriorityOnClassOrder implements IMethodInterceptor {

  public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
    // 1. Group by instance/class
    Map<Class<?>, List<IMethodInstance>> map = ...
    for (IMethodInstance method : methods) {
      map.get(method.getInstance().getClass()).add(method);
    }

    List<IMethodInstance> result = ...
    // 2. Order methods from an instance/clas according to their priority
    for(Map.Entry entry : map.entries()) {
      List<IMethodInstance> m = entry.value();
      Collections.sort(m, new Comparator<IMethodInstance>() {
        public int compare(IMethodInstance o1, IMethodInstance o2) {
          return o1.getMethod().getPriority() - o2.getMethod().getPriority()
        }
      });
      result.addAll(m);
    }

    // 3. Return the result
    return result;
  }
}

回答2:

As you probably noticed, priority flag affect for whole not for single class. The easiest way is to jus increase priority level in second class.

@Test( priority = 4 )
public void testA1() {
    System.out.println("testA1");
}

@Test( priority = 5 )
public void testA2() {
    System.out.println("testA2");
}

@Test( priority = 6 )
public void testA3() {
    System.out.println("testA3");
}

Also you can put single class in single , i think it's even better if you want to separate tests domain.

 <test name="Test1" verbose="3" >
  <classes>
    <class name="tests.NewTest"></class>
  </classes>
  </test> <!-- Test -->
  <test name="Test2" verbose="3" >
   <classes>
    <class name="tests.NewTest2"></class>
  </classes>
  </test>

And verbose flag. I hardly recommend this during debugging.

回答3:

Finally I got solution of problem.

Please refer below XML code for same.

<classes>
            <class name="Class1">
                <methods>
                    <include name="testA1" />
                    <include name="testA2" />
                    <include name="testA3" />
                </methods>
            </class>
           <class name="Class2">
                <methods>
                    <include name="testB1" />
                    <include name="testB2" />
                    <include name="testB3" />
                </methods>
            </class>
            <class name="Class3">
                <methods>
                    <include name="testC1" />
                    <include name="testC2" />
                    <include name="testC3" />
                </methods>
            </class>

This will give you expected result. Plus it will be easier to manage also. As if we want to see my all tests or remove some tests, i can look into test.xml