How to run ClassCleanup (MSTest) after each class

问题:

I have several classes with tests suites.

Each test class starts from ClassInitialize and finishes by ClassCleanup. My problem is that ClassCleanup isn't called at the end of each class, it's called only after all tests in three classes. Can I fix this issue? Thanks!

[ClassInitialize]
public static void SetUpBrowser(TestContext context)
{
    pageObjectBase.SetBrowser("chrome");
    pagesManager.GetPageObjectBase();
}

[TestMethod]
public void FindCriticalBug()
{
    bla-bla-bla();
}

[ClassCleanup]
public static void CloseBrowser()
{
    pageObjectBase.Stop();
    pagesManager.GeneralClearing();
}

回答1:

Tests are run unordered, including tests across classes. See this blog post:

https://docs.microsoft.com/archive/blogs/ploeh/classcleanup-may-run-later-than-you-think

To quote:

In any case, here's the result from my Output Window:

AssemblyInitialize
TestClass1: ClassInitialize
TestClass1: TestInitialize
TestClass1: MyTestCase1
TestClass1: TestCleanup
TestClass2: ClassInitialize
TestClass2: TestInitialize
TestClass2: MyTestCase2
TestClass2: TestCleanup
TestClass1: ClassCleanup
TestClass2: ClassCleanup
AssemblyCleanup

...this doesn't mean that TestClass1's ClassCleanup executes immediately after the last test case in the class! In fact, it waits until all test cases are executed, and the executes together with TestClass2's ClassCleanup.

This surprised me at first, but that was obviously only because I hadn't really thought it through: Since tests are, in principle, unordered, there's not guarantee that all tests in TestClass1 are executed in immediate succession. Theoretically, the execution engine may pick a test case from TestClass1, then one from TestClass2, then another from TestClass1, etc. Since that is the case, there's no guarantee that all tests from one test class have been executed before a new test class is initialized, and thusly, all ClassCleanup methods may as well be deferred until all test cases have been executed.

Unfortunately, you'll have to look at ordered tests or at a different unit testing framework if this doesn't work for you.

回答2:

There is a different attribute called TestCleanupAttribute, which will run after every test.

There is also an attribute to run before every test called TestInitializeAttribute.

Here is an example of them running together.

[TestClass]
public class MyTests
{
   [ClassInitialize]
   public void ClassInitialize() { Debug.Print("Running ClassInitialize"); }

   [TestInitialize]
   public void TestInitialize() { Debug.Print("Running    TestInitialize"); }

   [TestMethod]
   public void TestMethod1() { Debug.Print("Running       TestMethod1....."); }

   [TestMethod]
   public void TestMethod2() { Debug.Print("Running       TestMethod2....."); }

   [TestCleanup]
   public void TestCleanup() { Debug.Print("Running    TestCleanup"); }

   [ClassCleanup]
   public void ClassCleanup() { Debug.Print("Running ClassCleanup"); }
}

This will result in

Running ClassInitialize
Running    TestInitialize
Running       TestMethod1.....
Running    TestCleanup
Running    TestInitialize
Running       TestMethod2.....
Running    TestCleanup
Running ClassCleanup

回答3:

I made a few tests and using a static field in the class to "tell" the TestCleanup method that all methods have run seems to work. You can then drop the ClassCleanup and do something like this:

private static int runs = 0;

[ClassInitialize]
public static void SetUpBrowser(TestContext context)
{
    pageObjectBase.SetBrowser("chrome");
    pagesManager.GetPageObjectBase();
}

[TestMethod]
public void FindCriticalBug()
{
    runs++;
    bla-bla-bla();
}

[TestMethod]
public void FindCriticalBug2()
{
    runs++;
    ble-ble-ble();
}

[TestCleanup]
public static void CloseBrowser()
{
    if (runs == 2)
    {
        pageObjectBase.Stop();
        pagesManager.GeneralClearing();
    }
}

I'd stay very far away from this solution though, but if you have no other alternative, and you can't refactor your design to use the provided lifecycle, it might be an option. You could probably get fancier here and write your own base class that counts executions and gets the total amount of test methods using reflection to automate this stuff.

回答4:

Normal unittest are 'unordered', which means they can run in any order. You are probably looking for something like ordered test (see your comment on dominic). Ordered test is a special unittest project. When running ordered tests the test will run how you configure them, and teardown testclasses when they are finished. If unit test have to run in order, it is a smell that the test are interfering with each other. Interfering test are unreliable because they fail because a earlier test left some bad data or the test it self fails. You have no way a knowing what is really wrong with your code.