Execute unit tests serially (rather than in parall

Question

I am attempting to unit test a WCF host management engine that I have written. The engine basically creates ServiceHost instances on the fly based on configuration. This allows us to dynamically reconfigure which services are available without having to bring all of them down and restart them whenever a new service is added or an old one is removed.

I have run into a difficulty in unit testing this host management engine, however, due to the way ServiceHost works. If a ServiceHost has already been created, opened, and not yet closed for a particular endpoint, another ServiceHost for the same endpoint can not be created, resulting in an exception. Because of the fact that modern unit testing platforms parallelize their test execution, I have no effective way to unit test this piece of code.

I have used xUnit.NET, hoping that because of its extensibility, I could find a way to force it to run the tests serially. However, I have not had any luck. I am hoping that someone here on SO has encountered a similar issue and knows how to get unit tests to run serially.

NOTE: ServiceHost is a WCF class, written by Microsoft. I don't have the ability to change it's behavior. Hosting each service endpoint only once is also the proper behavior...however, it is not particularly conducive to unit testing.

Answer 1

For .NET Core projects, you can configure xUnit with an xunit.runner.json file, as documented at https://xunit.github.io/docs/configuring-with-json.html.

The setting you need to change to stop parallel test execution is parallelizeTestCollections, which defaults to true:

Set this to true if the assembly is willing to run tests inside this assembly in parallel against each other. ... Set this to false to disable all parallelization within this test assembly.

JSON schema type: boolean
Default value: true

So a minimal xunit.runner.json for this purpose looks like

{
    "parallelizeTestCollections": false
}

As noted in the docs, remember to include this file in your build, either by:

• Setting Copy to Output Directory to Copy if newer in the file's Properties in Visual Studio, or

• Adding

  <Content Include=".\xunit.runner.json">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </Content>
  

  to your .csproj file, or

• Adding

  "buildOptions": {
    "copyToOutput": {
      "include": [ "xunit.runner.json" ]
    }
  }
  

  to your project.json file

depending upon your project type.

Finally, in addition to the above, if you're using Visual Studio then make sure that you haven't accidentally clicked the Run Tests In Parallel button, which will cause tests to run in parallel even if you've turned off parallelisation in xunit.runner.json. Microsoft's UI designers have cunningly made this button unlabelled, hard to notice, and about a centimetre away from the "Run All" button in Test Explorer, just to maximise the chance that you'll hit it by mistake and have no idea why your tests are suddenly failing: