Running MsTest from the command line with a custom

Question

I did quite a lot of research on the web and tried a few settings, but I couldn't reproduce the behavior of running MsTest in Visual Studio 2012 on the command line.

Our solution consists of many projects that build to the same bin folder residing at the solution level (e.g. C:\MySolution\bin) - this is the code-under-test (CUT). The tests are grouped in a separate project that resides in its own solution and builds in its own bin folder (e.g. C:\MySolution\Tests\bin). There are really a lot of plugins thus we want MsTest to reference the CUT bin folder when running the test intead of copying everything to the TestResults folder. We did achieve this in Visual Studio 2012 by editing the .testrunconfig and specifying ".\bin" as "root folder for the assemblies to be loaded" (in tab "Unit Test" when editing the testrunconfig). So we can load the test solution in VS2012 and run the tests there without having to copy the bin folder contents to the TestResults directory.

Now I wanted to create a .bat file that would run MsTest the same way as in VS2012 so that we can omit launching Visual Studio just for running the tests. I'm now working on how to execute MsTest on the command line, but have been quite frustrated. This is what I tried (command executed on the solution level in a VS command prompt):

MsTest /testcontainer:Tests\bin\Tests.dll

This did not work at all, it couldn't even find the referenced dlls that Tests.dll need to run. So I re-used the configuration and ran it again:

MsTest /runconfig:LocalTestRun.testrunconfig /testcontainer:Tests\bin\Tests.dll

Still it did not work. It could start the tests, but all of them failed. I got a lot of warnings of the kind

Warning: Test Run deployment issue: The assembly or module '....' directly or indirectly referenced by the test container 'C:\MySolution\Tests\bin\tests.dll' was not found.

and in the end it said:

The configured application base directory 'C:\MySolution\TestResults\User_Machine 2013-07-28 13_16_59\Out\bin' does not exist. The test directory will be used instead.

When I changed the option applicationBaseDirectory in the testrunconfig to an absolute path (C:\MySolution\bin), it worked. Still I get many warnings such as:

Warning: Test Run deployment issue: The assembly or module '....' directly or indirectly referenced by the test container 'C:\MySolution\Tests\bin\tests.dll' was not found.

But anyway, it's not really a feasible solution to specify an absolute path. How can I run MsTest on the command line with a different, but relative assembly base directory?

My LocalTestRun.testrunconfig is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local Test Run" id="...." xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Description>This is a default test run configuration for a local test run.</Description>
  <Deployment>
    <DeploymentItem filename="Tests\....\Resources\" />
  </Deployment>
  <Execution hostProcessPlatform="MSIL">
    <TestTypeSpecific>
      <UnitTestRunConfig testTypeId="....">
        <AssemblyResolution applicationBaseDirectory=".\bin">
           <TestDirectory useLoadContext="true" />
        </AssemblyResolution>
      </UnitTestRunConfig>
      <WebTestRunConfiguration testTypeId="....">
        <Browser name="Internet Explorer 7.0">
          <Headers>
            <Header name="User-Agent" value="Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" />
            <Header name="Accept" value="*/*" />
            <Header name="Accept-Language" value="{{$IEAcceptLanguage}}" />
            <Header name="Accept-Encoding" value="GZIP" />
          </Headers>
        </Browser>
      </WebTestRunConfiguration>
    </TestTypeSpecific>
    <AgentRule name="LocalMachineDefaultRole">
    </AgentRule>
  </Execution>
</TestSettings>

Answer 1

This is due to an MSTest bug that sets the current directory to its own working directory, rather than the test project's bin (or deployment) folder. The workaround is to execute the following code in the constructor of your test class:

Environment.CurrentDirectory(AppDomain.CurrentDomain.BaseDirectory);

I got the idea from http://www.ademiller.com/blogs/tech/2008/01/gotchas-mstest-appdomain-changes-in-vs-2008/; however, note that, in my case at least, it required setting Environment.CurrentDirectory, rather than the reverse, as suggested in the article.

Answer 2

After more searching we changed to use the test console runner that is included with VS2012:

VSTest.Console.exe Tests\bin\Tests.dll /Framework:framework40 /Settings:LocalTestRun.testrunconfig

This works with a relative path as applicationBaseDirectory.