In MSTest, How can I verify exact error message us

问题:

Using MSTest how can I verify the exact error message coming from a test method? I know [ExpectedException(typeof(ApplicationException), error msg)] doesn't compare the error message coming from my test method, though in other unit test framework it is doing.

One way to solve this problem is to write my unit test using some try catch block, but again I need to write 4 lines more.

Is there any smartest way to check the error message.

Cheers, Pritam

回答1:

You can create your own ExpectedException attribute where you can Assert the message of the Exception that was thrown.

Code

namespace TestProject
{
    public sealed class MyExpectedException : ExpectedExceptionBaseAttribute
    {
        private Type _expectedExceptionType;
        private string _expectedExceptionMessage;

        public MyExpectedException(Type expectedExceptionType)
        {
            _expectedExceptionType = expectedExceptionType;
            _expectedExceptionMessage = string.Empty;
        }

        public MyExpectedException(Type expectedExceptionType, string expectedExceptionMessage)
        {
            _expectedExceptionType = expectedExceptionType;
            _expectedExceptionMessage = expectedExceptionMessage;
        }

        protected override void Verify(Exception exception)
        {
            Assert.IsNotNull(exception);

            Assert.IsInstanceOfType(exception, _expectedExceptionType, "Wrong type of exception was thrown.");

            if(!_expectedExceptionMessage.Length.Equals(0))
            {
                Assert.AreEqual(_expectedExceptionMessage, exception.Message, "Wrong exception message was returned.");
            }
        }
    }
}

Usage

[TestMethod]
[MyExpectedException(typeof(Exception), "Error")]
public void TestMethod()
{
    throw new Exception("Error");
}

回答2:

Use this little helper class:

public static class ExceptionAssert
{
    public static void Throws<TException>(Action action, string message)
        where TException : Exception
    {
        try
        {
            action();

            Assert.Fail("Exception of type {0} expected; got none exception", typeof(TException).Name);
        }
        catch (TException ex)
        {
            Assert.AreEqual(message, ex.Message);
        }
        catch (Exception ex)
        {
            Assert.Fail("Exception of type {0} expected; got exception of type {1}", typeof(TException).Name, ex.GetType().Name);               
        }
    }
}

Usage:

Foo foo = new Foo();
foo.Property = 42;

ExceptionAssert.Throws<InvalidOperationException>(() => foo.DoSomethingCritical(), "You cannot do anything when Property is 42.");

The advantage of explicit catching exceptions is that teh test does not succeed when another member (e.g. during the initialization) throws the exception.

回答3:

I was looking for a way to check the presence and type of inner exception with mstest and I found this question. I known it a 2 years old topic but since my solution is not here, let me share it.

To me, the most elegant way to solve the problem is to create a derived attribute, here's mine (sorry but comments and strings are in french, my natural language, but it should be obvious) :

#region Références
using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Text.RegularExpressions;
#endregion

namespace MsTestEx
{
    /// <summary>
    /// Extention de l'attribut ExpectedException permettant de vérifier plus d'éléments (Message, InnerException, ...)
    /// </summary>
    public class ExpectedExceptionEx
        : ExpectedExceptionBaseAttribute
    {
        #region Variables locales
        private Type    _ExpectedException              = null;
        private string  _ExpectedMessage                = null;
        private Type    _ExpectedInnerException         = null;
        private string  _ExpectedInnerExceptionMessage  = null;
        private bool    _IsExpectedMessageRegex         = false;
        private bool    _IsExpectedInnerMessageRegex    = false;
        private bool    _AllowDerivedType               = false;
        private bool    _AllowInnerExceptionDerivedType = false;

        private bool    _CheckExpectedMessage           = false;
        private bool    _CheckInnerExceptionType        = false;
        private bool    _CheckInnerExceptionMessage     = false;
        #endregion

        #region Propriétés
        /// <summary>
        /// Vérifie que le message de l'exception correspond à celui-ci.
        /// </summary>
        public string ExpectedMessage
        {
            get { return _ExpectedMessage; }
            set { _ExpectedMessage = value; _CheckExpectedMessage = true; }
        }

        /// <summary>
        /// Vérifie que le message de l'inner-exception correspond à celui-ci.
        /// </summary>
        public string ExpectedInnerExceptionMessage
        {
            get { return _ExpectedInnerExceptionMessage; }
            set { _ExpectedInnerExceptionMessage = value; _CheckInnerExceptionMessage = true; }
        }

        /// <summary>
        /// Vérifie que l'exception possède bien une inner-exception du type spécifié.
        /// Spécifier "null" pour vérifier l'absence d'inner-exception.
        /// </summary>
        public Type ExpectedInnerException
        {
            get { return _ExpectedInnerException; }
            set { _ExpectedInnerException = value; _CheckInnerExceptionType = true; }
        }

        /// <summary>
        /// Indique si le message attendu est exprimé via une expression rationnelle.
        /// </summary>
        public bool IsExpectedMessageRegex
        {
            get { return _IsExpectedMessageRegex; }
            set { _IsExpectedMessageRegex = value; }
        }

        /// <summary>
        /// Indique si le message attendu de l'inner-exception est exprimé via une expression rationnelle.
        /// </summary>
        public bool IsExpectedInnerMessageRegex
        {
            get { return _IsExpectedInnerMessageRegex; }
            set { _IsExpectedInnerMessageRegex = value; }
        }

        /// <summary>
        /// Indique si les exceptions dérivées sont acceptées.
        /// </summary>
        public bool AllowDerivedType
        {
            get { return _AllowDerivedType; }
            set { _AllowDerivedType = value; }
        }

        /// <summary>
        /// Indique si les inner-exceptions dérivées sont acceptées.
        /// </summary>
        public bool AllowInnerExceptionDerivedType
        {
            get { return _AllowInnerExceptionDerivedType; }
            set { _AllowInnerExceptionDerivedType = value; }
        }
        #endregion

        #region Constructeurs
        /// <summary>
        /// Indique le type d'exception attendu par le test.
        /// </summary>
        /// <param name="expectedException">Type de l'exception attendu.</param>
        public ExpectedExceptionEx(Type expectedException)
        {
            _ExpectedException = expectedException;
        }
        #endregion

        #region Méthodes
        /// <summary>
        /// Effectue la vérification.
        /// </summary>
        /// <param name="exception">Exception levée.</param>
        protected override void Verify(Exception exception)
        {
            Assert.IsNotNull(exception); // Pas eu d'exception, ce n'est pas normal

            // Vérification du type de l'exception
            Type actualType = exception.GetType();
            if (_AllowDerivedType) Assert.IsTrue(_ExpectedException.IsAssignableFrom(actualType), "L'exception reçue n'est pas du type spécifié ni d'un type dérivé.");
            else Assert.AreEqual(_ExpectedException, actualType, "L'exception reçue n'est pas du type spécifié.");

            // Vérification du message de l'exception
            if (_CheckExpectedMessage)
            {
                if (_IsExpectedMessageRegex)
                    Assert.IsTrue(Regex.IsMatch(exception.Message, _ExpectedMessage), "Le message de l'exception ne correspond pas à l'expression rationnelle");
                else
                {
                    string s1, s2;
                    if (exception.Message.Length > _ExpectedMessage.Length)
                    {
                        s1 = exception.Message;
                        s2 = _ExpectedMessage;
                    }
                    else
                    {
                        s1 = _ExpectedMessage;
                        s2 = exception.Message;
                    }
                    Assert.IsTrue(s1.Contains(s2), "Le message de l'exception ne contient pas et n'est pas contenu par le message attendu.");
                }
            }

            if (_CheckInnerExceptionType)
            {
                if (_ExpectedInnerException == null) Assert.IsNotNull(exception.InnerException);
                else
                {
                    // Vérification du type de l'exception
                    actualType = exception.InnerException.GetType();
                    if (_AllowInnerExceptionDerivedType) Assert.IsTrue(_ExpectedInnerException.IsAssignableFrom(actualType), "L'inner-exception reçue n'est pas du type spécifié ni d'un type dérivé.");
                    else Assert.AreEqual(_ExpectedInnerException, actualType, "L'inner-exception reçue n'est pas du type spécifié.");
                }
            }

            if (_CheckInnerExceptionMessage)
            {
                Assert.IsNotNull(exception.InnerException);
                if (_IsExpectedInnerMessageRegex)
                    Assert.IsTrue(Regex.IsMatch(exception.InnerException.Message, _ExpectedInnerExceptionMessage), "Le message de l'exception ne correspond pas à l'expression rationnelle");
                else
                {
                    string s1, s2;
                    if (exception.InnerException.Message.Length > _ExpectedInnerExceptionMessage.Length)
                    {
                        s1 = exception.InnerException.Message;
                        s2 = _ExpectedInnerExceptionMessage;
                    }
                    else
                    {
                        s1 = _ExpectedInnerExceptionMessage;
                        s2 = exception.InnerException.Message;
                    }
                    Assert.IsTrue(s1.Contains(s2), "Le message de l'inner-exception ne contient pas et n'est pas contenu par le message attendu.");
                }
            }
        }
        #endregion
    }
}

Now, use this attribute with named parameters instead of the "ExpectedException". With my attribute you can check if there is an inner exception, message of the exception and inner exception, use a regex to match messages, etc... You can adapt as you want.

回答4:

Fluent Assertions (NuGet) has a very "language natural" syntax for defining expectations in unit tests:

objectundertest.Invoking(o => o.MethodUnderTest()).ShouldThrow<ExpectedException>()
    .WithMessage("the expected error message");

There are multiple variations for checking the error message with any algorithm (Where(e => ...) as well as checking into inner exceptions and their messages.

回答5:

In MSTest there's no built-in way of doing it. This is about as 'elegant' as it gets:

[TestMethod]
public void Test8()
{
    var t = new Thrower();
    try
    {
        t.DoStuffThatThrows();
        Assert.Fail("Exception expected.");
    }
    catch (InvalidOperationException e)
    {
        Assert.AreEqual("Boo hiss!", e.Message);
    }
}

However, you could consider porting xUnit.NET's Assert.Throws API to a custom library - that's what we did.

You could also go to Microsoft Connect an vote on this suggestion.

回答6:

MSTest v2 supports Assert.Throws and Assert.ThrowsAsync which returns the captured exception.

Here is an article on how to upgrade to MSTest v2: https://blogs.msdn.microsoft.com/devops/2017/09/01/upgrade-to-mstest-v2/

Here is example usage:

var myObject = new MyObject();
var ex = Assert.Throws<ArgumentNullException>(() => myObject.Do(null));
StringAssert.Contains(ex.Message, "Parameter name: myArg");

回答7:

The annoyance with annotations and try/catch blocks is that you don't have a clean separation between the ACT and ASSERT phases of the test. A simpler appraoch is to "capture" the exception as part of the ACT phase using a utitlity routine such as:

public static class Catch
{
    public static Exception Exception(Action action)
    {
        Exception exception = null;

        try
        {
            action();
        }
        catch (Exception ex)
        {
            exception = ex;
        }

        return exception;
    }
}

This allows you to do:

// ACT
var actualException = Catch.Exception(() => DoSomething())

// ASSERT
Assert.IsNotNull(actualException, "No exception thrown");
Assert.IsInstanceOfType(actualException, expectedType);
Assert.AreEqual(expectedExceptionMessage, actualException.Message);

回答8:

With MSTest, you can't do this.

You already know the solution to this problem: assert the message of an exception in a catch block.

回答9:

MbUnit can also do this:

[Test]
[Row(ExpectedExceptionMessage="my message")]
void TestBlah(...

回答10:

Update: Oops.. see that you want this in MSTest. Sorry. Speed read & Misled by your title.

Try this extension project from Callum Hibbert and see if it works.

Old response:

You can do this with NUnit 2.4 and above. See documentation of ExpectedException here

[ExpectedException( typeof( ArgumentException), ExpectedMessage="unspecified", MatchType=MessageMatch.Contains )]
public void TestMethod()
{
...

MatchType can be Exact (default), Contains or Regex.. which pretty much handles the 80% use-case. There is also an advanced exception handler method approach if the verification gets too complex.. never used it personally.. didn't need it yet.