ServiceStack Validation Not Always Firing

Question

So I was trying to build a End To End integration testing suite with RavenDB and ServiceStack but I ran into a really weird issue where the validation doesn't run on some requests. This is really strange and I am not sure what I am doing wrong. I am using NCrunch. Sometimes the test passes, sometimes it fails.

Hope this is an easy fix and something bone headed I'm doing.

You can download the whole project on http://github.com/khalidabuhakmeh/endtoend

You don't need anything other than VS2012 and NuGet Package Restore enabled.

UPDATE: I decided to run this in both NCrunch and Resharper Test Runner and both give the same result [see image below].

UPDATE UPDATE: I thought it could be XUnit, so I tried to use NUnit. Nope still the same problem.

**Another Update: Put in console writes as per user1901853's request. This was the result."

Latest Update: the RequestFilters are getting wiped out and I'm not sure why. It seems like it could be a threading issue, but I can't see where.

My AppHost is using AppHostListenerBase.

    using EndToEnd.Core;
    using Funq;
    using Raven.Client;
    using ServiceStack.ServiceInterface.Validation;
    using ServiceStack.WebHost.Endpoints;

    namespace EndToEnd
    {
        public class TestAppHost
            : AppHostHttpListenerBase
        {
            private readonly IDocumentStore _documentStore;

            public TestAppHost(IDocumentStore documentStore)
                : base("Test AppHost Api", typeof(TestAppHost).Assembly)
            {
                _documentStore = documentStore;
            }

            public override void Configure(Container container)
            {
                ServiceStack.Text.JsConfig.EmitCamelCaseNames = true;

                // Register RavenDB things
                container.Register(_documentStore);
                container.Register(c =>
                {
                    var db = c.Resolve<IDocumentStore>();
                    return db.OpenSession();
                }).ReusedWithin(ReuseScope.Request);

                Plugins.Add(new ValidationFeature());
                container.RegisterValidators(typeof(CreateWidgetValidator).Assembly);

                // todo: register all of your plugins here
                AuthConfig.Start(this, container);
            }
        }
    }

My base test class for all of my tests looks like this:

    using Raven.Client;
    using Raven.Client.Indexes;
    using Raven.Tests.Helpers;
    using ServiceStack.Authentication.RavenDb;
    using ServiceStack.ServiceClient.Web;
    using ServiceStack.ServiceInterface.Auth;

    namespace EndToEnd
    {
        public abstract class ServiceStackTestBase
            : RavenTestBase
        {
            protected IDocumentStore DocumentStore { get; set; }
            protected TestAppHost Host { get; set; }
            protected JsonServiceClient Client { get; set; }

            protected const string ListeningOn = "http://localhost:1337/";

            protected string Username { get { return "testuser"; } }
            protected string Password { get { return "password"; } }

            protected ServiceStackTestBase()
            {
                DocumentStore = NewDocumentStore();
                IndexCreation.CreateIndexes(typeof(ServiceStackTestBase).Assembly, DocumentStore);
                IndexCreation.CreateIndexes(typeof(RavenUserAuthRepository).Assembly, DocumentStore);

                Host = new TestAppHost(DocumentStore);
                Host.Init();
                Host.Start(ListeningOn);

                Client = new JsonServiceClient(ListeningOn)
                {
                    AlwaysSendBasicAuthHeader = true,
                    UserName = Username,
                    Password = Password
                };

                RegisterUser();

                WaitForIndexing(DocumentStore);
            }

            private void RegisterUser()
            {
                Client.Send(new Registration
                {
                    UserName = Username,
                    Password = Password,
                    DisplayName = "Test User",
                    Email = "[e-mail]",
                    FirstName = "test",
                    LastName = "user"
                });
            }

            public override void Dispose()
            {
                DocumentStore.Dispose();
                Host.Dispose();
            }
        }
    }

my test class looks like this:

    using System;
    using EndToEnd.Core;
    using FluentAssertions;
    using ServiceStack.FluentValidation;
    using ServiceStack.ServiceClient.Web;
    using ServiceStack.ServiceInterface.Auth;
    using Xunit;

    namespace EndToEnd
    {
        public class RegistrationTests
            : ServiceStackTestBase
        {
            [Fact]
            public void Throws_validation_exception_when_bad_widget()
            {
                var validator = Host.Container.Resolve<IValidator<CreateWidget>>();
                validator.Should().NotBeNull();


                try
                {
                    var response = Client.Post(new CreateWidget
                    {
                        Name = null
                    });
                    // It get's here every once in a while
                    throw new Exception("Should Not Get Here!");
                }
                catch (WebServiceException wex)
                {
                    wex.StatusCode.Should().Be(400);
                    wex.ErrorMessage.Should().Be("'Name' should not be empty.");
                }
            }
        }
    }

My code looks like this for the service:

    using System;
    using Raven.Client;
    using ServiceStack.FluentValidation;
    using ServiceStack.ServiceHost;
    using ServiceStack.ServiceInterface;
    using ServiceStack.ServiceInterface.ServiceModel;

    namespace EndToEnd.Core
    {
        [Authenticate]
        public class WidgetsService
            : Service
        {
            private readonly IDocumentSession _session;

            public WidgetsService(IDocumentSession session)
            {
                _session = session;
            }

            public CreateWidgetResponse Post(CreateWidget input)
            {
                var widget = new Widget { Name = input.Name };
                _session.Store(widget);
                _session.SaveChanges();

                return new CreateWidgetResponse { Widget = widget };
            }
        }

        [Route("/widgets", "POST")]
        public class CreateWidget : IReturn<CreateWidgetResponse>
        {
            public string Name { get; set; }
        }

        public class CreateWidgetResponse
        {
            public CreateWidgetResponse()
            {
                ResponseStatus = new ResponseStatus();
            }

            public Widget Widget { get; set; }
            public ResponseStatus ResponseStatus { get; set; }   
        }

        public class Widget
        {
            public Widget()
            {
                Created = DateTimeOffset.UtcNow;
            }

            public string Id { get; set; }
            public string Name { get; set; }
            public DateTimeOffset Created { get; set; }
        }

        public class CreateWidgetValidator : AbstractValidator<CreateWidget>
        {
            public CreateWidgetValidator()
            {
                RuleFor(m => m.Name).NotEmpty();
            }
        }
    }

Answer 1

I don't have the ability to duplicate your environment but while running in VS2010, using .NET 4, NUnit and ReSharper Test Runner I have not been able to reproduce your issue of 'Validation not firing'. I have run your tests 30+ times. A couple of reasons I can think of Validation not firing would be not having the plugin added or the plugin not registering the validation filters. The 2 if statements below might give you some 'introspection' if either of the cases I listed are the issue. Hope this is somewhat helpful.

if (!TestAppHost.Instance.Plugins.Any(x => x.GetType() == typeof(ValidationFeature)))
{
    Console.Write("Validation Plugin is not added");
    //TestAppHost.Instance.Plugins.Add(new ValidationFeature());
}

if (!TestAppHost.Instance.RequestFilters.Any(x => x.Target.ToString() == "ServiceStack.ServiceInterface.Validation.ValidationFilters"))
{
    Console.Write("No validation request filter");
   //TestAppHost.Instance.Container.RegisterValidators(typeof(CreateWidgetValidator).Assembly);
}

Below is my packages.config so you can see the differences in our environments.

<packages>
  <package id="FluentAssertions" version="2.0.1" targetFramework="net40" />
  <package id="NUnit" version="2.6.2" targetFramework="net40" />
  <package id="RavenDB.Client" version="2.0.2261" targetFramework="net40" />
  <package id="RavenDB.Database" version="2.0.2261" targetFramework="net40" />
  <package id="RavenDB.Embedded" version="2.0.2261" targetFramework="net40" />
  <package id="RavenDB.Tests.Helpers" version="2.0.2261" targetFramework="net40" />
  <package id="ServiceStack" version="3.9.38" targetFramework="net40-Client" />
  <package id="ServiceStack.Authentication.RavenDB" version="3.9.35" targetFramework="net40" />
  <package id="ServiceStack.Common" version="3.9.38" targetFramework="net40-Client" />
  <package id="ServiceStack.OrmLite.SqlServer" version="3.9.39" targetFramework="net40-Client" />
  <package id="ServiceStack.Redis" version="3.9.38" targetFramework="net40-Client" />
  <package id="ServiceStack.Text" version="3.9.38" targetFramework="net40-Client" />
  <package id="xunit" version="1.9.1" targetFramework="net40" />
</packages>