Coded ui Test fails at random times on server

问题:

I am fairly new to the testing environment but I am stuck with a problem.

i have created a test case (a few tests combined in an ordered test) and when I run them on my own pc they always pass (tried it many times to be sure) but when I run the test on our server the test fails at random times. examples - he doesn't right click a list to get the context menu - he seems to forget to click on a button so he cant access the next window. etc if i run the test again he may go over the previous fail but fails on something else.

so far. out of +- 30 times i have run he test i had 5 success runs. and this should be a base test so there should be no bug or known problem. i have the feeling that the server needs more time to complete the test. so i did research and already added many playbacksettings and a Playback_PlaybackError. test case made in Visual studio 2013 part with recording part written code. build in visual studio and server tested with microsoft test manager 2013, win8 envir

is there anything i do wrong? or is there something wrong with the server configuration?

Thanks in advance.

so far I tried some of these (and repeat in every testmethod)

    public CodedUITest1()
    {
        Playback.PlaybackSettings.MatchExactHierarchy = true;

        Playback.PlaybackSettings.SmartMatchOptions = SmartMatchOptions.Control;
        Playback.PlaybackSettings.SmartMatchOptions = SmartMatchOptions.TopLevelWindow;
        Playback.PlaybackSettings.SmartMatchOptions = SmartMatchOptions.None;

        Playback.PlaybackSettings.SearchTimeout = 2000;
        Playback.PlaybackSettings.ShouldSearchFailFast = true;

        Playback.PlaybackSettings.ThinkTimeMultiplier = 2;

        Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.AllThreads;
        Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.UIThreadOnly;
        Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.Disabled;

        Playback.PlaybackSettings.WaitForReadyTimeout = 2000;
        Playback.PlaybackError -= Playback_PlaybackError;
        Playback.PlaybackError += Playback_PlaybackError;
    }

    /// <summary> PlaybackError event handler. </summary>
    private static void Playback_PlaybackError(object sender, PlaybackErrorEventArgs e)
    {
        // Wait a second
        System.Threading.Thread.Sleep(1000);

        // Retry the failed test operation
        e.Result = PlaybackErrorOptions.Retry;

    }

回答1:

Try to use

yourcontrol.WaitForControlReady() 

function as well before performing any action on control. This function will stop the thread till your control becomes ready for accepting any action.

回答2:

Seems the problem was server based. I added a global delaybetweenactions and the test seems to run very smooth. Problem fixed

回答3:

I had the similar problem with a set of 20 odd Coded UI Tests in my project, which used to randomly fail on the server, but always ran fine locally. We looked out for a number of troubleshooting techniques to overcome this mysterious 'random' factor. The biggest problem while analyzing these test failures is that the error stack trace might indicate the line of code which might be completely unrelated to the actual cause of failure.

We figured out that we can enable HTML logging in our Coded UI tests. This is very easy and can either be enabled for individual tests or for all the tests in the project. Just add the below code to your app.config file

Once you have the tracing enabled, the tests will display step by step details of how Coded UI tried to execute the tests - with screenshots of your application. This is very beneficial in troubleshooting the actual cause of test failures.