I am running a Web Performance test in Visual studio 2013 Ultimate version, I need some clarification on my scenario:
- I have a Login web test case, its using data source and running a
test case for 10 different user.
- I also have a Load test which is using above web test case and
running it for 50 constant user.
- I also have a "Percentage of new users" value set to 100
Does the above settings mean my Load test will run the web test for 50 concurrent users and it will each user will be randomly selected from data source of the web test ?
If the login web test has a data source containing 10 different users and the load test is running it for 50 constant users then each data source entry will be logged in for (on average) 5 virtual users at any point throughout the test.
When the Percentage of new users is 100: Whenever a virtual user finishes a web test that user finishes and a new virtual user is started so that the number of virtual users remains correct.
When the Percentage of new users is 0: Whenever a virtual user finishes a web test that user stays active so that the number of virtual users remains correct.
When the Percentage of new users is between 0 and 100: Whenever a virtual user finishes a web test a decision is made, based on the percentage, as to whether the current user finishes and a new user starts or whether the user stays active.
The points above about Percentage of new users should be interpreted to match the load pattern required. If the required number of virtual users differs from the actual number then new virtual users are created or existing users stopped as necessary. At the very start of a test run there are zero virtual users so enough are created to make the required number for a constant load or the initial number for a step load. At the end of a test run the required number is zero so users finish. (During the cool down period the required number is zero, so users and their tests are allowed to finish naturally. At the real end of the run the test just stops, all running tests and their users are just terminated.)
When a virtual user starts a new test values are read from the data source and (for Sequential and Unique accesses) the data source pointer is moved to the next entry. Thus with 10 data source entries and 50 virtual users with Sequential access we expect the first data source entry to be used by virtual user numbers 1, 11, 21, 31 and 41. Similarly the second entry will be used by 2, 12, 22, 32 and 42. And so on. If the data access is Random then you would expect each data source entry to be used by 5 virtual users, but as the entries are chosen randomly some are likely to be used by more than 5 and some less than 5 at any point in time. Over the whole duration of the test you should expect each data source value to be used approximately the same number of times.
Having 10 data source entries for 50 users is valid provided the system being tested allows a user to log in from multiple computers at the same time. (Note that each of these user will also log in from the same IP address, this can be changed but it can be complicated.) Generally I would recommend that the number of different logins in the data source exceeds the number of virtual users. Having at least twice as many would be good.
Two good sources of further information are the Content Index for Visual Studio Web Tests and Load Tests
and the Visual Studio Performance Testing Quick Reference Guide.
