Performing a Stress Test on Web Application?

Question

In the past, I used Microsoft Web Application Stress Tool and Pylot to stress test web applications. I'd written a simple home page, login script, and site walkthrough (in an ecommerce site adding a few items to a cart and checkout).

Just hitting the homepage hard with a handful of developers would almost always locate a major problem. More scalability problems would surface at the second stage, and even more - after the launch.

The URL of the tools I used were Microsoft Homer (aka Microsoft Web Application Stress Tool) and Pylot.

The reports generated by these tools never made much sense to me, and I would spend many hours trying to figure out what kind of concurrent load the site would be able to support. It was always worth it because the stupidest bugs and bottlenecks would always come up (for instance, web server misconfigurations).

What have you done, what tools have you used, and what success have you had with your approach? The part that is most interesting to me is coming up with some kind of a meaningful formula for calculating the number of concurrent users an app can support from the numbers reported by the stress test application.

Answer 1

I second the opensta suggestion. I would just add that it allows you to do things to monitor the server you're testing using SMTP. We keep track of processor load, memory used, byes sent, etc. The only downside is that if you find something boken and want to do a fix it relies on several open-source libraries that are no longer kept up, so getting a compiling version of the source is more tricky than with most OSS.

Answer 2

I had good results with FunkLoad :

• easy to script user interaction
• reports are clear
• can monitor server load

Answer 3

I vote for jMeter too and I want add some quotes to @PeterBernier answer.

The main question that load testing answers is how many concurrent users can my web application support? In order to get a proper answer, load testing should represent real application usage, as close as possible.

Keep above in mind, jMeter has many building blocks Logical Controllers, Config Elements, Pre Processors, Listeners ,... which can help you in this.

You can mimic real world situation with jMeter, for example you can:

1. Configure jMeter to act as real Browser by configuring (concurrent resource download, browser cache, http headers, setting request time out, cookie management, https support, encoding , ajax support ,... )
2. Configure jMeter to generate user requests (by defining number of users per second, ramp-up time, scheduling ,...)
3. Configure lots of client with jMeter on them, to do a distributed load test.
4. Process response to find if the server is responding correctly during test. ( For example assert response to find a text in it)

Please consider:

• It is easy to start a real web application test with jMeter in minutes. The jMeter has a very easy tool which record your test scenario ( know as HTTP(S) Test Script Recorder).
• jMeter has lots of plugins at http://jmeter-plugins.org.
• The jMeter UI is swing based and has made good changes in jMeter 3.2. On the other hand please consider that JMeter GUI should only be used for test and debugging. It is not good practice to use it in GUI mode for actual test. https://www.blazemeter.com/blog/5-ways-launch-jmeter-test-without-using-jmeter-gui. Configure and test your scenario and run it on non-gui mode.
• The are lots of reporting showing tools in jMeter (Known as listeners) but there are not meant to be on during test. You must run your test and generate reports ( .jtl files). Then you must use these tools to analyze result. Please have a look at https://www.blazemeter.com/blog/jmeter-listeners-part-1-basic-display-formats or https://www.tutorialspoint.com/jmeter/jmeter_listeners.htm.

The https://www.blazemeter.com/jmeter has very good and practical information to help you configure your test environment.

Answer 4

Take a look at LoadBooster(https://www.loadbooster.com). It utilizes headless scriptable browser PhantomJS/CasperJs to test web sites. Phantomjs will parse and render every page, execute the client-side script. The headless browser approach is easier to write test scenarios to support complex AJAX heavy Web 2.0 app，browser navigation, mouse click and keystrokes into the browser or wait until an element exists in DOM. LoadBooster support selenium HTML script too.

Disclaimer: I work for LoadBooster.

Answer 5

A little late to this party. I agree that Pylot is the best up-and-coming open source tool out there. It's simple to use and is actively worked on by a great guy (Corey Goldberg). As the founder of OpenQA, I'm also happy that Pylot now is listed on our home page and uses some of our infrastructure (namely the forums).

However, I also recently decided that the entire concept of load testing was flawed: emulating HTTP traffic, with applications as complex as they have become, is a pain in the butt. That's why I created the commercial tool BrowserMob. It's an external load testing service that uses Selenium to control real web browsers when playing back load.

The approach obviously requires a ton more hardware than normal load testing techniques, but hardware is actually pretty cheap when you are using cloud computing. And a nice side effect of this is that the scripting is much easier than normal load testing. You don't have to do any advanced regex matching (like JMeter requires) to extract out cookies, .NET session state, Ajax request parameters, etc. Since you're using real browsers, they just do what they are supposed to do.

Sorry to blatantly pitch a commercial product, but hopefully the concept is interesting to some folks and at least gets them thinking about some new ways to deal with load testing when you have access to a bunch of extra hardware!

Answer 6

As this question is still open, I might as well weigh in.

The good news is that over the past 5 or so years the Open Source tools have really matured and taken off in the space, the bad news is there are so many of them out there.

Here are my thoughts:-

Jmeter vs Grinder

Jmeter is driven from an XML style specification, that is constructed via a GUI.

Grinder uses Jython scripting within a muti-threaded Java framework, so more oriented to programmers.

Both tools will handle HTTP and HTTPS and have a proxy recorder to get you started. Both tools use the Controller model to drive multiple test agents so scalability is not an issue (given access to the Cloud).

Which is better:-

A hard call as the learning curve is steep with both tools as you get into the more complicated scripting requirements for url rewriting, correlation, providing unique data per Virtual User and simulating first time or returning Users (by manipulating the HTTP Headers).

That said I would start with Jmeter as this tool has a huge following and there are many examples and tutorials on the web for using this tool. If and when you come to a 'road block', that is something you can't 'easily' do with Jmeter then have a look at the Grinder. The good news is both these tools have the same Java requirement and a 'mix and match' solution is not out of the question.

Something new to add – Headless browsers running multiple instances of Selenium WebDriver.

This is a relatively new approach because it relies on the availability of resources that can now be provisioned from the Cloud. With this approach a Selenium (WebDriver) script is taken and run within a headless browser (i.e. WebDriver = New HtmlUnitDriver()) driver in multiple threads.

From experience around 25 instances of 'headless browsers' can be executed from the Amazon M1 Small Instance.

What this means is that all of the correlation, url rewriting issues disappear as you repurpose your functional testing scripts to become performance testing scripts.

The scalability is compromised as more VMs will be needed to drive the load, as compared with a HTTP driver such as the Grinder or Jmeter. That said, if you are looking to drive 500 Virtual Users then with 20 Amazon Small Instances (6 cents an hour each) at a cost of just $1.20 per hour gives you load that is very close to the Real User Experience.