I am working on a proposal for my company into purchasing some load testing software. I have to give 2 choices. One of my choices I am proposing is HP(Mecurary) Loadrunner. Is there another program out there that is comparible with Loadrunner? We are looking at using the software to test the performance of the following.
C# .NET Template
Microsoft® .NET 2.0, 3.0
Microsoft® SQL Server Informix
Web (HTTP/HTML)
Web Services
Web and Multimedia
Wireless Application Protocol (WAP)
Domain Name System (DNS)
Microsoft® Exchange (MAPI)
OpenSourceTesting.com has a list of OSS performance tools.
The Software Testing FAQ also has a list of commercial and OSS performance testing tools.
Since it looks like you're on a Windows platform, you can also use the following tools:
- Visual Studio Application Center Test (Enterprise Editions only?)
- Microsoft Web Application Stress tool (aka WAS, aka "Homer"; predecessor to Application Center Test)
There are a lot of options out there, but it seems like you have a good handle on your requirements. I recommend doing a trade study of your top choices, using weighted criteria and a rank of how each product scores (sort of like a custom "Consumer Reports" style comparison). This will help you reduce any preconceived notions towards or against a product, and also gives your management a sound empirical reason as to why you choose the tool you did. This can go a long way in justifying the cost for a big ticket item.
Also, for the commercial tools, the vendors will often come on site and set up a demo system to test your code. We did this a few years back with Mercury, Rational and NuMega. It really helped cut through the presentation B.S. and see exactly how the products could be used with our apps.
We have used JMeter for HTTP and SOAP load testing
TestComplete from Automated QA is also very well respected:
http://www.automatedqa.com
The product is comprehensive and you can script it in a variety of languages including C#. I'd try the 30-day trial and how it compares to LoadRunner for your specific requirements.
I've used openSTA.
This allows a session with a web site to be recorded and then played back via a relatively simple script language.
You can easily test web services and write your own scripts.
It allows you to put scripts together in a test in any way you want and configure the number of iterations, the number of users in each iteration, the ramp up time to introduce each new user and the delay between each iteration. Tests can also be scheduled in the future.
It's open source and free.
It produces a number of reports which can be saved to a spreadsheet. We then use a pivot table to easily analyse and graph the results.
I have used WebLOAD in previous projects - it's comparable to LoadRunner, but more cost-effective.
It you must have all of the interfaces noted, then you are only going to be looking at commercial tools. The closest comparable in the commercial space is Microfocus' SilkPerformer product, but the breadth of your interfaces may push you back into the Loadrunner space, as it does have the greatest number of out of the box protocols supported of any tool out there. This is one of its distinguishing features from an enterprise sale perspective.