Alternatives for automated web application front-e

问题:

I am looking for a way to regularly test the front-end performance of my web application.

Specifically, I'm interested in the time it takes to resolve hostnames, to get responses from the servers, to download/render/execute resources, etc. I also want to measure these things when interacting with the application, not just when loading the page.

I could use Firebug or Chrome Developer Tools while interacting with my web application manually, but I want this front-end performance testing to be a part of my continuous integration process. I want to have nightly front-end performance test results, so that it becomes easy to catch performance regressions. That is why one of my preliminary conclusions is that I want to leverage my existing functional Selenium/WebDriver test suite (or a subset of the test suite).

So assuming I'm using Selenium to interact with my web application, what I really need is a tool to measure the front-end performance in the browser.

So far I've looked at three options:

1) browsermob-proxy

browsermob-proxy is an open source proxy that captures performance data using the HAR format. It provides an HTTP REST API in addition to Java and NodeJS bindings. Because it's a proxy which analyzes HTTP traffic, it doesn't give me any information about what is happening in the browser (e.g. rendering and so on), which is what I am most interested in.

2) WebPagetest

WebPagetest is also open source software, and available as a free cloud service at webpagetest.org. It is possible (and not too hard) to set up a private instance of this service. I give it a URL and I get a performance report of the loading of that page. However, it doesn't allow me to monitor the application while I am interacting with it - which is vital - especially in a single page web application with a lot of stuff going on on the client-side - like mine.

3) Compuware APM AJAX Edition (formerly DynaTrace AJAX Edition)

Compuware APM AJAX Edition is free (as in gratis) proprietary software that uses browser plugins to capture information from the browser while browsing, meaning it can automatically gather data during a Selenium test suite execution.

It's pretty easy to automate it and integrate the execution of it into a continuous integration process, but the reporting leaves a lot to be desired. Test results are grouped into "sessions" which is the period from the browser window is opened until it is closed. There is no functionality for comparing test results over time, which makes it hard to track progress and catch regressions.

Compuware APM AJAX Edition is the best alternative I've seen so far - but there has to be something better out there - right?

回答1:

How about using YSlow and/or Google Page Speed with Selenium.

1. Create separate firefox profiles - one for Google Page Speed and one for YSlow.
2. Install extensions in respective profiles
3. Install Firebug for both profiles and Page Speed and YSlow in their respective profiles (extensions are installed and configured separately for each profile).
4. Using Selenium load the custom profile

Read more here and here

Above links give an idea. Hope this helps.

回答2:

Compuware AJAX edition does allow for comparisons, long term, analysis, and CI integration for a view of builds and features over time. There is a license associated with that functionality as well as linking the Browser side PurePaths all the way back to SQL..

http://apmblog.compuware.com/2012/09/11/top-performance-mistakes-when-moving-from-test-to-production-supersized-content-2/

回答3:

My feeling on this subject is the best approach is to inject javascript into the page and have that report back metrics on the user experience. There are various approaches to this but they basically resolve down to either rolling your own code (more work but ultimately better results) or using a third party offering. New Relic have a pretty nice solution.

The problem with this approach is you still need something to exercise the browser in order for the javascript to get fired - it is hard to automate. That's the gotcha but other than that you have a reliable, consistent, maintenance-free method for recording and logging the user experience.

回答4:

I've been working on a service that runs WebPagetest in the background and allows you to track front-end performance over time.

http://speedcurve.com

You can track and dig into waterfalls of each test to understand some of what you're after. To capture interaction performance, WebPagetest has also recently added user timings so you can add simple JS events to your page and have them show up in WebPagetest and SpeedCurve.

http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user-experience.html

If it's a single page app then Caliper is new and looks really promising.

http://caliper.io/

回答5:

I was looking for how to implement front end performance testing recently. I haven't implemented it yet, but I plan on using yslow with phantomjs http://yslow.org/phantomjs/#screenshots

回答6:

To start with - i am using browsermob proxy for response times and codes, but indeed it does not give you anything like JS execution time. You already mentioned DynaTrace so I will not discuss this. What else is there - I have not used myself, but i have heard people using httpWatch. It integrates very vell with C# and is said to be capable of analyzing the performance of ajax calls.

Also, don't forget, that you can inject javascript with webdriver, so you can try to make use of boomerang or something like this.

回答7:

http://html5testing.co uses phantom.js and casper.js together. They custom build a test that interacts with your app across any browser using webkit. You get reports with screenshots, can run the tests whenever you want, it does latency monitoring, and you can see all the previous tests you ran.

It's not free, but it's certainly a lot cheaper than all the time you could be putting into testing with a patchwork of stuff.

回答8:

Apache JMeter

Description: Open source load testing tool: It is a Java platform application. It is mainly considered as a performance testing tool and it can also be integrated with the test plan. In addition to the load test plan, you can also create a functional test plan. This tool has the capacity to be loaded into a server or network so as to check on its performance and analyze its working under different conditions. Initially, it was introduced for testing the web applications, but later its scope had widened. It is of great use in testing the functional performance of the resources such as Servlets, Perl Scripts and JAVA objects. Need JVM 1.4 or higher to run.

Apache JMeter System Requirements: It works under Unix and Windows OS
Download link: Apache JMeter download

NeoLoad

Description: Load and performance testing software: This is a tool used for measuring and analyzing the performance of the website. The performance and the end result can be evaluated by using this tool and any further steps can be taken. This helps you in improving and optimizing the performance of your web application. This tool analysis the performance of the web application by increasing the traffic to the website and the performance under heavy load can be determined. You can get to know the capacity of the application and the amount of users it can handle at the same time. This tool was developed by a French company named as Netosys and it was written in JAVA. It is available in two different languages; English and French.

NeoLoad System Requirements: This tool is compatible on operating systems like Microsoft windows, Linux and Solaris.
Download link: NeoLoad download

LoadRunner

Description: This is a HP product which can be used as a performance testing tool. This can be bought as a HP product from its HP software division. Also, it is very much useful in understanding and determining the performance and outcome of the system when there is actual load. One of the key attractive features of this testing tool is that, it can create and handle thousands of users at the same time. This tool enables you to gather all the required information with respect to the performance and also based on the infrastructure. The LoadRunner comprises of different tools; namely, Virtual User Generator, Controller, Load Generator and Analysis.

LoadRunner System Requirements: Microsoft Windows and Linux are the favourable OS for this measuring tool.
Download link: LoadRunner download

LoadUI

Description: Open Source Stress Testing Tool: Load UI is yet another open source and load testing software used for measuring the performance of the web applications. This tool works effectively when it is integrated with the functional testing tool soapUI. LoadUI is the most flexible and interactive testing tools. This allows you to create, configure and update your tests while the application is being tested. It also gives a visual Aid for the user with a drag and drop experience. This is not a static performance tool. The advanced analysis and report generating features allows you to examine the actual performance by pumping in new data even while the application is being tested. You need not bother to restart the LoadUI each and every time you modify or change the application. It automatically gets updated in the interface.

System Requirements: Cross platform. Price: This tool being a open source application, it is available for free and everyone can have the easy access to its full source code.
Download link: LoadUI download

WebLOAD

Description: Load testing and stress testing tool for web application: To find out the bottlenecks of the website, it is necessary to examine the pros and cons. There are many performance testing tools available for measuring the performance of the certain web application. WebLoad is one such tool used for load testing and stress testing. This tool can be used for Load testing any internet applications such as Ajax, Adobe Flex, Oracle Forms and much more. Through this tool, you have the ability to measure the working performance and also its response to the users. This was developed by the Radview Software for the testing of Web applications. This tool was a famous and award fetching software of the Radview Software. This tool is widely used in the environment where there is a high demand for maximum Load testing. This tool gives you a clear information on the functionality and the actual capacity of the web applications.

WebLOAD System Requirements: Cross Platform.
Download link: WebLOAD download

WAPT

Description: Performance testing tool for web sites and intranet applications: WAPT refers to the Web Application Performance tool. These are scales or analyzing tools for measuring the performance and output of any web application or web related interfaces. These tools help us to measure the performance of any web services, web applications or for any other web interfaces. With this tool you have the advantage of testing the web application performances in various different environment and different load conditions. WAPT provides detailed information about the virtual users and its output to its users during the load testing. This is considered to be the best cost effective tool for analyzing the performance of the web services. The WAPT tools can tests the web application on its compatibility with browser and operating system. It is also used for testing the compatibility with the windows application in certain cases.

WAPT System Requirement: Windows OS is required for this testing tool.
Download link: WAPT download

Loadster

Description: Loadster is a desktop based advanced HTTP load testing tool. The web browser can be used to record the scripts which is easy to use and record. Using the GUI you can modify the basic script with dynamic variables to validate response. With control over network bandwidth you can simulate large virtual user base for your application stress tests. After test is executed HTML report is generated for analysis. This tool is best to identify performance bottlenecks in your application.

Loadster System Requirements: Windows 7/Vista/XP
Download link:Loadster download

LoadImpact

Description: LoadImpact is a load testing tool which is mainly used in the cloud-based services. This also helps in website optimization and improvising the working of any web application. This tools generates traffic to the website by simulating users so as to find the stress and maximum load it can work. This LoadImpact comprises of two main parts; the load testing tool and the page analyzer. The load testing can be divided into three types such as Fixed, Ramp up and Timeout. The page analyzer works similar to a browser and it gives information regarding the working and statistics of the website. The fame of developing this load testing tool belongs to Gatorhole AB. This is a freemium service which means that, it can be acquired for free and also available for premium price. But, you have the advantage of many options and features when you buy them for premium price.

System Requirement: This works well on Windows OS and Linux.
Download link: LoadImpact download

Rational Performance Tester

Description: The Rational performance tester is an automated performance testing tool which can be used for a web application or a server based application where there is a process of input and output is involved. This tool creates a demo of the original transaction process between the user and the web service. By the end of it all the statistical information are gathered and they are analyzed to increase the efficiency. Any leakage in the website or the server can be identified and rectified immediately with the help of this tool. This tool can be the best option in building a effective and error free cloud computing service. This Rational Performance tester was developed by IBM (Rational software division). They have come up with many versions of this automated testing tool.

Rational Performance Tester System Requirement: Microsoft Windows and Linux AIX good enough for this performance testing tool.
Download link: Rational Performance Tester download

Testing Anywhere

Description: Test Anywhere is a automated testing tool which can be employed for testing the performance of any web sites, web applications or any other objects. Many developers and testers make use if this tool to find out any bottlenecks in their web application and rectify them accordingly. It is a powerful tool which can test any application automatically. This testing tool comes along with a built in editor which allows the users to edit the testing criteria according to their needs. The testing anywhere tool involves 5 simple steps to create a test. They are object recorder, advanced web recorder, SMART test recorder, Image recognition and Editor with 385+ comments. Originally, this testing software was developed by San Jose based Automation Anywhere Inc. Today, there are more than 25000 users for this product.

Testing Anywhere System Requirement: This tool is compatible with all versions of Windows OS.
Download link: Testing Anywhere download

OpenSTA

Description: Open source HTTP performance test tools: Open STA stands for Open System Testing Architecture. This is a GUI based performance tool used by application developers for load testing and analyzing. This is believed to be a complex tool among the all other performance testing tools. It has proven capability in the past and the current toolset is capable of performing the heavy load test and analyses for the scripted HTTP and HTTPS. Here, the testing is carried out by using the recordings and simple scripts. To carry out the test successfully, results and other statistics are taken through various test runs. These data and results can be later exported to software for creating reports. This is a free testing tool and it is distributed under GNU GPL and it will remain free forever. This tool was originally developed by Cyrano, which was later taken over by Quotium.

OpenSTA System Requirement: OpenSTA runs only on Windows operating system.
Download link: OpenSTA download

QEngine (ManageEngine)

Description: QEngine (ManageEngine) is a most common and easy-to-use automated testing tool helping in performance testing and load testing of your web applications. Many developers find it to be the most simple and easy tool to use for finding out any leakage in their web services or websites. The key important feature of this testing tool is its ability to perform remote testing of web services from any geographical location. Other than that, QEngine (ManageEngine) also offers carious other testing options such as functionality testing, compatibility testing, stress testing , load testing and regression testing. This automated testing tool has the capacity to generate and simulate lot if users so that the performance can be well analyzed in the during the maximum load. This is a free software available for the users online.

QEngine System Requirement: This tool works with the Microsoft Windows and Linux.
Download link: QEngine download

Loadstorm

Description: Cloud load testing for web applications: Loadstorm is the cheapest available performance and load testing tool. Here, you have the option of creating your own test plans, testing criteria and testing scenario. You can generate up to 50000 concurrent users by generating traffic to your website and then carry out the testing. Through this tool, you can bring an end to all the expensive performance testing tools. The cloud infrastructure is used in this tool, which enables you to send huge amount of requests per second. There are thousands of servers available around the world for this software. They are proudly known as the lowest cloud load testing tool. There is no need of any scripting knowledge for using this tool. You will be provided with many graphs and reports which measures the performance in various metrics such as error rates, average response time and number of users. This tool is available for free, but the premium account comes with added features.

Loadstorm System Requirement: Windows OS.
Download link: Loadstorm download

Soasta cloudtest

Description: SOASTA CloudTest is a performance testing tool for the cloud computers. The users or the developers can use the cloud platform as their virtual testing lab. The developers can carry out their per4formance or load testing in the cloud platform in the cost effective way through this CloudTest tool. This CloudTest has the capacity to enable number of users to use the website at the same time. It also increases the traffic of the website to know the actual performance under stress and heavy load. The credit of developing this software goes to an American Technology company, SOASTA Inc. They provide many services for testing the websites and other web applications and now they also help in testing the mobile applications. They are not free services, the price differs according to the number of load injector machines required per hour by you. The trial version with power of 100 concurrent users is available for free.

SOASTA CloudTest System Requirement: It runs on Windows, Linux and Mac OS.
Download link: SOASTA CloudTest download

HTTPerf

Description: Httperf is a high performance testing tool for measuring and analyzing the performance of any web services and web applications. This is mainly used to the test the HTTP servers and its performance. The main objective of this testing tool would be to count the number of responses generated from this particular server. This generates the HTTP GET requests from the server which helps in summarizing the overall performance of the server. Through this tool, you will be able to conclude the rate at which the response is sent from each server and thereby the efficiency can be calculated. The ability to sustain the server overload, support the HTTP/1.1 protocol and compatibility with new workload are the three key features of this performance testing tool. This was originally developed by David Mosberger and many others at HP. This is a Hewlett Packard product.

Httperf System Requirement: Windows and Linux.
Download link: Httperf

Hope this Helps..

回答9:

Have you looked at what your server will capture for you on your requests? Every web server has the ability to capture and record time in seconds (some in microseconds) for every single request made by your front end. You can pull the data on the backend and visualize it with any number of tools. Combine it with a free GEoIP database and you can even look at the data by country and city. Pay a little more for GEoIP data and you can look at mobile on mobile networks vs mobile on home or corporate networks to observe key differences in behavior by network connection (which you cannot control).

As to the obsession with front end performance, leave that to your functional testers. Optimization can be handled on a single PC with tools in the browser combined with something like Charles proxy to vary caching options and speeds. Once your application gets into the wild you are going to have very little control over the front end performance anyway. Rendering time will vary from PC to PC depending upon browser type and load on the individual PC. Have you ever seen what happens to a browser with just a few toolbars loaded and tracking all user actions? Even the fastest of sites with the most optimized pages slow to a crawl.

Do your job and optimize the page design, number of components, caching models, etc... and then for performance and scalability turn your attention to the server and away from the client.

回答10:

Since you have tagged ASP.Net, you might want to look at ANTS performance profiler- http://www.red-gate.com/products/dotnet-development/

回答11:

Front end Performance testing with AgileLoad

AgileLoad Script Editor capture and analyze all the requests made between the user and the application to build a test scenario.

The Replay function validate the script generated by replaying and comparing each request with the initial scenario.

The Replay tab contains for each page of your test scenario a graphical bar chart which shows the time spent for the primary request (in blue) and the overall response time (in orange). It also gives you details of all the resources loaded, the time spent for each resources, the detailed HTTP response (Body, Client HTTP Header, Server HTTP Header) associated with each HTTP request.

Agileload front end performance

For each page, you also retrieve, the HTML view, the source view, structure view, HTML tree view, HTML server headers view

Http data pane

A web page performance summary gives you details on DNS time, tcp connection time, SSL handshake time, the send time, server time, receive time, response http status, response size in bytes ecc..

Web performance details

This page speed waterfall highlights problematic resources to be optimized for each pages of your test scenario.

Web applications are becoming richer and richer in design and content and at the same time good user experience has become the most desirable attribute. There is misconception that application desired response time can be achieved by optimizing server side only. Research has showed that 80-90% of page load time is spend on client side and 40-50% page load time can be optimized by just focusing on front-end of the application as compared to 20% of server side optimization.

Also front-end performance optimization is not the same than back-end optimization. One is about improving the performance from a single user point of view, the other is focused on improving the performance from a multiple user point of view when resources are used concurrently.

Both tasks are complementary and can be tested with Agileload.

References:Agile Load

回答12:

It's worth having a look at Chrome Telemetry (http://www.chromium.org/developers/telemetry)

It's obviously Chrome only but it's what Adobe use to test the performance of TopCoat