Automated Unit Testing with JavaScript

Question

I'm trying to incorporate some JavaScript unit testing into my automated build process. Currently JSUnit works well with JUnit, but it seems to be abandonware and lacks good support for AJAX, debugging, and timeouts.

Has anyone had any luck automating (with ANT) a unit testing library such as YUI test, JQuery's QUnit, or jQUnit (http://code.google.com/p/jqunit/)?

Note: I use a custom built AJAX library, so the problem with Dojo's DOH is that it requires you to use their own AJAX function calls and event handlers to work with any AJAX unit testing.

Answer 1

Another JS testing framework that can be run with Ant is CrossCheck. There's an example of running CrossCheck via Ant in the build file for the project.

CrossCheck attempts, with limited success, to emulate a browser, including mock-style implementations of XMLHttpRequest and timeout/interval.

It does not currently handle loading javascript from a web page, though. You have to specify the javascript files that you want to load and test. If you keep all of your JS separated from your HTML, it might work for you.

Answer 2

I am in agreement that jsunit is kind of dying on the vine. We just finished up replacing it with YUI Test.

Similar to the example using qUnit, we are running the tests using Selenium. We are running this test independently from our other selenium tests simply because it does not have the dependencies that the normal UI regression tests have (e.g. deploying the app to a server).

To start out, we have a base javascript file that is included in all of our test html files. This handles setting up the YUI instance, the test runner, the YUI.Test.Suite object as well as the Test.Case. It has a methods that can be accessed via Selenium to run the test suite, check to see if the test runner is still running (results are not available until after it's done), and get the test results (we chose JSON format)

var yui_instance; //the YUI instance
var runner;  //The YAHOO.Test.Runner
var Assert; //an instance of YAHOO.Test.Assert to save coding
var testSuite; //The YAHOO.Test.Suite that will get run.

/**
 * Sets the required value for the name property on the given template, creates
 * and returns a new YUI Test.Case object.
 * 
 * @param template the template object containing all of the tests
 */
function setupTestCase(template) {
    template.name = "jsTestCase";
    var test_case = new yui_instance.Test.Case(template);
    return test_case;
}

/**
 * Sets up the test suite with a single test case using the given 
 * template.
 * 
 * @param template the template object containing all of the tests
 */
function setupTestSuite(template) {
    var test_case = setupTestCase(template);
    testSuite = new yui_instance.Test.Suite("Bond JS Test Suite");
    testSuite.add(test_case);
}

/**
 * Runs the YAHOO.Test.Suite
 */
function runTestSuite() {
    runner = yui_instance.Test.Runner;
    Assert = yui_instance.Assert;

    runner.clear();
    runner.add(testSuite);
    runner.run();
}

/**
 * Used to see if the YAHOO.Test.Runner is still running.  The
 * test results are not available until it is done running.
 */
function isRunning() {
    return runner.isRunning();
}

/**
 * Gets the results from the YAHOO.Test.Runner
 */
function getTestResults() {
    return runner.getResults(yui_instance.Test.Format.JSON);
}

As for the selenium side of things, we used a parameterized test. We run our tests in both IE and FireFox in the data method, parsing the test results into a list of Object arrays with each array containing the browser name, the test file name, the test name, the result (pass, fail or ignore) and the message.

The actual test just asserts the test result. If it is not equal to "pass" then it fails the test with the message returned from the YUI Test result.

    @Parameters
public static List<Object[]> data() throws Exception {
    yui_test_codebase = "file:///c://myapppath/yui/tests";

    List<Object[]> testResults = new ArrayList<Object[]>();

    pageNames = new ArrayList<String>();
    pageNames.add("yuiTest1.html");
    pageNames.add("yuiTest2.html");

    testResults.addAll(runJSTestsInBrowser(IE_NOPROXY));
    testResults.addAll(runJSTestsInBrowser(FIREFOX));
    return testResults;
}

/**
 * Creates a selenium instance for the given browser, and runs each
 * YUI Test page.
 * 
 * @param aBrowser
 * @return
 */
private static List<Object[]> runJSTestsInBrowser(Browser aBrowser) {
    String yui_test_codebase = "file:///c://myapppath/yui/tests/";
    String browser_bot = "this.browserbot.getCurrentWindow()"
    List<Object[]> testResults = new ArrayList<Object[]>();
    selenium = new DefaultSelenium(APPLICATION_SERVER, REMOTE_CONTROL_PORT, aBrowser.getCommand(), yui_test_codebase);
    try {
        selenium.start();

        /*
         * Run the test here
         */
        for (String page_name : pageNames) {
            selenium.open(yui_test_codebase + page_name);
            //Wait for the YAHOO instance to be available
            selenium.waitForCondition(browser_bot + ".yui_instance != undefined", "10000");
            selenium.getEval("dom=runYUITestSuite(" + browser_bot + ")");

            //Output from the tests is not available until 
            //the YAHOO.Test.Runner is done running the suite
            selenium.waitForCondition("!" + browser_bot + ".isRunning()", "10000");
            String output = selenium.getEval("dom=getYUITestResults(" + browser_bot + ")");

            JSONObject results = JSONObject.fromObject(output);
            JSONObject test_case = results.getJSONObject("jsTestCase");
            JSONArray testCasePropertyNames = test_case.names();
            Iterator itr = testCasePropertyNames.iterator();

            /*
             * From the output, build an array with the following:
             *  Test file
             *  Test name
             *  status (result)
             *  message
             */
            while(itr.hasNext()) {
                String name = (String)itr.next();
                if(name.startsWith("test")) {
                    JSONObject testResult = test_case.getJSONObject(name);
                    String test_name = testResult.getString("name");
                    String test_result = testResult.getString("result");
                    String test_message = testResult.getString("message");
                    Object[] testResultObject = {aBrowser.getCommand(), page_name, test_name, test_result, test_message};
                    testResults.add(testResultObject);
                }
            }

        }
    } finally {
        //if an exception is thrown, this will guarantee that the selenium instance
        //is shut down properly
        selenium.stop();
        selenium = null;
    }
    return testResults;
}
/**
 * Inspects each test result and fails if the testResult was not "pass"
 */
@Test
public void inspectTestResults() {
    if(!this.testResult.equalsIgnoreCase("pass")) {
        fail(String.format(MESSAGE_FORMAT, this.browser, this.pageName, this.testName, this.message));
    }
}

I hope this is helpful.

Answer 3

I looked on your question date and back then there were a few good JS testing lib/framework. Today you can find much more and in different focus like TDD, BDD, Assetion and with/without runners support.

There are Many players in this game like Mocha, Chai, QUnit, Jasmine, etc... You can find some more information in this blog about JS/Mobile/web testing...

Answer 4

The project I'm working on uses Js-Test-Driver hosting Jasmine on Chrome 10 with Jasmine-JSTD-Adapter including making use of Code Coverage tests included in JS-Test-Driver. While there are some problems each time we change or update browsers on the CI environment the jasmine tests are running pretty smoothly with only minor issues with ansynchronous tests, but as far as I'm aware these can be worked around using Jasmine Clock but I haven't had a chance to patch them yet.

Answer 5

This is a good evaluation of several testing tools.

JavaScript unit test tools for TDD

I personally prefer https://code.google.com/p/js-test-driver/

Answer 6

I'm just about to start doing Javascript TDD on a new project I am working on. My current plan is to use qunit to do the unit testing. While developing the tests can be run by simply refreshing the test page in a browser.

For continuous integration (and ensuring the tests run in all browsers), I will use Selenium to automatically load the test harness in each browser, and read the result. These tests will be run on every checkin to source control.

I am also going to use JSCoverage to get code coverage analysis of the tests. This will also be automated with Selenium.

I'm currently in the middle of setting this up. I'll update this answer with more exact details once I have the setup hammered out.

---

Testing tools:

• qunit
• JSCoverage
• Selenium