Selenium - Wait for network traffic

Question

We're using Selenium with the Java API and some Javascript user extensions. We use a lot of AJAX calls in our app. A lot of our tests fail randomly because sometimes the AJAX calls finish slower than other times so the page isn't fully loaded. We fix that by waiting for specific elements or Thread.sleep. I was trying to find a way to instead just wait for the network traffic to finish. So that we could do this:

selenium.click("some JS button");
selenium.waitForNetwork();
assertTrue(something);

That way we can get rid of the thread sleep and have tests pass faster when the server responds faster and not have so many tests fail due to timing issues.

I haven't been able to find a way to do this searching Google. Does anyone have any ideas how we can accomplish this? (Preferably either through Javascript or the Java API but all suggestions are welcome).

Note: the other variations of "waitFor" are not what I'm looking for. We're already using those in clicks and other things. I'm looking for something that waits for the NETWORK TRAFFIC. Thanks for all the feedback, I'll be trying out a couple of the suggestions, but I'm still open to other ideas.

Thanks.

Answer 1

Update: I use the same method (wrapRequest) in the same way with Selenium2 and it still works. The only difference is that I don't use a user extension. I just execute javascript to query the code that is loaded by the JSP.

Here's what we ended up doing. (Note, there are easier ways of getting if an Ajax call is running if you are always using only a single framework like JQuery. We had to jimmy-rig this a bit more because we have a few pages that use JQuery and a bunch that use GWT)

I created a Javascript file that every page loads only if you're testing. (I did this by including a snippet in our JSP's template: if the testing querystring param is passed in as true, set a cookie. If that cookie is set, include the javascript file) This javascript file essentially wraps the XMLHttpRequest object to keep count of all the send requests:

function wrapRequest(xmlRequest) {
    var oldSend = xmlRequest.prototype.send;
    xmlRequest.prototype.send = function() {
        var oldOnReady = this.onreadystatechange;
        oldOnReady = function() {
            oldOnReady.apply(this, arguments);
            // if call is complete, decrement counter.
        }
        // increment counter
        if(oldSend.apply)
            oldSend.apply(this, arguments);
        else if (oldOnReady.handleEvent) { //Firefox sometimes will need this
            oldOnReady.handleEvent.apply(this, arguments);
        }
    }
}

There's a little more to it than that if async == false, but that should be easy enough to figure out.

Then I added a function to the selenium user-extensions.js that simply looks like this:

Selenium.prototype.getIsNetworkReady = function() {
    if(this.browserbot.topWindow.isNetworkReady) { //sometimes this method is called before the page has loaded the isNetworkReady function
        return this.browserbot.topWindow.isNetworkReady();
    }
    return false;
}

Also, notice the check on topWindow: this is because you get issues when an iframe is selected.

Also, you will have to call the wrapRequest method on the XMLHttpRequest object of every iFrame that gets loaded. I do that right now using: document.addEventListener("DOMNodeInserted", onElementAdded, true); and then in onElementAdded I just grab the iframe when it's loaded and pass in the XMLHttpRequest object. but that doesn't work in IE so I'm looking for an alternative. If anyone knows of a better way to do that so it works in both IE and FF I would love to hear it.

Anyway, that's the gist of the implementation. Hope this helps anyone in the future.

Answer 2

Yes, do something like this (Pseudocode):

int i = 0;
while (element is not yet present) {
  i++;
  if (i>TRESHOLD) {
    throw an exception
  }
  Thread.sleep(200);
}
//go on

You propably want to stuff it into a function.

Answer 3

After trying all the variations described on this page, we ended up with this strategy:

Inject a special javascript that basically overrides all your relevant framework methods and keep a counter of the number of uncompleted requests, which is incremented upon start and decremented on completion. Due to the asynchronous nature of javascript it is not sufficient to keep a boolean variable, you're likely to end up with race conditions within your own logic.

If you're using visual effects, you probably want to do this for visual effects too; morphing/fading and other visuals will have the same problem.

So we basically do a regular "click" and then always wait for this counter to reach 0 again, using javascript much like the others suggest here. After we got this in place we reduced transient issues to none.

Remember that selenium does not return from "click" until all the synchronous onclick events are processed, so you have to make sure get those counters incremented as a direct consequence of onclick.

For instructions on how to override framework methods you need to consult your framework documentation; we use addMethods in prototype. It's possible to keep this overriding in a special javascript that is only added to the page when running tests.

Answer 4

We already have client side code to change the mouse cursor during an AJAX-request (using a4j:status with attributes onstart and onstop). Therefore we can simply waitFor the value of the JavaScript expression

window.document.body.style.cursor

However, I have found that this doesn't work reliably, i.e. sometimes the test proceeds too early. I am not sure about the cause, but suspect that updating the DOM is not necessarily fully done when onstop is invoked.

As for reducing unnecesssary delay, you can implement polling yourself and poll more frequently than selenium does.

Answer 5

There are few types of waitFor available in Selenium. In your test case if you are aware that a particular text would appear on page load you can use waitForText. If there are different DOM elements getting populated (via Ajax/pageload) you can sequentially add waitForText.

Answer 6

This may not be the acceptable for you, but because of the similar problems we moved to Selenium 2 (aka WebDriver). We use PageFactory with AjaxElementLocatorFactory

To make a migration easier, there's a Selenium Emulation and the plan is to completely merge Selenium 1 and WebDriver which is the main goal of Selenium 2.