Selenium WebDriver tests with JavaScript disabled

问题:

One of our internal applications (written in angularjs) has a special error box appearing if javascript is disabled in the browser (using noscript), similar to the one on stackoverflow:

I'm trying to write an automated test for it, but having difficulties.

We are using protractor, but I'm pretty sure this is not about it. Here is the protractor configuration file:

'use strict';

var helper = require('./helper.js');

exports.config = {
    seleniumAddress: 'http://localhost:4444/wd/hub',
    baseUrl: 'http://localhost:9001',

    capabilities: helper.getFirefoxProfile(),

    framework: 'jasmine',
    allScriptsTimeout: 20000,

    jasmineNodeOpts: {
        showColors: true,
        isVerbose: true,
        includeStackTrace: true
    }
};

where helper.js is:

var q = require('q');
var FirefoxProfile = require('firefox-profile');

exports.getFirefoxProfile = function() {
    var deferred = q.defer();

    var firefoxProfile = new FirefoxProfile();
    firefoxProfile.setPreference("javascript.enabled", false);
    firefoxProfile.encoded(function(encodedProfile) {
        var capabilities = {
            'browserName': 'firefox',
            'firefox_profile' : encodedProfile,
            'specs': [
                '*.spec.js'
            ]
        };
        deferred.resolve(capabilities);
    });

    return deferred.promise;
};

As you see, we are setting javascript.enabled firefox preference to false which has been proven to work if you manually open up about:config in firefox, change it to false - you would see the contents of noscript section.

But, when I run the tests, I am getting the following error:

Exception thrown org.openqa.selenium.WebDriverException: waiting for evaluate.js load failed

Here is the complete traceback.

FYI, selenium 2.44.0 and firefox 33.1.1 are used.

As far as I understand (with the help of several points raised here), disabling javascript is killing the javascript webdriver itself. Is it true? If yes, what are my options or workarounds?

---

Notes:

• in case of chrome, in the past it was possible to disable javascript via --disable-javascript command-line argument, but not anymore.

• this leads to a workaround number 0 - downgrade chrome to an old version which supported the command-line flag - this would be a not-tested plan B

• setting javascript.enabled=false firefox preference works with python selenium bindings:

  from selenium import webdriver
  
  profile = webdriver.FirefoxProfile()
  profile.set_preference('javascript.enabled', False)
  driver = webdriver.Firefox(firefox_profile=profile)
  
  driver.get('https://my_internal_url.com')
  # no errors and I can assert the error is present
  

^{I'm open to any suggestions and can provide you with any additional information.}

回答1:

Here is what actually happened.

As it turns out, after exploring the source code of protractor and selenium js webdriver, the key problem is not in the js webdriver or protractor, it was in the way my test was written.

There is a setting called ignoreSynchronization which by default is false:

  /**
   * If true, Protractor will not attempt to synchronize with the page before
   * performing actions. This can be harmful because Protractor will not wait
   * until $timeouts and $http calls have been processed, which can cause
   * tests to become flaky. This should be used only when necessary, such as
   * when a page continuously polls an API using $timeout.
   *
   * @type {boolean}
   */
  this.ignoreSynchronization = false;

And I was not setting it to true which made protractor to try synchronizing with the page and execute client side scripts, which evaluate.js is responsible for.

The solution was so simple I could not imagine - just setting ignoreSynchronization to true solved the problem:

'use strict';

require('jasmine-expect');

describe('Disabled Javascript', function () {
    beforeEach(function () {
        browser.ignoreSynchronization = true;
        browser.get('index.html');
    });

    it('should show disabled js', function () {
        var element = browser.findElement(by.tagName('noscript'));
        expect(element.getText()).toEqual('Please enable Javascript and try again.');
    });
});

Hope this would help somebody in the future.

回答2:

As far as I understand (with the help of several points raised here), disabling javascript is killing the javascript webdriver itself. Is it true?

Yes. Note that WebDriver itself runs as a Firefox extension, so its code isn't affected by you disabling JavaScript. However, the error message indicates that Selenium is attempting to run some code in the context of the webpage. For that it injects a <script> tag into the webpage and puts the contents of evaluate.js into it. Then it waits for that script to set a flag indicating that it is ready - something that never happens of course because scripts aren't running in that webpage.

If yes, what are my options or workarounds?

The important question is what Selenium is trying to run there. Your traceback is merely indicating that some code is calling executeScript() or executeAsyncScript(). I couldn't find any implied calls to either method in the Selenium codebase (at least not in any files related to Firefox web driver). Given that Selenium is open source, you should be able to debug it and add a breakpoint on org.openqa.selenium.remote.RemoteWebDriver.executeScript() and org.openqa.selenium.remote.RemoteWebDriver.executeAsyncScript() - that way you will actually see the script being executed there. Alternatively, you can also recompile RemoteWebDriver.java and make it produce debugging output.

Looking into how these methods are implemented, there might also be another way: telling Selenium that JavaScript is disabled. This should work via capabilities:

'javascriptEnabled': false,

According to documentation this capability only works on HTMLUnitDriver, yet Selenium will always consider it and from the source code specifying it for Firefox seems possible. With this capability any calls executing scripts on web pages will fail early.

回答3:

as per your stacktrace, it seems you are using ff 33.1 with selenium 2.43.1, so it wont support. Please upgrade your selenium version to 2.44 to enable tests on ff-33.

回答4:

Found an alternative solution/workaround - "NoScript Security Suite" firefox extension.

It disables javascript per-domain and, by default, it does it for all sites except listed in the whitelist. This makes the webdriver run without problems, open up a web page, then the extension disables javascript for this particular site and I see the contents of the noscript tag.

Here's the protractor config:

'use strict';

var helper = require('./disabledJavascript.helper.js');

exports.config = {
    seleniumAddress: 'http://localhost:4444/wd/hub',

    capabilities: helper.getFirefoxProfile(),
    chromeOnly: false,

    specs: [
        'disabledJavascript.spec.js'
    ],

    framework: 'jasmine',

    allScriptsTimeout: 20000,

    baseUrl: 'http://localhost:9001',

    jasmineNodeOpts: {
        showColors: true,
        isVerbose: true,
        includeStackTrace: true
    }
};

where disabledJavascript.helper.js contains:

'use strict';

var q = require('q');
var FirefoxProfile = require('firefox-profile');

exports.getFirefoxProfile = function() {
    var deferred = q.defer();
    var firefoxProfile = new FirefoxProfile();

    firefoxProfile.addExtension('./test/e2e/disabledJavascript/noscript.xpi', function () {
        firefoxProfile.encoded(function(encodedProfile) {
            var capabilities = {
                'firefox_profile' : encodedProfile,
                'browserName': 'firefox'
            };
            deferred.resolve(capabilities);
        });
    });

    return deferred.promise;
};

where noscript.xpi is a downloaded "Noscript" extension.