How to programmatically skip a test in mocha?

Question

I have a code where certain tests will always fail in CI environment. I would like to disable them based on an environment condition.

How to programmatically skip a test in mocha during the runtime execution?

Answer 1

We can write a nice clean wrapper function to conditionally run tests as follows:

function ifConditionIt(title, test) {
  // Define your condition here
  return condition ? it(title, test) : it.skip(title, test);
}

This can then be required and used in your tests as follows:

ifConditionIt('Should be an awesome test', (done) => {
  // Test things
  done();
});

Answer 2

Please don't. A test that doesn't work consistently across environments should be acknowledged as such by your build infrastructure. And it can be very disorienting when the CI builds have a different number of tests run than local.

Also it screws up repeatability. If different tests run on the server and local I can have tests failing in dev and passing in CI or vice versa. There's no forcing function and I have no way to quickly and accurately correct a failed build.

If you must turn off tests between environments, instead of conditionally running tests, tag your tests and use a filter to eliminate tests that don't work in certain build targets. That way everybody knows what's going on and it tempers their expectations. It also lets everybody know that there's inconsistency in the test framework, and someone might have a solution that gets them running properly again. If you just mute the test they might not even know there's a problem.

Answer 3

It depends how you want to programmatically skip the test. If the conditions for skipping can be determined before any test code is run, then you can just call it or it.skip as needed, based on a condition. For instance, this will skip some tests if the environment variable ONE is set to any value:

var conditions = {
    "condition one": process.env["ONE"] !== undefined
    // There could be more conditions in this table...
};

describe("conditions that can be determined ahead of time", function () {
    function skip_if(condition, name, callback) {
        var fn = conditions[condition] ? it.skip: it;
        fn(name, callback);
    };

    skip_if("condition one", "test one", function () {
        throw new Error("skipped!");
    });

    // async.
    skip_if("condition one", "test one (async)", function (done) {
        throw new Error("skipped!");
    });

    skip_if("condition two", "test two", function () {
        console.log("test two!");
    });

});

If the conditions you want to check can only be determined at test time, it is a bit more complicated. If you do not want to access anything not strictly speaking part of the testing API, then you could do this:

describe("conditions that can be determined at test time", function () {
    var conditions = {};
    function skip_if(condition, name, callback) {
        if (callback.length) {
            it(name, function (done) {
                if (conditions[condition])
                    done();
                else
                    callback(done);
            });
        }
        else {
            it(name, function () {
                if (conditions[condition])
                    return;
                callback();
            });
        }
    };

    before(function () {
        conditions["condition one"] = true;
    });

    skip_if("condition one", "test one", function () {
        throw new Error("skipped!");
    });

    // async.
    skip_if("condition one", "test one (async)", function (done) {
        throw new Error("skipped!");
    });

    skip_if("condition two", "test two", function () {
        console.log("test two!");
    });

});

Whereas my first example was marking the tests as formally skipped (aka "pending"), the method I've just shown will just avoid performing the actual test but the tests won't be marked as formally skipped. They will be marked as passed. If you absolutely want to have them skipped I don't know of any way short of accessing parts that are not properly speaking part of the testing API:

describe("conditions that can be determined at test time", function () {
    var condition_to_test = {}; // A map from condition names to tests.
    function skip_if(condition, name, callback) {
        var test = it(name, callback);
        if (!condition_to_test[condition])
            condition_to_test[condition] = [];
        condition_to_test[condition].push(test);
    };

    before(function () {
        condition_to_test["condition one"].forEach(function (test) {
            test.pending = true; // Skip the test by marking it pending!
        });
    });

    skip_if("condition one", "test one", function () {
        throw new Error("skipped!");
    });

    // async.
    skip_if("condition one", "test one (async)", function (done) {
        throw new Error("skipped!");
    });

    skip_if("condition two", "test two", function () {
        console.log("test two!");
    });

});

Answer 4

I am not sure if this qualifies as „programmatic skipping“, but in order to selectively skip some specific tests for our CI environment, I use Mocha's tagging feature (https://github.com/mochajs/mocha/wiki/Tagging). In describe() or it() messages, you can add a tag like @no-ci. To exclude those tests, you could define a specific "ci target" in your package.json and use --grep and --invert parameters like:

"scripts": {
  "test": "mocha",
  "test-ci" : "mocha --reporter mocha-junit-reporter --grep @no-ci --invert"
}

Answer 5

I use runtime skipping from Mocha for the same scenario as you're describing. It is the copy paste from the docs:

it('should only test in the correct environment', function() {
  if (/* check test environment */) {
    // make assertions
  } else {
    this.skip();
  }
});

As you can see, it skips the test based on environment. My own condition is if(process.env.NODE_ENV === 'continuous-integration').

Answer 6

This is not really using mocha's features, rather tweaking it to get the behaviour I wanted.

I wanted to skip any subsequent 'it's' in my protractor mocha tests and one 'it' failed. This was because once one step of a journey test failed it was almost certain the rest would fail, and may take a long time and hog the build server if they are using browser waits for elements to appear on a page etc.

When just running standard mocha tests (not protractor) this can be achieved with global beforeEach and afterEach hooks by attaching a 'skipSubsequent' flag to the test's parent (describe) like this:

    beforeEach(function() {
      if(this.currentTest.parent.skipSubsequent) {
            this.skip();
      }
    }); 


    afterEach(function() {
      if (this.currentTest.state === 'failed') {
        this.currentTest.parent.skipSubsequent = 'true'
      }
    })

When attempting this with protractor and mocha it the scope of 'this' has changed and the code above does not work. You end up with an error message like 'error calling done()' and protractor halts.

Instead I ended up with the code below. Not the prettiest, but it ends up replacing the implementation of remaining test functions with a this.skip(). This will probably stop working if/when the internals of mocha change with later versions.

It was figured out through some trial and error by debugging and inspecting mocha's internals...helps make browser test suites complete sooner when the tests fail though.

beforeEach(function() {

    var parentSpec = this.currentTest.parent;

    if (!parentSpec.testcount) {
        parentSpec.testCount = parentSpec.tests.length;
        parentSpec.currentTestIndex = 0;
    } else {
        parentSpec.currentTestIndex = parentSpec.currentTestIndex + 1;
    }

    if (parentSpec.skipSubsequent) {

        parentSpec.skipSubsequent = false;
        var length = parentSpec.tests.length;
        var currentIndex = parentSpec.currentTestIndex;

        for (var i = currentIndex + 1; i < length; i++) {
            parentSpec.tests[i].fn = function() {
                this.skip();
            };
        }
    }
});


afterEach(function() {
    if (this.currentTest.state === 'failed') {
        this.currentTest.parent.skipSubsequent = 'true'
    }
});