How to synch JavaScript callbacks?

Question

I've been developing in JavaScript for quite some time but net yet a cowboy developer, as one of the many things that always haunts me is synching JavaScript's callbacks.

I will describe a generic scenario when this concern will be raised: I have a bunch of operations to perform multiple times by a for loop, and each of the operations has a callback. After the for loop, I need to perform another operation but this operation can only execute successfully if all the callbacks from the for loop are done.

Code Example:

for ... in ... {
  myFunc1(callback); // callbacks are executed asynchly
}

myFunc2(); // can only execute properly if all the myFunc1 callbacks are done

Suggested Solution:

Initiate a counter at the beginning of the loop holding the length of the loop, and each callback decrements that counter. When the counter hits 0, execute myFunc2. This is essentially to let the callbacks know if it's the last callback in sequence and if it is, call myFunc2 when it's done.

Problems:

1. A counter is needed for every such sequence in your code, and having meaningless counters everywhere is not a good practice.
2. If you recall how thread conflicts in classical synchronization problem, when multiple threads are all calling var-- on the same var, undesirable outcomes would occur. Does the same happen in JavaScript?

Ultimate Question:

Is there a better solution?

Answer 1

You can achieve this by using a jQuery deferred object.

var deferred = $.Deferred();
var success = function () {
    // resolve the deferred with your object as the data
    deferred.resolve({
        result:...;
    });
};

Answer 2

The good news is that JavaScript is single threaded; this means that solutions will generally work well with "shared" variables, i.e. no mutex locks are required.

If you want to serialize asynch tasks, followed by a completion callback you could use this helper function:

function serializeTasks(arr, fn, done)
{
    var current = 0;

    fn(function iterate() {
        if (++current < arr.length) {
            fn(iterate, arr[current]);
        } else {
            done();
        }
    }, arr[current]);
}

The first argument is the array of values that needs to be passed in each pass, the second argument is a loop callback (explained below) and the last argument is the completion callback function.

This is the loop callback function:

function loopFn(nextTask, value) {
    myFunc1(value, nextTask);
}

The first argument that's passed is a function that will execute the next task, it's meant to be passed to your asynch function. The second argument is the current entry of your array of values.

Let's assume the asynch task looks like this:

function myFunc1(value, callback)
{
  console.log(value);
  callback();
}

It prints the value and afterwards it invokes the callback; simple.

Then, to set the whole thing in motion:

serializeTasks([1,2, 3], loopFn, function() {
    console.log('done');
});

Demo

To parallelize them, you need a different function:

function parallelizeTasks(arr, fn, done)
{
    var total = arr.length,
    doneTask = function() {
      if (--total === 0) {
        done();
      }
    };

    arr.forEach(function(value) {
      fn(doneTask, value);
    });
}

And your loop function will be this (only parameter name changes):

function loopFn(doneTask, value) {
    myFunc1(value, doneTask);
}

Demo

Answer 3

The second problem is not really a problem as long as every one of those is in a separate function and the variable is declared correctly (with var); local variables in functions do not interfere with each other.

The first problem is a bit more of a problem. Other people have gotten annoyed, too, and ended up making libraries to wrap that sort of pattern for you. I like async. With it, your code might look like this:

async.each(someArray, myFunc1, myFunc2);

It offers a lot of other asynchronous building blocks, too. I'd recommend taking a look at it if you're doing lots of asynchronous stuff.

Answer 4

single thread is not always guaranteed. do not take it wrong.

Case 1: For example, if we have 2 functions as follows.

var count=0;
function1(){
  alert("this thread will be suspended, count:"+count);
}
function2(){
  //anything
  count++;
  dump(count+"\n");
}

then before function1 returns, function2 will also be called, if 1 thread is guaranteed, then function2 will not be called before function1 returns. You can try this. and you will find out count is going up while you are being alerted.

Case 2: with Firefox, chrome code, before 1 function returns (no alert inside), another function can also be called.

So a mutex lock is indeed needed.

Answer 5

There are many, many ways to achieve this, I hope these suggestions help!

First, I would transform the callback into a promise! Here is one way to do that:

function aPromise(arg) {
    return new Promise((resolve, reject) => {
        aCallback(arg, (err, result) => {
            if(err) reject(err);
            else resolve(result);
        });
    })
}

Next, use reduce to process the elements of an array one by one!

const arrayOfArg = ["one", "two", "three"];
const promise = arrayOfArg.reduce(
    (promise, arg) => promise.then(() => aPromise(arg)), // after the previous promise, return the result of the aPromise function as the next promise
    Promise.resolve(null) // initial resolved promise
    );
promise.then(() => {
    // carry on
});

If you want to process all elements of an array at the same time, use map an Promise.all!

const arrayOfArg = ["one", "two", "three"];
const promise = Promise.all(arrayOfArg.map(
    arg => aPromise(arg)
));
promise.then(() => {
    // carry on
});

If you are able to use async / await then you could just simply do this:

const arrayOfArg = ["one", "two", "three"];
for(let arg of arrayOfArg) {
    await aPromise(arg); // wow
}

// carry on

You might even use my very cool synchronize-async library like this:

const arrayOfArg = ["one", "two", "three"];
const context = {}; // can be any kind of object, this is the threadish context

for(let arg of arrayOfArg) {
    synchronizeCall(aPromise, arg); // synchronize the calls in the given context
}

join(context).then(() => { // join will resolve when all calls in the context are finshed
    // carry on
});

And last but not least, use the fine async library if you really don't want to use promises.

const arrayOfArg = ["one", "two", "three"];
async.each(arrayOfArg, aCallback, err => {
    if(err) throw err; // handle the error!
    // carry on
});

Answer 6

With this helper function:

function afterAll(callback,what) {
  what.counter = (what.counter || 0) + 1;
  return function() {
    callback(); 
    if(--what.counter == 0) 
      what();
  };
}

your loop will look like this:

function whenAllDone() { ... }
for (... in ...) {
  myFunc1(afterAll(callback,whenAllDone)); 
}

here afterAll creates proxy function for the callback, it also decrements the counter. And calls whenAllDone function when all callbacks are complete.