How do you structure sequential AWS service calls

Question

I'm coming from a java background so a bit of a newbie on Javascript conventions needed for Lambda.

I've got a lambda function which is meant to do several AWS tasks in a particular order, depending on the result of the previous task.

Given that each task reports its results asynchronously, I'm wondering if the right way make sure they all happen in the right sequence, and the results of one operation are available to the invocation of the next function.

It seems like I have to invoike each function in the callback of the prior function, but seems like that will some kind of deep nesting and wondering if that is the proper way to do this.

For example on of these functions requires a DynamoDB getItem, following by a call to SNS to get an endpoint, followed by a SNS call to send a message, followed by a DynamoDB write.

What's the right way to do that in lambda javascript, accounting for all that asynchronicity?

Answer 1

I would like to offer the following solution, which simply creates a nested function structure.

// start with the last action
var next = function() { context.succeed(); };

// for every new function, pass it the old one
next = (function(param1, param2, next) {
    return function() { serviceCall(param1, param2, next); };
})("x", "y", next);

What this does is to copy all of the variables for the function call you want to make, then nests them inside the previous call. You'll want to schedule your events backwards. This is really just the same as making a pyramid of callbacks, but works when you don't know ahead of time the structure or quantity of function calls. You have to wrap the function in a closure so that the correct value is copied over.

In this way I am able to sequence AWS service calls such that they go 1-2-3 and end with closing the context. Presumably you could also structure it as a stack instead of this pseudo-recursion.

Answer 2

I don't know Lambda but you should look into the node async library as a way to sequence asynchronous functions.

async has made my life a lot easier and my code much more orderly without the deep nesting issue you mentioned in your question.

Typical async code might look like:

async.waterfall([
    function doTheFirstThing(callback) {
         db.somecollection.find({}).toArray(callback);
    },
    function useresult(dbFindResult, callback) {
         do some other stuff  (could be synch or async)
         etc etc etc
         callback(null);
],
function (err) {
    //this last function runs anytime any callback has an error, or if no error
    // then when the last function in the array above invokes callback.
    if (err) { sendForTheCodeDoctor(); }
});

Have a look at the async doco at the link above. There are many useful functions for serial, parallel, waterfall, and many more. Async is actively maintained and seems very reliable.

good luck!