error handling in asynchronous node.js calls

Question

I'm new to node.js although I'm pretty familiar with JavaScript in general. My question is regarding "best practices" on how to handle errors in node.js.

Normally when programming web servers, FastCGI servers or web pages in various languages I'm using Exceptions with blocking handlers in a multi-threading environment. When a request comes in I usually do something like this:

function handleRequest(request, response) {
  try {

    if (request.url=="whatever")
      handleWhateverRequest(request, response);
    else
      throw new Error("404 not found");

  } catch (e) {
    response.writeHead(500, {'Content-Type': 'text/plain'});
    response.end("Server error: "+e.message);
  }
}

function handleWhateverRequest(request, response) {
  if (something) 
    throw new Error("something bad happened");
  Response.end("OK");
}

This way I can always handle internal errors and send a valid response to the user.

I understand that with node.js one is supposed to do non-blocking calls which obviously leads to various number of callbacks, like in this example:

var sys    = require('sys'),
    fs     = require('fs');

require("http").createServer(handleRequest).listen(8124);

function handleRequest(request, response) {

  fs.open("/proc/cpuinfo", "r",
    function(error, fd) {
      if (error)
        throw new Error("fs.open error: "+error.message);

      console.log("File open.");

      var buffer = new require('buffer').Buffer(10);
      fs.read(fd, buffer, 0, 10, null,
        function(error, bytesRead, buffer) {

          buffer.dontTryThisAtHome();  // causes exception

          response.end(buffer);
        }); //fs.read

    }); //fs.open

}

This example will kill the server completely because exceptions aren't being catched. My problem is here that I can't use a single try/catch anymore and thus can't generally catch any error that may be raised during the handling of the request.

Of course I could add a try/catch in each callback but I don't like that approach because then it's up to the programmer that he doesn't forget a try/catch. For a complex server with lots of different and complex handlers this isn't acceptable.

I could use a global exception handler (preventing the complete server crash) but then I can't send a response to the user since I don't know which request lead to the exception. This also means that the request remains unhandled/open and the browser is waiting forever for a response.

Does someone have a good, rock solid solution?

Answer 1

I've recently created a simple abstraction named WaitFor to call async functions in sync mode (based on Fibers): https://github.com/luciotato/waitfor

It's too new to be "rock solid".

using wait.for you can use async function as if they were sync, without blocking node's event loop. It's almost the same you're used to:

var wait=require('wait.for');

function handleRequest(request, response) {
      //launch fiber, keep node spinning
      wait.launchFiber(handleinFiber,request, response); 
}

function handleInFiber(request, response) {
  try {
    if (request.url=="whatever")
      handleWhateverRequest(request, response);
    else
      throw new Error("404 not found");

  } catch (e) {
    response.writeHead(500, {'Content-Type': 'text/plain'});
    response.end("Server error: "+e.message);
  }
}

function handleWhateverRequest(request, response, callback) {
  if (something) 
    throw new Error("something bad happened");
  Response.end("OK");
}

Since you're in a fiber, you can program sequentially, "blocking the fiber", but not node's event loop.

The other example:

var sys    = require('sys'),
    fs     = require('fs'),
    wait   = require('wait.for');

require("http").createServer( function(req,res){
      wait.launchFiber(handleRequest,req,res) //handle in a fiber
  ).listen(8124);

function handleRequest(request, response) {
  try {
    var fd=wait.for(fs.open,"/proc/cpuinfo", "r");
    console.log("File open.");
    var buffer = new require('buffer').Buffer(10);

    var bytesRead=wait.for(fs.read,fd, buffer, 0, 10, null);

    buffer.dontTryThisAtHome();  // causes exception

    response.end(buffer);
  }
  catch(err) {
    response.end('ERROR: '+err.message);
  }

}

As you can see, I used wait.for to call node's async functions in sync mode, without (visible) callbacks, so I can have all the code inside one try-catch block.

wait.for will throw an exception if any of the async functions returns err!==null

more info at https://github.com/luciotato/waitfor

Answer 2

One idea: You could just use a helper method to create your call backs and make it your standard practice to use it. This does put the burden on the developer still, but at least you can have a "standard" way of handling your callbacks such that the chance of forgetting one is low:

var callWithHttpCatch = function(response, fn) {
    try {
        fn && fn();
    }
    catch {
        response.writeHead(500, {'Content-Type': 'text/plain'}); //No
    }
}

<snipped>
      var buffer = new require('buffer').Buffer(10);
      fs.read(fd, buffer, 0, 10, null,
        function(error, bytesRead, buffer) {

          callWithHttpCatch(response, buffer.dontTryThisAtHome());  // causes exception

          response.end(buffer);
        }); //fs.read

    }); //fs.open

I know that probably isn't the answer you were looking for, but one of the nice things about ECMAScript (or functional programming in general) is how easily you can roll your own tooling for things like this.

Answer 3

Also in synchronous multi-threaded programming (e.g. .NET, Java, PHP) you can't return any meaningful information to the client when a custom unkown Exception is caught. You may just return HTTP 500 when you have no info regarding the Exception.

Thus, the 'secret' lies in filling a descriptive Error object, this way your error handler can map from the meaningful error to the right HTTP status + optionally a descriptive result. However you must also catch the exception before it arrives to process.on('uncaughtException'):

Step1: Define a meaningful error object

function appError(errorCode, description, isOperational) {
    Error.call(this);
    Error.captureStackTrace(this);
    this.errorCode = errorCode;
    //...other properties assigned here
};

appError.prototype.__proto__ = Error.prototype;
module.exports.appError = appError;

Step2: When throwing an Exception, fill it with properties (see step 1) that allows the handler to convert it to meannigul HTTP result:

throw new appError(errorManagement.commonErrors.resourceNotFound, "further explanation", true)

Step3: When invoking some potentially dangerous code, catch errors and re-throw that error while filling additional contextual properties within the Error object

Step4: You must catch the exception during the request handling. This is easier if you use some leading promises library (BlueBird is great) which allows you to catch async errors. If you can't use promises than any built-in NODE library will return errors in callback.

Step5: Now that your error is caught and contains descriptive information about what happens, you only need to map it to meaningful HTTP response. The nice part here is that you may have a centralized, single error handler that gets all the errors and map these to HTTP response:

    //this specific example is using Express framework
    res.status(getErrorHTTPCode(error))
function getErrorHTTPCode(error)
{
    if(error.errorCode == commonErrors.InvalidInput)
        return 400;
    else if...
}

You may other related best practices here

Answer 4

Node 0.8 introduces a new concept called "Domains". They are very roughly analogousness to AppDomains in .net and provide a way of encapsulating a group of IO operations. They basically allow you to wrap your request processing calls in a context specific group. If this group throws any uncaught exceptions then they can be handled and dealt with in a manner which gives you access to all the scope and context specific information you require in order to successfully recover from the error (if possible).

This feature is new and has only just been introduced, so use with caution, but from what I can tell it has been specifically introduced to deal with the problem which the OP is trying to tackle.

Documentation can be found at: http://nodejs.org/api/domain.html