Looping through JSON with node.js

Question

I have a JSON file which I need to iterate over, as shown below...

{
    "device_id": "8020",
    "data": [{
        "Timestamp": "04-29-11 05:22:39 pm",
        "Start_Value":  0.02,
        "Abstract": 18.60,
        "Editor": 65.20
    }, {
        "Timestamp": "04-29-11 04:22:39 pm",
        "End_Value":  22.22,
        "Text": 8.65,
        "Common": 1.10,
        "Editable": "true",
        "Insert": 6.0
    }]
}

The keys in data will not always be the same (i've just used examples, there are 20 different keys), and as such, I cannot set up my script to statically reference them to get the values.

Otherwise I could state

var value1 = json.data.Timestamp;
var value2 = json.data.Start_Value;
var value3 = json.data.Abstract;
etc

In the past i've used a simple foreach loop on the data node...

foreach ($json->data as $key => $val) {
    switch($key) {
        case 'Timestamp':
            //do this;
        case: 'Start_Value':
            //do this
    }
}

But don't want to block the script. Any ideas?

Answer 1

I would recommend taking advantage of the fact that nodeJS will always be ES5. Remember this isn't the browser folks you can depend on the language's implementation on being stable. That said I would recommend against ever using a for-in loop in nodeJS, unless you really want to do deep recursion up the prototype chain. For simple, traditional looping I would recommend making good use of Object.keys method, in ES5. If you view the following JSPerf test, especially if you use Chrome (since it has the same engine as nodeJS), you will get a rough idea of how much more performant using this method is than using a for-in loop (roughly 10 times faster). Here's a sample of the code:

 var keys = Object.keys( obj );
 for( var i = 0,length = keys.length; i < length; i++ ) {
     obj[ keys[ i ] ];
 }

Answer 2

You may also want to use hasOwnProperty in the loop.

for (var prop in obj) {
    if (obj.hasOwnProperty(prop)) {
        switch (prop) {
            // obj[prop] has the value
        }
    }
}

node.js is single-threaded which means your script will block whether you want it or not. Remember that V8 (Google's Javascript engine that node.js uses) compiles Javascript into machine code which means that most basic operations are really fast and looping through an object with 100 keys would probably take a couple of nanoseconds?

However, if you do a lot more inside the loop and you don't want it to block right now, you could do something like this

switch (prop) {
    case 'Timestamp':
        setTimeout(function() { ... }, 5);
        break;
    case 'Start_Value':
        setTimeout(function() { ... }, 10);
        break;
}

If your loop is doing some very CPU intensive work, you will need to spawn a child process to do that work or use web workers.

Answer 3

Take a look at Traverse. It will recursively walk an object tree for you and at every node you have a number of different objects you can access - key of current node, value of current node, parent of current node, full key path of current node, etc. https://github.com/substack/js-traverse. I've used it to good effect on objects that I wanted to scrub circular references to and when I need to do a deep clone while transforming various data bits. Here's some code pulled form their samples to give you a flavor of what it can do.

var id = 54;
var callbacks = {};
var obj = { moo : function () {}, foo : [2,3,4, function () {}] };

var scrubbed = traverse(obj).map(function (x) {
    if (typeof x === 'function') {
        callbacks[id] = { id : id, f : x, path : this.path };
        this.update('[Function]');
        id++;
    }
});

console.dir(scrubbed);
console.dir(callbacks);

Answer 4

My most preferred way is,

var objectKeysArray = Object.keys(yourJsonObj)
objectKeysArray.forEach(function(objKey) {
    var objValue = yourJsonObj[objKey]
})

Answer 5

Not sure if it helps, but it looks like there might be a library for async iteration in node hosted here:

https://github.com/caolan/async

Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript. Although originally designed for use with node.js, it can also be used directly in the browser.

Async provides around 20 functions that include the usual 'functional' suspects (map, reduce, filter, forEach…) as well as some common patterns for asynchronous control flow (parallel, series, waterfall…). All these functions assume you follow the node.js convention of providing a single callback as the last argument of your async function.

Answer 6

If we are using nodeJS, we should definitely take advantage of different libraries it provides. Inbuilt functions like each(), map(), reduce() and many more from underscoreJS reduces our efforts. Here's a sample

    var _=require("underscore");
    var fs=require("fs");

    var jsonObject=JSON.parse(fs.readFileSync('YourJson.json', 'utf8'));


    _.map( jsonObject, function(content) {
        _.map(content,function(data){
           if(data.Timestamp)
              console.log(data.Timestamp)          
           })
      })