Write objects into file with Node.js

Question

I've searched all over stackoverflow / google for this, but can't seem to figure it out.

I'm scraping social media links of a given URL page, and the function returns an object with a list of URLs.

When I try to write this data into a different file, it outputs to the file as [object Object] instead of the expected: [ 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'] as it does when I console.log() the results.

This is my sad attempt to read and write a file in Node, trying to read each line(the url) and input through a function call request(line, gotHTML):

fs.readFileSync('./urls.txt').toString().split('\n').forEach(function (line){
    console.log(line); 
    var obj = request(line, gotHTML); 
    console.log(obj); 
    fs.writeFileSync('./data.json', obj , 'utf-8'); 
});   

for reference -- the gotHTML function:

function gotHTML(err, resp, html){ 
    var social_ids = []; 

    if(err){
        return console.log(err); 
    } else if (resp.statusCode === 200){ 
        var parsedHTML = $.load(html); 

        parsedHTML('a').map(function(i, link){
            var href = $(link).attr('href');
            for(var i=0; i<socialurls.length; i++){
                if(socialurls[i].test(href) && social_ids.indexOf(href) < 0 ) {
                    social_ids.push(href); 
                }; 
            }; 
        })
    };

    return social_ids;
};

Answer 1

could you try doing JSON.stringify(obj);

Like this

 var stringify = JSON.stringify(obj);
fs.writeFileSync('./data.json', stringify , 'utf-8'); 

Answer 2

Building on what deb2fast said I would also pass in a couple of extra parameters to JSON.stringify() to get it to pretty format:

fs.writeFile('./data.json', JSON.stringify(obj, null, 2) , 'utf-8');

The second param is an optional replacer function which you don't need in this case so null works.

The third param is the number of spaces to use for indentation. 2 and 4 seem to be popular choices.

Answer 3

If you're geting [object object] then use JSON.stringify

fs.writeFile('./data.json', JSON.stringify(obj) , 'utf-8');

It worked for me.

Answer 4

In my experience JSON.stringify is slightly faster than util.inspect. I had to save the result object of a DB2 query as a json file, The query returned an object of 92k rows, the conversion took very long to complete with util.inspect, so I did the following test by writing the same 1000 record object to a file with both methods.

1. JSON.Stringify

   fs.writeFile('./data.json', JSON.stringify(obj, null, 2));
   

Time: 3:57 (3 min 57 sec)

Result's format:

[
  {
    "PROB": "00001",
    "BO": "AXZ",
    "CNTRY": "649"
   },
  ...
]

2. util.inspect

   var util = require('util');
   fs.writeFile('./data.json', util.inspect(obj, false, 2, false));
   

Time: 4:12 (4 min 12 sec)

Result's format:

[ { PROB: '00001',
    BO: 'AXZ',
    CNTRY: '649' },
    ...
]

Answer 5

obj is an array in your example.

fs.writeFileSync(filename, data, [options]) requires either String or Buffer in the data parameter. see docs.

Try to write the array in a string format:

// writes 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'
fs.writeFileSync('./data.json', obj.join(',') , 'utf-8'); 

Or:

// writes ['https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks']
var util = require('util');
fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8');

edit: The reason you see the array in your example is because node's implementation of console.log doesn't just call toString, it calls util.format see console.js source