Node Js :is it possible to hit 1000 http get reque

2019-01-29 01:05发布

问题:

I was trying to perform nearly 2000+ simultaneous http get request to some web api ( abc.com/query?val=somekey).below is my code.

async.each(keysArray,function(key,callback1){
  sails.http.get({
    hostname:'abc.com',
    path:'/query?val='+key,
    headers:{
      "Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
      "User-Agent":"MYBROWSER"
    }
  },function(response){
    var str='';
    response.on('data',function(chunk){
      str+=chunk;
    });
    response.on('end',function(){
      console.log(new Buffer(str,'utf8'));
    //  some job on each str from each key
    });
  });
});

length of keysArray is around 2000.So the above code is performing nearly 2000 http get request.But i was getting error like

events.js:141
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at exports._errnoException (util.js:870:11)
    at TCP.onread (net.js:544:26)

Though i found a way to limit concurrent using async.eachLimits(). Is it possible to perform that much requests or any kind of dependency of machine or server is there.