I am using requests module in nodejs to make a request and I am streaming response I got. This response stream is piped to csv parser and populating records onto an array. Once I get a preset count of records, I want to end csv parsing and close the response stream. How do I properly cleanup response stream. Here is the pseudo code
var stream = request.get(url);
stream.pipe(csvParser);
var count = 15;
csvParser.on("readable",function(){
while(record = csvParser.read()){
if(records.length<count){
records.push(record);
} else {
csvParser.end();
//stream.close();
//stream.unpipe();
// stream.destroy();
}
}
});
csvParser.on("error",function(err){
console.log("Error",err.message);
})
csvParser.on("finish",function(){
//console.log("records",records);
console.log("done");
})
when I try stream.close() , it's saying undefined method. What's the correct way of cleaning it up..?
There is no
close
method for readable stream.close
method is old hack to close onlyfs
read stream (I do not know if it's still working)Also there is no method
destroy
in readable stream.I'm afraid there is no method to end stream before it ends.
You can stop reading from stream. In your case stream work in
flowing mode
, therefore this methodcsvParser.read()
when is called, is requests for next parts of data. If you stop callingcsvParser.read()
then this eventcsvParser.on("readable",function(){
stopped to fire.There is some good conversation https://groups.google.com/forum/#!msg/nodejs/eGukJUQrOBY/URD8I7tNxRUJ about similar case.