Streaming data in node.js with ws and websocket-st

Question

I'm trying to stream data from a server to a client using websockets -- in particular, ws and websocket stream. However, the stream seems to be closing too early, and I only receive two chunks worth of data. Here is the simple example.

I have a server:

var WebSocketServer = require('ws').Server;
var websocketStream = require('websocket-stream');

var wss = new WebSocketServer({port: 8098});
var fs = require('fs');
var util = require('util');

wss.on('connection', function connect(ws) {
  var stream = websocketStream(ws);
  var rs = fs.createReadStream('foo.big');
  rs.pipe(stream);
});

and a client that connects and streams foo.big to a local file using pipe():

var fs = require('fs');
var util = require('util');
var websocket = require('websocket-stream');

var rs = fs.createWriteStream('big.out');

rs.on('open', function () {
  var ws = websocket('http://localhost:8098');

  ws.on('open', function () {
    ws.pipe(rs);
  }).on('error', function (err) {
    console.log(err);
  }).on('close', function (err) {
    console.log("Closing");
  });
});

Any ideas why I would only be getting a few packets of data? If I remove the call in the client to pipe, and just receive the data in chunks, it seems to receive all the data.

Answer 1

There is a couple of things wrong with the above (as of 2017).

1. You would rather want to listen for the fs.readStream() close event
2. You should not be listening to the websocket-stream open event, as you can assume it to be open

Also, performance: ws is likely not suited for server to server communication. Consider alternatives, like net, like here, or various other lower than ws level ways.

Find a full example here on my GH.

const fs = require('fs')
const websocket = require('websocket-stream')

const writeStream = fs.createWriteStream('big.out')

writeStream.on('open', () => {
  let ws = websocket('http://localhost:8098')

  ws.pipe(writeStream)

  ws.on('error', (err) => {
    if (err) throw err
  }).on('close', (err) => {
    if (err) throw err
    console.log(`Closing ws with: ${fs.statSync('big.out').size} bytes`)
  })

  writeStream.on('close', () => {
    console.log(`Closing file stream with: ${fs.statSync('big.out').size} bytes`)
  })
})