How to mock streams in NodeJS

Question

I'm attempting to unit test one of my node-js modules which deals heavily in streams. I'm trying to mock a stream (that I will write to), as within my module I have ".on('data/end)" listeners that I would like to trigger. Essentially I want to be able to do something like this:

var mockedStream = new require('stream').readable();

mockedStream.on('data', function withData('data') {
  console.dir(data);
});

mockedStream.on('end', function() { 
  console.dir('goodbye');
});

mockedStream.push('hello world');
mockedStream.close();

This executes, but the 'on' event never gets fired after I do the push (and .close() is invalid).

All the guidance I can find on streams uses the 'fs' or 'net' library as a basis for creating a new stream (https://github.com/substack/stream-handbook), or they mock it out with sinon but the mocking gets very lengthy very quicky.

Is there a nice way to provide a dummy stream like this?

Answer 1

Building on @flacnut 's answer, I did this (in NodeJS 12+) using Readable.from() to construct a stream preloaded with data (a list of filenames):

    const mockStream = require('stream').Readable.from([
       'file1.txt',
       'file2.txt',
       'file3.txt',
    ])

In my case, I wanted to mock the stream of filenames returned by fast-glob.stream:

    const glob = require('fast-glob')
    // inject the mock stream into glob module
    glob.stream = jest.fn().mockReturnValue(mockStream)

In the function being tested:

  const stream = glob.stream(globFilespec)
  for await (const filename of stream) {
    // filename = file1.txt, then file2.txt, then file3.txt
  }

Works like a charm!

Answer 2

Instead of using Push, I should have been using ".emit(<event>, <data>);"

My mock code now works and looks like:

var mockedStream = new require('stream').Readable();
mockedStream._read = function(size) { /* do nothing */ };

myModule.functionIWantToTest(mockedStream); // has .on() listeners in it

mockedStream.emit('data', 'Hello data!');
mockedStream.emit('end');

Answer 3

There's a simpler way: stream.PassThrough

I've just found Node's very easy to miss stream.PassThrough class, which I believe is what you're looking for.

From Node docs:

The stream.PassThrough class is a trivial implementation of a Transform stream that simply passes the input bytes across to the output. Its purpose is primarily for examples and testing...

link to docs

The code from the question, modified:

const { PassThrough } = require('stream');
const mockedStream = new PassThrough(); // <----

mockedStream.on('data', (d) => {
    console.dir(d);
});

mockedStream.on('end', function() {
    console.dir('goodbye');
});

mockedStream.emit('data', 'hello world');
mockedStream.end();   //   <-- end. not close.
mockedStream.destroy();

mockedStream.push() works too but as a Buffer so you'll might want to do: console.dir(d.toString());

Answer 4

The accept answer is only partially correct. If all you need is events to fire, using .emit('data', datum) is okay, but if you need to pipe this mock stream anywhere else it won't work.

Mocking a Readable stream is surprisingly easy, requiring only the Readable lib.

  let eventCount = 0;
  const mockEventStream = new Readable({
    objectMode: true,
    read: function (size) {
      if (eventCount < 10) {
        eventCount = eventCount + 1;
        return this.push({message: `event${eventCount}`})
      } else {
        return this.push(null);
      }
    }
  });

Now you can pipe this stream wherever and 'data' and 'end' will fire.

Another example from the node docs: https://nodejs.org/api/stream.html#stream_an_example_counting_stream