Read a bunch of JSON files, transform them, and sa

Question

I'm trying to achieve this with Gulp.

1. Read every .json file in a given directory including subdirectories.
2. Transform them in some way, for example add a new root level, etc.
3. Save them into a new directory keeping original structure.

The point where I'm lost is how to pipe reading/writing JSON to src.

I have the following skeleton now.

gulp.task("migratefiles", function () {
  return gulp.src("files/**/*.json")
      .pipe(/* WHAT HERE? */)
      .pipe(gulp.dest("processed"));
});

Answer 1

There's a number of way you can do this:

(1) Use the gulp-json-transform plugin:

var jsonTransform = require('gulp-json-transform');

gulp.task("migratefiles", function () {
  return gulp.src("files/**/*.json")
    .pipe(jsonTransform(function(json, file) {
      var transformedJson = {
        "newRootLevel": json
      };
      return transformedJson;
    }))
    .pipe(gulp.dest("processed"));
 });

Pros:

• Easy to use
• Supports asynchronous processing (if you return a Promise)
  
• Gives access to path of each file

Cons:

• Only rudimentary output formatting

(2) Use the gulp-json-editor plugin:

var jeditor = require('gulp-json-editor');

gulp.task("migratefiles", function () {
   return gulp.src("files/**/*.json")
     .pipe(jeditor(function(json) {
       var transformedJson = {
         "newRootLevel": json
       };
       return transformedJson;
     }))
     .pipe(gulp.dest("processed"));
});

Pros:

• Easy to use
• Automatically recognizes the indentation your input files use (two spaces, four spaces, tabs etc.) and formats your output files accordingly
• Supports various js-beautify options

Cons:

• Doesn't seem to support asynchronous processing
• Doesn't seem to have a way to access path of each file

(3) Do it manually (by directly accessing the vinyl file object using map-stream):

var map = require('map-stream');

gulp.task("migratefiles", function () {
   return gulp.src("files/**/*.json")
     .pipe(map(function(file, done) {
       var json = JSON.parse(file.contents.toString());
       var transformedJson = {
         "newRootLevel": json
       };
       file.contents = new Buffer(JSON.stringify(transformedJson));
       done(null, file);
     }))
     .pipe(gulp.dest("processed"));
});

Pros:

• Full control/access over everything
• Supports asynchronous processing (through a done callback)

Cons:

• Harder to use