Can I ungzip files in Google Drive with apps scrip

Question

This is a follow-up to this thread.

I am trying to use the code provided by @July.Tech there, but I keep getting unknown compression method error or incorrect header check error. The error came up when either of two different gzip methods were used to create the compressed file, so I would think the file is correctly gzipped.

Any suggestions? (My input file is gzipped so I cannot use Utilities.unzip().)

Here is the entire code:

reports_folder_id = 'xxxxx'; //id of folder where gzipped csv reports are saved
report_name = 'xxxxxx.gz'; // name of gzipped CSV file

function importData() { 
  var fSource = DriveApp.getFolderById(reports_folder_id);
  var fi = fSource.getFilesByName(report_name); // latest report file

  eval(UrlFetchApp.fetch('https://cdn.rawgit.com/nodeca/pako/master/dist/pako.js').getContentText());

  if ( fi.hasNext() ) { // proceed if report_name file exists in the reports folder
    var file = fi.next();

    var charData = file.getBlob().getDataAsString(); // same error if .getBytes() is used

    var binData = [];
    for (var i = 0; i < charData.length; i++) {
      binData.push(charData[i] < 0 ? charData[i] + 256 : charData[i]);
    }

    var data = pako.ungzip(binData); // I get same error for pako.inflate(binData);

    var decoded = '';
    for (var i = 0; i < data.length; i++) {
      decoded += String.fromCharCode(data[i]);
    }

  }
}

If no suggestion for fixing the above, any ideas on how to ungzip a gDrive file programatically?

Thanks.

Answer 1

As of January 19, 2018 (see release notes) Apps Script now supports gzip compression accessible using the following Utilities methods:

• gzip(blob)
• ungzip(blob)

Answer 2

You need to find out if pako supports gzip compression. If not, you should look for another compression package that supports gzip.

Answer 3

Running the supplied example code indeed results in unknown compression method error in my environment as well.

Try changing

var charData = file.getBlob().getDataAsString(); // same error if .getBytes() is used

To

var charData = file.getBlob().getBytes();

So that is

function myFunction() {
  reports_folder_id = '<FOLDER_ID>';
  report_name = 'zip3.csv.gz'; // name of gzipped CSV file

  var fSource = DriveApp.getFolderById(reports_folder_id);
  var fi = fSource.getFilesByName(report_name);
  eval(UrlFetchApp.fetch('https://cdn.rawgit.com/nodeca/pako/master/dist/pako.js').getContentText());

  if ( fi.hasNext() ) { 
    var file = fi.next();
    var blobData = file.getBlob();

    var charData = blobData.getBytes();

    var binData = [];
    for (var i = 0; i < charData.length; i++) {
      binData.push(charData[i] < 0 ? charData[i] + 256 : charData[i]);
    }

    var data = pako.inflate(binData); 

    var decoded = '';
    for (var i = 0; i < data.length; i++) {
      decoded += String.fromCharCode(data[i]);
    }
    Logger.log(decoded);

  }

}

Try running this script on a subset of your original "GlicemiaMisurazioni.csv.gz" file: https://drive.google.com/file/d/0B8geUNXmd4J2YzJoemFLMnBTbVU/view?usp=sharing

(I truncated the original csv to 32 rows to speed up execution for the sake of test – the original takes way too long to run)

Checking the logs shows that the uncompressing worked:

4;02/07/2017 03.00.30;;;158.0;158.0;1;0M0000UMQ5D;;;0;;
4;02/07/2017 02.59.30;;;158.0;158.0;1;0M0000UMQ5D;;;0;;
4;02/07/2017 02.58.30;;;159.0;159.0;1;0M0000UMQ5D;;;0;;
4;02/07/2017 02.57.30;;;159.0;159.0;1;0M0000UMQ5D;;;0;;
4;02/07/2017 02.56.30;;;158.0;158.0;1;0M0000UMQ5D;;;0;;
4;02/07/2017 02.56.00;;;;;0;;0.4;Novorapid ;0;Left flank;Test