Parsing inlineImages from Gmail raw content

Question

Gmail message getAttachments function is not returning inlineImages - see issue 2810 https://code.google.com/p/google-apps-script-issues/issues/detail?id=2810

I need to do that, so I wrote the code below to parse the inline image in blob format out of the message raw content, knowing the image cid within the message, in advance.

However, I am afraid this parsing is quite fragile in the way I find the first and last character in the base64 image content, isn't it?

Is there a better way of doing this?

Regards, Fausto

var rawc = message.getRawContent();
var b64c1 = rawc.lastIndexOf(cid) + cid.length + 3; // first character in image base64
var b64cn = rawc.substr(b64c1).indexOf("--") - 3; // last character in image base64
var imgb64 = rawc.substring(b64c1, b64c1 + b64cn + 1); // is this fragile or safe enough?
var imgblob = Utilities.newBlob(Utilities.base64Decode(imgb64), "image/jpeg", cid); // decode and blob

Answer 1

I've had this problem a number of times, and I think I have a pretty general case solution. Getting non-embedded images has also been a problem.

I'm not sure my parsing is any less fragile than yours. In the end, I'm sucking out the part of the multipart by grabbing the surrounding lines that start with '--'. Everything else is just making sure I can use this without modifying the code too much when I need it next. I have had some emails which don't seem follow the \r\n and cause problems: something to lookout for.

The getInlineImages function will take the raw content of the message and return an array of objects. Each object will have the src of the img tag and the blob that goes with the image. If you just want inline images, you can choose to ignore anything that doesn't start with 'cid'.

The getBlobFromMessage function will take the raw content of the message and the src of the img tag (including 'cid') and return the associated blob.

You can see the code commented here.

function getInlineImages(rawContent) {
  var url = /^https?:\/\//, cid = /^cid:/;
  var imgtags = rawContent.match(/<img.*?>(.*?<\/img>)?/gi);
  return imgtags ? imgtags.map(function(imgTag) {
    var img = {src: Xml.parse(imgTag,true).html.body.img.src};
    img.blob = url.test(img.src) ? UrlFetchApp.fetch(img.src).getBlob()
             : cid.test(img.src) ? getBlobFromMessage(rawContent,img.src)
             : null;
    return img;
  }) : [];
}

function getBlobFromMessage(rawContent,src) {
  var cidIndex = src.search(/cid:/i);
  if(cidIndex === -1) throw Utilities.formatString("Did not find cid: prefix for inline refenece: %s", src)

  var itemId = src.substr(cidIndex + 4);
  var contentIdIndex = rawContent.search("Content-ID:.*?" + itemId);
  if(contentIdIndex === -1) throw Utilities.formatString("Item with ID %s not found.",src);

  var previousBoundaryIndex = rawContent.lastIndexOf("\r\n--",contentIdIndex);
  var nextBoundaryIndex = rawContent.indexOf("\r\n--",previousBoundaryIndex+1);
  var part = rawContent.substring(previousBoundaryIndex,nextBoundaryIndex);

  var contentTransferEncodingLine = part.match(/Content-Transfer-Encoding:.*?\r\n/i)[0];
  var encoding = contentTransferEncodingLine.split(":")[1].trim();
  if(encoding != "base64") throw Utilities.formatString("Unhandled encoding type: %s",encoding);

  var contentTypeLine = part.match(/Content-Type:.*?\r\n/i)[0];
  var contentType = contentTypeLine.split(":")[1].split(";")[0].trim();

  var startOfBlob = part.indexOf("\r\n\r\n");
  var blobText = part.substring(startOfBlob).replace("\r\n",""); 
  return Utilities.newBlob(Utilities.base64Decode(blobText),contentType,itemId);
}

Answer 2

A more recent approach for this issue.

The issue

For example, here's an email body retrieved with .getBody()

<div dir="ltr"><div><img src="?view=att&amp;th=1401f70d4881e07f&amp;attid=0.3&amp;disp=emb&amp;realattid=ii_1401f6fc7824ebe1&amp;zw&amp;atsh=1" alt="Inline image 4" width="200" height="180"><br></div><div><br></div><img src="?view=att&amp;th=1401f70d4881e07f&amp;attid=0.2&amp;disp=emb&amp;realattid=ii_1401f6e6c1d46c4b&amp;zw&amp;atsh=1" alt="Inline image 2" width="200" height="65"><div><br></div><div>
jtykuykyu</div><div><br></div><div><img src="?view=att&amp;th=1401f70d4881e07f&amp;attid=0.1&amp;disp=emb&amp;realattid=ii_1401f6e9df3a4b1c&amp;zw&amp;atsh=1" alt="Inline image 3" width="200" height="82"><br><div><br></div><div><br></div></div></div>

And here is the list of attachments for the email (among which are our inline images):

[13-07-30 08:28:08:378 CEST] Screen Shot 2013-07-12 at 1.54.31 PM.png

[13-07-30 08:28:08:379 CEST] Screen Shot 2013-07-23 at 5.38.51 PM.png

[13-07-30 08:28:08:380 CEST] Screen Shot 2013-07-25 at 9.05.15 AM.png

[13-07-30 08:28:08:381 CEST] test2.png

As you can see, there's no link between the name of those images and the information available in the img tags, so there's no safe way to rebuild a correct email with only those information.

The solution

How to solve that ? We can use the method .getRawContent() to get the actual email and parse it to get the information we need. Specifically, this method give us a relationship between the name of an attachment and the 'realattid' available in the email body:

Content-Type: image/png; name="Screen Shot 2013-07-25 at 9.05.15 AM.png"

Content-Transfer-Encoding: base64

Content-ID:

X-Attachment-Id: ii_1401f6e9df3a4b1c

Code snippet

Here's a code snippet to:

-Retrieve the body & attachments of an email

-Get all the img tags inside the body and see which ones are linked to attachments in the email

-Get the 'realattid' of each image and use .getRawContent() to link this 'realattid' to the right attachment

-Replace the img tag to correctly link it to the right attachment

-Indicate that this attachment is no longer a simple attachment but an inline image

-Once all that is done you have all the data you need to send a copy of this email with the correct inline images displayed.

 //////////////////////////////////////////////////////////////////////////////
  // Get inline images and make sure they stay as inline images
  //////////////////////////////////////////////////////////////////////////////
  var emailTemplate = selectedTemplate.getBody();
  var rawContent = selectedTemplate.getRawContent();
  var attachments = selectedTemplate.getAttachments();

  var regMessageId = new RegExp(selectedTemplate.getId(), "g");
  if (emailTemplate.match(regMessageId) != null) {
    var inlineImages = {};
    var nbrOfImg = emailTemplate.match(regMessageId).length;
    var imgVars = emailTemplate.match(/<img[^>]+>/g);
    var imgToReplace = [];
    if(imgVars != null){
      for (var i = 0; i < imgVars.length; i++) {
        if (imgVars[i].search(regMessageId) != -1) {
          var id = imgVars[i].match(/realattid=([^&]+)&/);
          if (id != null) {
            var temp = rawContent.split(id[1])[1];
            temp = temp.substr(temp.lastIndexOf('Content-Type'));
            var imgTitle = temp.match(/name="([^"]+)"/);
            if (imgTitle != null) imgToReplace.push([imgTitle[1], imgVars[i], id[1]]);
          }
        }
      }
    }
    for (var i = 0; i < imgToReplace.length; i++) {
      for (var j = 0; j < attachments.length; j++) {
        if(attachments[j].getName() == imgToReplace[i][0]) {
          inlineImages[imgToReplace[i][2]] = attachments[j].copyBlob();
          attachments.splice(j, 1);
          var newImg = imgToReplace[i][1].replace(/src="[^\"]+\"/, "src=\"cid:" + imgToReplace[i][2] + "\"");
          emailTemplate = emailTemplate.replace(imgToReplace[i][1], newImg);
        }
      }
    }
  }
  //////////////////////////////////////////////////////////////////////////////
  var message = {
    htmlBody: emailTemplate,
    subject: selectedTemplate.getSubject(),
    attachments: attachments,
    inlineImages: inlineImages
  }