How does the paste image from clipboard functional

Question

I noticed a blog post from Google that mentions the ability to paste images directly from the clipboard into a Gmail message if you're using the latest version of Chrome. I tried this with my version of Chrome (12.0.742.91 beta-m) and it works great using control keys or the context menu.

From that behavior I need to assume that the latest version of webkit used in Chrome is able to deal with images in the Javascript paste event, but I have been unable to locate any references to such an enhancement. I believe ZeroClipboard binds to keypress events to trigger its flash functionality and as such wouldn't work through the context menu (also, ZeroClipboard is cross-browser and the post says this works only with Chrome).

So, how does this work and where the enhancement was made to Webkit (or Chrome) that enables the functionality?

Answer 1

I spent some time experimenting with this. It seems to sort of follow the new Clipboard API spec. You can define a "paste" event handler and look at event.clipboardData.items, and call getAsFile() on them to get a Blob. Once you have a Blob, you can use FileReader on it to see what's in it. This is how you can get a data url for the stuff you just pasted in Chrome:

// window.addEventListener('paste', ... or
document.onpaste = function(event){
  var items = (event.clipboardData || event.originalEvent.clipboardData).items;
  console.log(JSON.stringify(items)); // will give you the mime types
  for (index in items) {
    var item = items[index];
    if (item.kind === 'file') {
      var blob = item.getAsFile();
      var reader = new FileReader();
      reader.onload = function(event){
        console.log(event.target.result)}; // data url!
      reader.readAsDataURL(blob);
    }
  }
}

Once you have a data url you can display the image on the page. If you want to upload it instead, you could use readAsBinaryString, or you could put it into an XHR using FormData.

Answer 2

Wow, that's cool. I haven't dived into the gmail source to figure it out yet (I did with the drag-out functionality), but I'm guessing that it's an extension of the drag/drop API that chrome has already extended. There's a decent write-up on how the drag-to-desktop feature works: http://www.thecssninja.com/javascript/gmail-dragout that may at least point you in the right direction.

Answer 3

As far as I know -

With HTML 5 features(File Api and the related) - accessing clipboard image data is now possible with plain javascript.

This however fails to work on IE (anything less than IE 10). Don't know much about IE10 support also.

For IE the optiens that I believe are the 'fallback' options are either using Adobe's AIR api or using a signed applet

Answer 4

The answer by Nick seems to need small changes to still work :)

// window.addEventListener('paste', ... or
document.onpaste = function (event) {
  // use event.originalEvent.clipboard for newer chrome versions
  var items = (event.clipboardData  || event.originalEvent.clipboardData).items;
  console.log(JSON.stringify(items)); // will give you the mime types
  // find pasted image among pasted items
  var blob = null;
  for (var i = 0; i < items.length; i++) {
    if (items[i].type.indexOf("image") === 0) {
      blob = items[i].getAsFile();
    }
  }
  // load image if there is a pasted image
  if (blob !== null) {
    var reader = new FileReader();
    reader.onload = function(event) {
      console.log(event.target.result); // data url!
    };
    reader.readAsDataURL(blob);
  }
}

Example running code: http://jsfiddle.net/bt7BU/225/

So the changes to nicks answer were:

var items = event.clipboardData.items;

to

var items = (event.clipboardData  || event.originalEvent.clipboardData).items;

Also I had to take the second element from the pasted items (first one seems to be text/html if you copy an image from another web page into the buffer). So I changed

  var blob = items[0].getAsFile();

to a loop finding the item containing the image (see above)

I didn't know how to answer directly to Nick's answer, hope it is fine here :$ :)

Answer 5

Here's a smooth jQuery plugin wrapping up the whole deal (basically the same principles as Nick's answer): http://strd6.com/2011/09/html5-javascript-pasting-image-data-in-chrome/

It's got a live demo, annotated source code, and everything.

Answer 6

Web browsers keep marching forward. I recently found this:

Code Snippet — Accessing Clipboard Images with Javascript

and this:

The Paste Wasteland (or, why the onPaste event is a mess)

The first link describes a way to get clipboard images using JavaScript only on Firefox and Chrome. The second link contains a postscript that mentions the same technique was adapted to IE (unknown version).