Looking to access 16-bit image data in Javascript/

Question

I'm trying to download 16-bit image data from a server and push it into a WebGL texture without browser plug-ins. texImage2d will work with: ImageData, HTMLImageElement, HTMLCanvasElement, or HTMLVideoElement. I'm looking for some javascript (a library or code sample) which can decode 16-bit TIFF or similar (hdf5, etc.) image data into one of these object types.

I have no problem doing this is 8-bit per channel RGB by using an to load a PNG but this doesn't work with 16-bit per channel data since there aren't any "standard" browser supported image formats which are 16-bit.

Answer 1

In case of combining two PNG images, one with the top 8 bits and the second with the low 8 bits, I think it should be:

highp vec4 texCol = texture2D(tex_low, vec2(vTexCoord.s, vTexCoord.t)) * (1.0 / 257.0);
texCol += texture2D(tex_up, vec2(vTexCoord.s, vTexCoord.t)) * (256.0 / 257.0);

In 8 bits per channel RGB colors will range from 0 to 255 = 2^8 - 1.
In 16 bits per channel RGB colors will range from 0 to 65535 = 2^16 - 1 = 255*257.

Explanation

WebGL works using colour values from 0 to 1 and makes it by dividing 8 bit color value by 255. So the divided value belongs to the range <0,1>.
In case of 16 bit per channel we would like to divide it by 65535 to get the proper number from range <0,1>.

What we want is 16 bit color value reduced to range <0,1>.
Let low and up be color value from range 0..255. up is top 8 bits and low is low 8 bits.
To get 16 bit value we can compute: low + up*256. Now we have number in range 0..65535. To get value from range <0,1> we divide it by 65535. Note that WebGL works using color values from range <0,1> , it is Lw=low/255 and Uw=up/255. So, we don't have to multiply it by 255 and divide it by 65535 because 65535 = 255*257. Instead we just divide by 257.

Also I could not find any software to split 16 bit / channel image into two 8 bit/channel image, so here is my code, feel free to use it, it splits 16 bit / channel Tiff into two 8 bit/channel PNGs:

https://github.com/czero69/ImageSplitter

Answer 2

PNGToy is a pretty featured library for extracting PNG chunks of almost all depths and channel modes with javascript (really client-side / without node.js, just Promise.js dependencies). The decode method will return the desired buffer. Here is an example for 16 bits grayscale PNG (16 bits RGB should work as well) :

var dataObj;
var img = new PngImage();
var buffer;

img.onload = function() {

    var pngtoy = this.pngtoy;

    dataObj = pngtoy.decode().then(function(results) {

        buffer = new Uint16Array(results.bitmap);

        for(var i = 0, j; i < buffer.length; i++) {

          j = buffer[i];
          buffer[i] = ((j & 0xff) << 8) | ((j & 0xff00) >>> 8); // needed to swap bytes for correct unsigned integer values  
        }

        console.log(buffer);
    });     
};

img.onerror = function(e) {
    console.log(e.message);
};

img.src = "image.png";

Answer 3

I don't think the main browsers natively support any 16-bit/channel image format at the moment.

One way to achieve the same effect would be to create two PNG images, one with the top 8 bits of each colour channel in the image and one with the bottom 8 bits. Then bind the images as two textures and combine the values in your shader, e.g.

highp float val = texture2d(samplerTop8bits, tex_coord) * (256.0 / 257.0);
val += texture2d(samplerBottom8bits, tex_coord) * (1.0 / 257.0);

(Note: you need highp precision to represent your data correctly in a 16-bit range)

Another method is only possible if floating point textures are supported in your target browser(s). You would, in the browser, combine the two PNG images into a floating point texture then access that normally. This may not be any faster and will probably use twice the amount of texture memory.