webgl readpixels is always returning 0,0,0,0

Question

I am trying to do picking in WebGl. I have two shapes rendered along with different texture mapped on each. I am trying to grab pixel on certain co-ordinates. Here is the example.

var pixelValues = new Uint8Array(4);
gl.readPixels(10, 35, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, pixelValues);
console.log(pixelValues);

But pixelValues always contain [0,0,0,0]. What am I doing wrong? Do I need to do something related to framebuffer?

Answer 1

According to WebGL latest specs, you now need to call getContext setting the preserveDrawingBuffer flag, like:

var ctx = canvas.getContext("experimental-webgl", {preserveDrawingBuffer: true});

this prevents the drawing buffer (color, depth, stencil) from being cleared after they are draw to screen. Keep in mind that settings this may cause a performance penalty.

Answer 2

You don't need preserveDrawingBuffer: true to call readPixels. What you need is to call readPixels before exiting the current event.

The spec says if you call any function that affects the canvas (gl.clear, gl.drawXXX) then the browser will clear the canvas after the next composite operation. When that composite operation happens is up to the browser. It could be after it processes several mouse events or keyboard events or click events. The order is undefined. What is defined is that it won't do it until the current event exits so

render
read

const gl = document.querySelector("canvas").getContext("webgl");

render();
read();  // read in same event

function render() {
  gl.clearColor(.25, .5, .75, 1);
  gl.clear(gl.COLOR_BUFFER_BIT);
}

function read() {
  const pixel = new Uint8Array(4);
  gl.readPixels(0, 0, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, pixel);
  log(pixel);
}

function log(...args) {
  const elem = document.createElement("pre");
  elem.textContent = [...args].join(' ');
  document.body.appendChild(elem);
}

<canvas></canvas>

works where as

render
setTimeout(read, 1000);  // some other event

does not work

const gl = document.querySelector("canvas").getContext("webgl");

render();
setTimeout(read, 1000);  // read in other event

function render() {
  gl.clearColor(.25, .5, .75, 1);
  gl.clear(gl.COLOR_BUFFER_BIT);
}

function read() {
  const pixel = new Uint8Array(4);
  gl.readPixels(0, 0, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, pixel);
  log(pixel);
}

function log(...args) {
  const elem = document.createElement("pre");
  elem.textContent = [...args].join(' ');
  document.body.appendChild(elem);
}

<canvas></canvas>

Note that since it's the composite operation (the browser actually drawing the canvas on the page with the rest of the HTML) that triggers the clear, if the canvas is not on the page then it's not composited and won't be cleared.

In other words the case that didn't work above does work here

// create an offscreen canvas. Because it's offscreen it won't be composited
// and therefore will not be cleared.
const gl = document.createElement("canvas").getContext("webgl");

render();
setTimeout(read, 1000);  // read in other event

function render() {
  gl.clearColor(.25, .5, .75, 1);
  gl.clear(gl.COLOR_BUFFER_BIT);
}

function read() {
  const pixel = new Uint8Array(4);
  gl.readPixels(0, 0, 1, 1, gl.RGBA, gl.UNSIGNED_BYTE, pixel);
  log(pixel);
}

function log(...args) {
  const elem = document.createElement("pre");
  elem.textContent = [...args].join(' ');
  document.body.appendChild(elem);
}

Now, if you want to call readPixels in some other event, like when the user clicks an element, then you have at least 2 options

1. Set preserveDrawingBuffer: true

2. Render again in your event

   screenshotElement.addEventListener('click', event => {
     render();  
     gl.readPixels(...);
   });