TangoImageBuffer RAW/YUV failed to match frame

I use the OnFrameAvailable callback to sync rgb with depth data. If I just render the rgb point cloud everything is fine.

But if I do some image processing the camera throws the following exceptions:

E/camera-metadata: /home/ubuntu/jobs/redwood_internal/RedwoodInternal/Redwood/common/player-engine/src/camera-metadata.cc:56 RAW failed to match frame

E/camera-metadata: /home/ubuntu/jobs/redwood_internal/RedwoodInternal/Redwood/common/player-engine/src/camera-metadata.cc:56 YUV failed to match frame

And TangoImageBuffer has complete garbage values. Sometimes black pixels, or the buffer half-half old and new pixel data.

I tried to solve it by threading. Everytime I got a new point cloud, the extra image processing thread needs about 1 sec cpu time. And it helped a little. After a few seconds the same behavior happend.

The problem is that I can't debug the nativ code properly. The monitoring of android studio shows normal cpu and gpu usage.

I've seen that user guppy had this problem with the Leipniz tango version, but no solution was posted. So I hope maybe someone else has managed this problem? Or has any suggestions?

EDIT

The behaviour disappeared, after using the tango_support library to copy xyz and yuv buffers.

The "YUV failed to match frame" is most likely caused by the callback thread is taking too much time to execute. In short, you shouldn't do heavy processing in the OnFrameAvailable callback. This also applies to all other Tango callbacks, i.e pose or depth callback.

The solution to this would be copying out the byte buffer data and process it in another thread, potentially, the render thread. In the tango-example-c video-overlay-jni-example, the application does a memcpy to copy the data from callback thread to the render thread, so the processing of the data would not block the callbacks keep coming. See this line.