Real Time Image Processing in Android using the ND

问题:

Using an Android (2.3.3) phone, I can use the camera to retrieve a preview with the onPreviewFrame(byte[] data, Camera camera) method to get the YUV image.

For some image processing, I need to convert this data to an RGB image and show it on the device. Using the basic java / android method, this runs at a horrible rate of less then 5 fps...

Now, using the NDK, I want to speed things up. The problem is: How do I convert the YUV array to an RGB array in C? And is there a way to display it (using OpenGL perhaps?) in the native code? Real-time should be possible (the Qualcomm AR demos showed us that).

I cannot use the setTargetDisplay and put an overlay on it!

I know Java, recently started with the Android SDK and have zero experience in C

回答1:

Have you considered using OpenCV's Android port? It can do a lot more than just color conversion, and it's quite fast.

回答2:

A Google search returned this page for a C implementation of YUV->RGB565. The author even included the JNI wrapper for it.

回答3:

You can also succeed by staying with Java. I did this for the imagedetectíon of the androangelo-app. I used the sample code which you find here by searching "decodeYUV".

For processing the frames, the essential part to consider is the image-size. Depending on the device you may get quite large images. i.e. for the Galaxy S2 the smallest supported previewsize is 640*480. This is a big amount of pixels.

What I did, is to use only every second row and every second column, after yuvtorgb decoding. So processing a 320*240 image works quite well and allowed me to get frame-rates of 20fps. (including some noise-reduction, a color-conversion from rgb to hsv and a circledetection)

In addition You should carefully check the size of the image-buffer provided to the setPreview function. If it is too small, the garbage-collection will spoil everything.

For the result you can check the calibration-screen of the androangelo-app. There I had an overlay of the detected image over the camera-preview.