How to get Android camera preview data?

问题:

My camera app displays a camera preview on the screen and also processes it in the background. Here is the relevant code, condensed as much as possible (e.g. no error handling or field declarations shown):

public final class CameraView extends SurfaceView implements
          SurfaceHolder.Callback, Runnable, PreviewCallback {

    public CameraView(Context context, AttributeSet attrs) {
        super(context, attrs);
        mHolder = getHolder();
        mHolder.addCallback(this);
        mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS); 

    void openCamera() {
        // Called from parent activity after setting content view to CameraView
        mCamera = Camera.open();
        mCamera.setPreviewCallbackWithBuffer(this);
    }

    public void surfaceCreated(SurfaceHolder holder) {
        new Thread(this).start(); 

        // Set CameraView to the optimal camera preview size

        final Camera.Parameters params = mCamera.getParameters();
        final List<Camera.Size> sizes = params.getSupportedPreviewSizes();
        final int screenWidth = ((View) getParent()).getWidth();
        int minDiff = Integer.MAX_VALUE;
        Camera.Size bestSize = null;

        if (getResources().getConfiguration().orientation 
                == Configuration.ORIENTATION_LANDSCAPE) {
            // Find the camera preview width that best matches the
            // width of the surface.
            for (Camera.Size size : sizes) {
                final int diff = Math.abs(size.width - screenWidth);
                if (diff < minDiff) {
                    minDiff = diff;
                    bestSize = size;
                }
            }
        } else {
            // Find the camera preview HEIGHT that best matches the 
            // width of the surface, since the camera preview is rotated.
            mCamera.setDisplayOrientation(90);
            for (Camera.Size size : sizes) {
                final int diff = Math.abs(size.height - screenWidth);
                if (Math.abs(size.height - screenWidth) < minDiff) {
                    minDiff = diff;
                    bestSize = size;
                }
            }
        }

        final int previewWidth = bestSize.width;
        final int previewHeight = bestSize.height;

        ViewGroup.LayoutParams layoutParams = getLayoutParams();
        layoutParams.height = previewHeight;
        layoutParams.width = previewWidth;
        setLayoutParams(layoutParams);

        params.setPreviewFormat(ImageFormat.NV21);
        mCamera.setParameters(params);

        int size = previewWidth * previewHeight * 
            ImageFormat.getBitsPerPixel(params.getPreviewFormat()) / 8;
        mBuffer = new byte[size];
        mCamera.addCallbackBuffer(mBuffer);

        mCamera.setPreviewDisplay(mHolder);
        mCamera.startPreview();
    }

    public void onPreviewFrame(byte[] data, Camera camera) {
        CameraView.this.notify();
    }

    public void run() {
        mThreadRun = true;
        while (mThreadRun) {
            synchronized (this) {
                this.wait();
                processFrame(mBuffer); // convert to RGB and rotate - not shown
            }
            // Request a new frame from the camera by putting 
            // the buffer back into the queue
            mCamera.addCallbackBuffer(mBuffer);
        }

        mHolder.removeCallback(this);
        mCamera.stopPreview();
        mCamera.setPreviewCallback(null);
        mCamera.release();
        mCamera = null;
    }

    public void surfaceDestroyed(SurfaceHolder holder) {
        mThreadRun = false;
    }
}

On all devices, the camera preview displays properly, and on most (emulator, Samsung Galaxy S3, etc.) the data stored in mBuffer is also correct (after NV21 to RGB conversion and rotation, of course). However, a number of devices do not supply the correct data in onPreviewFrame. I'm sure that the data is being converted to RGB correctly after it's received, so the problem appears to be in the raw data supplied to mBuffer. I've noticed this bug report relating to the YV12 (alias YUV420p) camera preview format, but I'm using the old default, NV21 (alias YUV420sp), which must be supported according to the compatibility standard (see 7.5.3.2, bottom of page 29).

For example, for this scene (shown here in Camera Preview on the Samsung Galaxy Tab 2):

the data passed to mBuffer on the Tab 2 looks like:

and on the Motorola Droid 4 looks like:

What is the correct way to get Android camera preview data across all devices?

Edit: for processFrame(), I used OpenCV to convert to RGB and rotate. See this answer and this answer.

回答1:

The only problem was that I didn't set the preview width and height:

params.setPreviewSize(previewWidth, previewHeight);
mCamera.setParameters(params);

This meant that the height and width I allocated for the array (proportional to previewWidth * previewHeight) tended to be a lot larger than the size of the actual data being returned (proportional to the default preview width and preview height). On some phones, the default was the same size as previewWidth and previewHeight, so there was no issue.

回答2:

You can also try this

public void takeSnapPhoto() {
camera.setOneShotPreviewCallback(new Camera.PreviewCallback() {
    @Override
    public void onPreviewFrame(byte[] data, Camera camera) {
        Camera.Parameters parameters = camera.getParameters();
        int format = parameters.getPreviewFormat();
        //YUV formats require more conversion
        if (format == ImageFormat.NV21 || format == ImageFormat.YUY2 || format == ImageFormat.NV16) {
            int w = parameters.getPreviewSize().width;
            int h = parameters.getPreviewSize().height;
            // Get the YuV image
            YuvImage yuv_image = new YuvImage(data, format, w, h, null);
            // Convert YuV to Jpeg
            Rect rect = new Rect(0, 0, w, h);
            ByteArrayOutputStream output_stream = new ByteArrayOutputStream();
            yuv_image.compressToJpeg(rect, 100, output_stream);
            byte[] byt = output_stream.toByteArray();
            FileOutputStream outStream = null;
            try {
                // Write to SD Card
                File file = createFileInSDCard(FOLDER_PATH, "Image_"+System.currentTimeMillis()+".jpg");
                //Uri uriSavedImage = Uri.fromFile(file);
                outStream = new FileOutputStream(file);
                outStream.write(byt);
                outStream.close();
            } catch (FileNotFoundException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            } finally {
            }
        }
    }
});}

回答3:

From working on Barcode Scanner, which relies heavily on preview data, I feel like I've seen every bug under the sun. My suggestion is simply to not call setPreviewFormat() and let it use the default. The default is what you want here fortunately. There seem to be fewer devices that fail to get the default right, than device that balls up the call to setPreviewFormat(). Try that at least, may or may not be it.