Android camera2 capture burst is too slow

Question

I am trying to modify the android-Camera2Basic code to capture a burst of pictures. However, I can't get the delay between pictures any faster than 200-300ms on my Nexus 5, running L 5.0.1.

I've tried a bunch of things, but this is the most basic. This is the only part of the Camera2Basic code that I've modified. My preview TextureView is only 50x50dp, but that shouldn't matter, right?

For what it's worth, this code only has delays around 50-100ms on my Nexus 6, with L 5.1.

private void captureStillPicture() {
    try {
        List<CaptureRequest> captureList = new ArrayList<CaptureRequest>();
        mPreviewRequestBuilder.addTarget(mImageReader.getSurface());

        for (int i=0;i<10;i++) {
            captureList.add(mPreviewRequestBuilder.build());
        }

        mCaptureSession.stopRepeating();
        mCaptureSession.captureBurst(captureList, cameraCaptureCallback, null);
        mPreviewRequestBuilder.removeTarget(mImageReader.getSurface());
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

CameraCaptureSession.CaptureCallback cameraCaptureCallback = new CameraCaptureSession.CaptureCallback() {
    @Override
    public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request,
            TotalCaptureResult result) {
        Log.d("camera","saved");
        mPictureCounter++;
        if (mPictureCounter >= 10)
            unlockFocus();
    }
};

Answer 1

Based on this PcMag article: http://www.pcmag.com/article2/0,2817,2428017,00.asp The Nexus 5 burst mode lets you take 3 pictures per second. So getting much less than 333ms between photos is highly unlikely.

Do you have a source for information that suggests the Nexus 5 burst should be faster?

Answer 2

The issue you are running into is an artifact of the image output format you have requested. The JPEG encoding process puts a large stall time on the camera pipeline, so there is a lot of downtime between when one exposure ends and the next begins while this encoding happens.

The 30fps rate that is quoted can be achieved by setting the output image format on the ImageReader as YUV, since that is a more "native" output for the camera. This would be the way to store the images as they are captured, and then you would have to do JPEG encoding afterwards, separate of the camera's inline processing.

For example, on the Nexus 5 the output stall time for JPEG encoding is 243ms, which you have been observing. For YUV_420_888 output, it is 0ms. Likewise, because of their large size, RAW_SENSOR encoding introduces a stall time of 200ms.

Note also that even if you remove the stall time obstacle by choosing a "faster" format, there is still a minimum frame time, depending on the output image size. But for a Nexus 5's full resolution output, this is 33ms, which is what you were expecting.

The relevant information is in the camera metadata's StreamConfigurationMap object, here. Check out the getOutputStallDuration(int format, Size size) and getOutputMinFrameDuration(int format, Size size) methods for confirmation.

Answer 3

Try to set following capture request parameters

requestBuilder = camDevice
        .createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE);

requestBuilder.set(CaptureRequest.EDGE_MODE,
        CaptureRequest.EDGE_MODE_OFF);
requestBuilder.set(
        CaptureRequest.LENS_OPTICAL_STABILIZATION_MODE,
        CaptureRequest.LENS_OPTICAL_STABILIZATION_MODE_ON);
requestBuilder.set(
        CaptureRequest.COLOR_CORRECTION_ABERRATION_MODE,
        CaptureRequest.COLOR_CORRECTION_ABERRATION_MODE_OFF);
requestBuilder.set(CaptureRequest.NOISE_REDUCTION_MODE,
        CaptureRequest.NOISE_REDUCTION_MODE_OFF);
requestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER,
        CaptureRequest.CONTROL_AF_TRIGGER_CANCEL);

requestBuilder.set(CaptureRequest.CONTROL_AE_LOCK, true);
requestBuilder.set(CaptureRequest.CONTROL_AWB_LOCK, true);

I am not sure about how fast info comes into CameraCaptureSession.CaptureCallback. It's not have image data, and it could be called before or after ImageReader.OnImageAvailableListener. Try to measure time between ImageReader.OnImageAvailableListener calls. And don't forget to read images and release them, because new images are not available if buffer is filled and images not released. For example:

private class imageAvailableListener implements
            ImageReader.OnImageAvailableListener {
        @Override
        public void onImageAvailable(ImageReader ir) {
            Log.i(TAG, "Time = " + System.currentTimeMillis());
            Image im = ir.acquireNextImage();
            im.close();
        }
    }

ImageReader mImageReader = ImageReader.newInstance(imageReaderWidth,
                    imageReaderHeight, ImageFormat.YUV_420_888, 2);
mImageReader.setOnImageAvailableListener(
                    new imageAvailableListener(), null);