Crop camera preview for TextureView

Question

I have a TextureView with a fixed width and height and I want to show a camera preview inside of it. I need to crop the camera preview so that it doesn't look stretched inside my TextureView. How to do the cropping? If I need to use OpenGL, how to tie the Surface Texture to OpenGL and how to do the cropping with OpenGL?

public class MyActivity extends Activity implements TextureView.SurfaceTextureListener 
{

   private Camera mCamera;
   private TextureView mTextureView;

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_options);

    mTextureView = (TextureView) findViewById(R.id.camera_preview);
    mTextureView.setSurfaceTextureListener(this);
}

@Override
public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
    mCamera = Camera.open();

    try 
    {
        mCamera.setPreviewTexture(surface);
        mCamera.startPreview();
    } catch (IOException ioe) {
        // Something bad happened
    }
}

@Override
public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
    mCamera.stopPreview();
    mCamera.release();
    return true;
}

@Override
public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {
}

@Override
public void onSurfaceTextureUpdated(SurfaceTexture surface)
 {
    // Invoked every time there's a new Camera preview frame
 }
}

Also, after doing the preview correctly, I need to be able to read in real-time the pixels found in the center of the cropped image.

Answer 1

you could manipulate the byte[] data from onPreview().

I think you'll have to:

• put it in an Bitmap
• do the cropping in the Bitmap
• do a little stretching/resizing
• and pass the Bitmap to your SurfaceView

This is not a very performant way. Maybe you can manipulate the byte[] directly, but you will have to deal with picture formats like NV21.

Answer 2

Hey you can simply define the height and width desired by you of the camera preview, so that it fit inside of screen. And for cropping the view, just use the android:scaleX and android:scaleY attributes in xml layout file. Just remember that android:scaleX="1" :> means that it is 100% wide of the original but in your case you need to crop, so make it like ... android:scaleX="1.5" as it would expand the image to 150% of the original and you'll able to see only the cropped part of the image what fits in your layour_height and layout_width parameter. If you want to crop in vertical, make the same changes in code. ~Sahil

Answer 3

Just calculate the aspect ratio, generate a scaling matrix and apply it to the TextureView. Based on the aspect ratio of the surface and the aspect ratio of the preview image, the preview image is cropped on the top and the bottom or left and right. Another solution I found out is that if you open the camera before the SurfaceTexture is available, the preview is already scaled automatically. Just try to move mCamera = Camera.open(); to your onCreate function after you set the SurfaceTextureListener. This worked for me on the N4. With this solution you'll probably get problems when you rotate from portrait to landscape. If you need portrait and landscape support, then take the solution with the scale matrix!

private void initPreview(SurfaceTexture surface, int width, int height) {
    try {
        camera.setPreviewTexture(surface);
    } catch (Throwable t) {
        Log.e("CameraManager", "Exception in setPreviewTexture()", t);
    }

    Camera.Parameters parameters = camera.getParameters();
    previewSize = parameters.getSupportedPreviewSizes().get(0);

    float ratioSurface = width > height ? (float) width / height : (float) height / width;
    float ratioPreview = (float) previewSize.width / previewSize.height;

    int scaledHeight = 0;
    int scaledWidth = 0;
    float scaleX = 1f;
    float scaleY = 1f;

    boolean isPortrait = false;

    if (previewSize != null) {
        parameters.setPreviewSize(previewSize.width, previewSize.height);
        if (display.getRotation() == Surface.ROTATION_0 || display.getRotation() == Surface.ROTATION_180) {
            camera.setDisplayOrientation(display.getRotation() == Surface.ROTATION_0 ? 90 : 270);
            isPortrait = true;
        } else if (display.getRotation() == Surface.ROTATION_90 || display.getRotation() == Surface.ROTATION_270) {
            camera.setDisplayOrientation(display.getRotation() == Surface.ROTATION_90 ? 0 : 180);
            isPortrait = false;
        }
        if (isPortrait && ratioPreview > ratioSurface) {
            scaledWidth = width;
            scaledHeight = (int) (((float) previewSize.width / previewSize.height) * width);
            scaleX = 1f;
            scaleY = (float) scaledHeight / height;
        } else if (isPortrait && ratioPreview < ratioSurface) {
            scaledWidth = (int) (height / ((float) previewSize.width / previewSize.height));
            scaledHeight = height;
            scaleX = (float) scaledWidth / width;
            scaleY = 1f;
        } else if (!isPortrait && ratioPreview < ratioSurface) {
            scaledWidth = width;
            scaledHeight = (int) (width / ((float) previewSize.width / previewSize.height));
            scaleX = 1f;
            scaleY = (float) scaledHeight / height;
        } else if (!isPortrait && ratioPreview > ratioSurface) {
            scaledWidth = (int) (((float) previewSize.width / previewSize.height) * width);
            scaledHeight = height;
            scaleX = (float) scaledWidth / width;
            scaleY = 1f;
        }           
        camera.setParameters(parameters);
    }

    // calculate transformation matrix
    Matrix matrix = new Matrix();

    matrix.setScale(scaleX, scaleY);
    textureView.setTransform(matrix);
}

Answer 4

Provided earlier solution by @Romanski works fine but it scales with cropping. If you need to scale to fit, then use the following solution. Call updateTextureMatrix every time when surface view is changed: i.e. in onSurfaceTextureAvailable and in onSurfaceTextureSizeChanged methods. Also note that this solution relies that activity ignores configuration changes (i.e. android:configChanges="orientation|screenSize|keyboardHidden" or something like that):

private void updateTextureMatrix(int width, int height)
{
    boolean isPortrait = false;

    Display display = getWindowManager().getDefaultDisplay();
    if (display.getRotation() == Surface.ROTATION_0 || display.getRotation() == Surface.ROTATION_180) isPortrait = true;
    else if (display.getRotation() == Surface.ROTATION_90 || display.getRotation() == Surface.ROTATION_270) isPortrait = false;

    int previewWidth = orgPreviewWidth;
    int previewHeight = orgPreviewHeight;

    if (isPortrait)
    {
        previewWidth = orgPreviewHeight;
        previewHeight = orgPreviewWidth;
    }

    float ratioSurface = (float) width / height;
    float ratioPreview = (float) previewWidth / previewHeight;

    float scaleX;
    float scaleY;

    if (ratioSurface > ratioPreview)
    {
        scaleX = (float) height / previewHeight;
        scaleY = 1;
    }
    else
    {
        scaleX = 1;
        scaleY = (float) width / previewWidth;
    }

    Matrix matrix = new Matrix();

    matrix.setScale(scaleX, scaleY);
    textureView.setTransform(matrix);

    float scaledWidth = width * scaleX;
    float scaledHeight = height * scaleY;

    float dx = (width - scaledWidth) / 2;
    float dy = (height - scaledHeight) / 2;
    textureView.setTranslationX(dx);
    textureView.setTranslationY(dy);
}

Also you need the following fields:

private int orgPreviewWidth;
private int orgPreviewHeight;

initialize it in onSurfaceTextureAvailable mathod before calling updateTextureMatrix:

Camera.Parameters parameters = camera.getParameters();
parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_CONTINUOUS_VIDEO);

Pair<Integer, Integer> size = getMaxSize(parameters.getSupportedPreviewSizes());
parameters.setPreviewSize(size.first, size.second);

orgPreviewWidth = size.first;
orgPreviewHeight = size.second;

camera.setParameters(parameters);

getMaxSize method:

private static Pair<Integer, Integer> getMaxSize(List<Camera.Size> list)
{
    int width = 0;
    int height = 0;

    for (Camera.Size size : list) {
        if (size.width * size.height > width * height)
        {
            width = size.width;
            height = size.height;
        }
    }

    return new Pair<Integer, Integer>(width, height);
}

And last thing - you need to correct camera rotation. So call setCameraDisplayOrientation method in Activity onConfigurationChanged method (and also make initial call in onSurfaceTextureAvailable method):

public static void setCameraDisplayOrientation(Activity activity, int cameraId, Camera camera)
{
    Camera.CameraInfo info = new Camera.CameraInfo();
    Camera.getCameraInfo(cameraId, info);
    int rotation = activity.getWindowManager().getDefaultDisplay().getRotation();
    int degrees = 0;
    switch (rotation)
    {
        case Surface.ROTATION_0:
            degrees = 0;
            break;
        case Surface.ROTATION_90:
            degrees = 90;
            break;
        case Surface.ROTATION_180:
            degrees = 180;
            break;
        case Surface.ROTATION_270:
            degrees = 270;
            break;
    }

    int result;
    if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT)
    {
        result = (info.orientation + degrees) % 360;
        result = (360 - result) % 360;  // compensate the mirror
    }
    else
    {  // back-facing
        result = (info.orientation - degrees + 360) % 360;
    }
    camera.setDisplayOrientation(result);

    Camera.Parameters params = camera.getParameters();
    params.setRotation(result);
    camera.setParameters(params);
}

Answer 5

I have just made a working app where I needed to show a preview with x2 of the actual input without getting a pixelated look. Ie. I needed to show the center part of a 1280x720 live preview in a 640x360 TextureView.

Here is what I did.

Set the camera preview to x2 resolution of what I needed:

params.setPreviewSize(1280, 720);

And then scale the texture view accordingly:

this.captureView.setScaleX(2f);
this.captureView.setScaleY(2f);

This runs without any hiccups on small devices.

Answer 6

For real-time manipulations of preview images of the camera, OpenCV for android is perfectly adapted. You'll find every sample needed here : http://opencv.org/platforms/android/opencv4android-samples.html, and as you'll see it works damn well in real time.

Disclaimer : setting up an opencv lib on android can be quite the headache depending on how you are experimented with c++/opencv/the ndk. In all cases, writing opencv code is never simple, but on the other hand it's a very powerful library.