Android Vision - Reduce bar code tracking window

Question

I'm trying to implement Google Visions scanner into an app im working on. By default its a full screen activity and barcodes are tracked over the entire screen.

However, I need a fullscreen camera but with a limited scanning window. For example, the surface view for the camera needs to be fullscreen, it has 2 transparent overlays set to 35% of the screen height top and bottom leaving a 30% viewport in the center.

I have changed the graphic overlay so it will only display in the middle viewport but havent been able to work out how to limit the barcode tracker to the same area.

Any ideas?

Answer 1

Based on @pm0733464's answer with an example of how to get the barcode that's nearest to the center of the preview.

public class CentralBarcodeFocusingProcessor extends FocusingProcessor<Barcode> {

    public CentralBarcodeFocusingProcessor(Detector<Barcode> detector, Tracker<Barcode> tracker) {
        super(detector, tracker);
    }

    @Override
    public int selectFocus(Detector.Detections<Barcode> detections) {

        SparseArray<Barcode> barcodes = detections.getDetectedItems();
        Frame.Metadata meta = detections.getFrameMetadata();
        double nearestDistance = Double.MAX_VALUE;
        int id = -1;

        for (int i = 0; i < barcodes.size(); ++i) {
            int tempId = barcodes.keyAt(i);
            Barcode barcode = barcodes.get(tempId);
            float dx = Math.abs((meta.getWidth() / 2) - barcode.getBoundingBox().centerX());
            float dy = Math.abs((meta.getHeight() / 2) - barcode.getBoundingBox().centerY());

            double distanceFromCenter =  Math.sqrt((dx * dx) + (dy * dy));

            if (distanceFromCenter < nearestDistance) {
                id = tempId;
                nearestDistance = distanceFromCenter;
            }
        }
        return id;
    }
}

Answer 2

The current API doesn't provide a way to limit the scan area. However, you could either filter the results coming out of the detector or crop the image that is passed into the detector.

Filter Results Approach

With this approach, the barcode detector would still scan the full image area, but detected barcodes outside of the target region would be ignored. One way of doing this is to implement a "focusing processor" that receives the results from the detector and only passes at most one barcode to your associated tracker. For example:

public class CentralBarcodeFocusingProcessor extends FocusingProcessor<Barcode> {

  public CentralBarcodeFocusingProcessor(Detector<Barcode> detector, Tracker<Barcode> tracker) {
    super(detector, tracker);
  }

  @Override
  public int selectFocus(Detections<Barcode> detections) {
    SparseArray<Barcode> barcodes = detections.getDetectedItems();
    for (int i = 0; i < barcodes.size(); ++i) {
      int id = barcodes.keyAt(i);
      if (/* barcode in central region */) {
        return id;
      }
    }
    return -1;
  }
}

You'd then associate this processor with the detector like this:

   BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(context).build();
   barcodeDetector.setProcessor(
                new CentralBarcodeFocusingProcessor(myTracker));

Cropping Images Approach

You'd need to crop the image yourself first, before the detector is called. This could be done by implementing a Detector subclass which wraps the barcode detector, crops the images received, and calls the barcode scanner with the cropped images.

For example, you'd make a detector to intercept and crop the image like this:

class MyDetector extends Detector<Barcode> {
  private Detector<Barcode> mDelegate;

  MyDetector(Detector<Barcode> delegate) {
    mDelegate = delegate;
  }

  public SparseArray<Barcode> detect(Frame frame) {
    // *** crop the frame here
    return mDelegate.detect(croppedFrame);
  }

  public boolean isOperational() {
    return mDelegate.isOperational();
  }

  public boolean setFocus(int id) {
    return mDelegate.setFocus(id);
  }
} 

You'd wrap the barcode detector with this one, putting it in between the camera source and the barcode detector:

BarcodeDetector barcodeDetector = new BarcodeDetector.Builder(context)
        .build();
MyDetector myDetector = new MyDetector(barcodeDetector);

myDetector.setProcessor(/* include your processor here */);

mCameraSource = new CameraSource.Builder(context, myDetector)
        .build();