Android Camera Frame processing with MultiThreadin

2019-04-01 21:53发布

问题:

I am using OpenCV,Vuforia and Tesseract(tess-two) for my app. My system works like this:

  1. Vuforia detects the target and sends the full frame to OpenCV (JNI)
  2. OpenCV gets the frame and does some image processing to make it readable for Tesseract (JNI)
  3. Tesseract(tess-two) takes the byte arrays from opencv and does the OCR processings on the image(byte-array)

My problem starts with the 3rd part. Since my aim is to do the ocr on real time camera frames, i am trying to use multi-threading to make UI smooth and not to cause live camera visual to go slow.

I decided to use AsyncTask for that like this :

public void getFromNative(final byte[] imagedata, final int width,final int height,final byte [] typeData, final int typeWidth,final int typeHeight) {

        new AsyncTask<Void, Void, Void>() {

            @Override
            protected Void doInBackground(Void... params) {
                ocr(imagedata, width, height,typeData,typeWidth,typeHeight);                    
                return null;
            }

        }.execute();                
}

It makes the live camera stream smoother than normal, single thread way. But the problem is since its starting so many AsyncTask because of the live camera frames, even when i turn the camera away from the target, it keeps on doing OCR on the previously created AsycnTask and returns results for them one after another. In addition to AsyncTask, i tried IntenService, but the result was the same.

I want to ask if there is a way to solve this issue and make the process more efficient.

Thanks

回答1:

I'm suggesting, that you don't try to recognize every single frame your camera peview fetches. We used something similar to the following onPreviewFrame-method:

public final void onPreviewFrame(final byte[] data, final Camera camera) {
    this.data = data;
    now = Calendar.getInstance().getTimeInMillis();

    if (now - lastRecognitionTime > TIME_BETWEEN_RECOGNITION_STARTS_IN_MILLIS) {
        lastRecognitionTime = now;
        startRecognition(); // here you can start your async task
    }
}

That means the recognition process starts only every few preview frames, depending how you configure TIME_BETWEEN_RECOGNITION_STARTS_IN_MILLIS.