How to use OpenCV 4 in native C++ of Flutter? [clo

Question

Closed. This question needs to be more focused. It is not currently accepting answers.

---

Want to improve this question? Update the question so it focuses on one problem only by editing this post.

Closed 7 days ago.

Improve this question

I need to write some C++ code which uses OpenCV, and the Flutter code will call those C++ code.

There are tutorials about writing C++ with Flutter, but I cannot find any up-to-date and easy-to-deploy solution about working with OpenCV. How to do that?

Answer 1

Here is my solution.

Features

1. Works for both Android and iOS.
2. Use static linking instead of dynamic linking. (Thus code size is much smaller.)
3. Up-to-date at 2020.07.28. (Since those APIs change rapidly and many articles are a little bit old.)

Getting Started

NOTE: If you already have an app, you can skip this section :) This section assumes that you have no code at all.

Sample code can be downloaded from here.

step 0: Ensure you have Flutter environment, and have followed the official "writing C++ with Flutter" tutorial.

NOTE: It is a must to follow the step of "On iOS, you need to tell Xcode to statically link the file: ...". Otherwise, at our last step iOS will complain the symbol cannot be found.

step 1: Write whatever code you like using OpenCV. For instance, I change ios/Classes/native_add.cpp to the following silly code, which is almost identical as in the official tutorial:

#include <stdint.h>
#include <opencv2/core.hpp>

extern "C" __attribute__((visibility("default"))) __attribute__((used))
int32_t native_add(int32_t x, int32_t y) {
    cv::Mat m = cv::Mat::zeros(x, y, CV_8UC3);
    return m.rows + m.cols;
}

Android

Step 0: Download the Android OpenCV sdk from the official website. Say I put it in /Users/tom/Others/OpenCVRelease/OpenCV-android-sdk in my desktop.

Step 1.1: Change the android/CMakeLists.txt to the following. NOTE: First change the OPENCV_BASE_DIR to your folder.

cmake_minimum_required(VERSION 3.4.1)

# TODO please change me!
set(OPENCV_BASE_DIR "TODO PLEASE PUT YOUR DIR HERE!!!")

set(OPENCV_INCLUDE_DIR "${OPENCV_BASE_DIR}/sdk/native/jni/include/")
set(OPENCV_STATIC_LIB_DIR "${OPENCV_BASE_DIR}/sdk/native/staticlibs/${ANDROID_ABI}")
set(OPENCV_3RDPARTY_STATIC_LIB_DIR "${OPENCV_BASE_DIR}/sdk/native/3rdparty/libs/${ANDROID_ABI}")

include_directories(${OPENCV_INCLUDE_DIR})

find_library(log-lib log)

add_library(highgui STATIC IMPORTED)
set_target_properties(highgui PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_highgui.a)

add_library(calib3d STATIC IMPORTED)
set_target_properties(calib3d PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_calib3d.a)

add_library(core STATIC IMPORTED)
set_target_properties(core PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_core.a)

add_library(dnn STATIC IMPORTED)
set_target_properties(dnn PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_dnn.a)

add_library(flann STATIC IMPORTED)
set_target_properties(flann PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_flann.a)

add_library(imgproc STATIC IMPORTED)
set_target_properties(imgproc PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_imgproc.a)

add_library(videoio STATIC IMPORTED)
set_target_properties(videoio PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_videoio.a)

add_library(imgcodecs STATIC IMPORTED)
set_target_properties(imgcodecs PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_imgcodecs.a)

add_library(features2d STATIC IMPORTED)
set_target_properties(features2d PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_features2d.a)

add_library(ml STATIC IMPORTED)
set_target_properties(ml PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_ml.a)

add_library(photo STATIC IMPORTED)
set_target_properties(photo PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_photo.a)

add_library(shape STATIC IMPORTED)
set_target_properties(shape PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_shape.a)

add_library(objdetect STATIC IMPORTED)
set_target_properties(objdetect PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_objdetect.a)

add_library(stitching STATIC IMPORTED)
set_target_properties(stitching PROPERTIES IMPORTED_LOCATION ${OPENCV_STATIC_LIB_DIR}/libopencv_stitching.a)

include(AndroidNdkModules)
android_ndk_import_module_cpufeatures()

set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
set(THREADS_PREFER_PTHREAD_FLAG TRUE)
find_package(Threads REQUIRED)

add_library(tbb STATIC IMPORTED)
set_target_properties(tbb PROPERTIES IMPORTED_LOCATION ${OPENCV_3RDPARTY_STATIC_LIB_DIR}/libtbb.a)

add_library(tegra_hal STATIC IMPORTED)
set_target_properties(tegra_hal PROPERTIES IMPORTED_LOCATION ${OPENCV_3RDPARTY_STATIC_LIB_DIR}/libtegra_hal.a)

add_library(ittnotify STATIC IMPORTED)
set_target_properties(ittnotify PROPERTIES IMPORTED_LOCATION ${OPENCV_3RDPARTY_STATIC_LIB_DIR}/libittnotify.a)

add_library(native_with_opencv

        # Sets the library as a shared library.
        SHARED

        # Provides a relative path to your source file(s).
        ../ios/Classes/native_add.cpp
        )

target_link_libraries(native_with_opencv
        ${log-lib}
        core
        # note: You can import whatever other modules you like (e.g. dnn)
        tbb # note: need to be placed *after* "core"
        cpufeatures
        ittnotify
        tegra_hal # NOTE if still have error, check abiFilters, since tegra does *not* exist in x86.
        Threads::Threads
        -lz
        )

Of course, the lib/native_with_opencv.dart should change the .so file name to "libnative_with_opencv.so".

Step 1.2: Change the android/build.gradle as following:

android {
    ...
    defaultConfig {
        ...
        // [[[CHANGE 1: Make minSdkVersion bigger]]]
        // see https://github.com/opencv/opencv/issues/14419
        minSdkVersion 21

        // [[[CHANGE 2: Add these flags and filters]]]
        externalNativeBuild {
            cmake {
                cppFlags "-frtti -fexceptions -std=c++11"
                abiFilters 'armeabi-v7a', 'arm64-v8a'
            }
        }
        ...
    }
    ...
}

Of course, the minSdkVersion in your actual project (native_with_opencv/example/android/app/build.gradle) should also change to 21.

**Done! **Compile and enjoy it (and go to the next section for iOS)! If you see 1 + 2 == 3, then everything is fine.

Bonus: If you build in release mode and look at the apk size, you will see our .so file is less than 1MB. Thus static linking and file size reduction does work :)

iOS

Step 0: In ios/native_with_opencv.podspec, add:

  s.static_framework = true
  s.dependency 'OpenCV', '~> 4.1'

Step 1: Compile and enjoy. NOTE: You may first need to run pod install under native_with_opencv/example/ios to let Cocoapod initialize.

---

(Optional) Explanations of how does the Android configuration work: (1) Originally, I just link the core, but there are hundreds of linking errors. Then I search and fix for each group of them. For instance, error: undefined reference to 'carotene_o4t::...' means I need to link with libtegra_hal, thus I add several lines. (2) Strangely, the tbb should be put after core, otherwise it still does not link. (3) The abiFilters is needed, since tegra_hal does not support x86 (thus no .a file exists). (4) minSdkVersion needs to be raised up, otherwise fegetenv will not be found.