Building OpenCV for Android and using it with the

2019-03-11 20:50发布

问题:

Context : I am currently developing an app on Android Studio for the Moverio BT 200 augmented reality glasses. I am using OpenCV, and specifically, the arUco module of the library. This module has to be used with the NDK. Also, it is not on the stable release, so I compiled the library myself (using this guide : https://zami0xzami.wordpress.com/2016/03/17/building-opencv-for-android-from-source/). The build of the library went well. After that, I made a android studio project (customOCVtest). I did it the way I always do when using OpenCV with Android Studio and the NDK, except this time it was with the custom build. I checked if the library was correctly loaded :

private static final String OCVdevTAG = "OCVmainAct";

static {
    System.loadLibrary("native-lib");
    if(!OpenCVLoader.initDebug()) {
        Log.d(OCVdevTAG, "OpenCV not loaded");
    } else {
        Log.d(OCVdevTAG, "OpenCV loaded");
    }
}

The library is indeed loaded when I build.

Now comes the problem : When I try to actually use the arUco module in my native code (here is my native-lib.cpp) :

#include <jni.h>
#include <string>
#include <opencv2/aruco.hpp>

extern "C" {
jstring
Java_com_jambonsama_customocvtest_MainActivity_stringFromJNI(
    JNIEnv *env,
    jobject /* this */) {
std::string hello = "Hello from C++";
cv::Ptr<cv::aruco::Dictionary> dict = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
cv::Mat marker;
cv::aruco::drawMarker(dict, 25, 200, marker, 1);
return env->NewStringUTF(hello.c_str());
}
}

the gradle sync works, but I can't build. I get the following error :

Error:FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:externalNativeBuildDebug'.
> Build command failed.
Error while executing 'C:\Users\JambonSama\AppData\Local\Android\Sdk\cmake\3.6.3155560\bin\cmake.exe' with arguments {--build C:\Users\JambonSama\AndroidStudioProjects\customOCVtest\app\.externalNativeBuild\cmake\debug\mips64 --target native-lib}
[1/1] Linking CXX shared library ..\..\..\..\build\intermediates\cmake\debug\obj\mips64\libnative-lib.so
FAILED: cmd.exe /C "cd . && C:\Users\JambonSama\AppData\Local\Android\sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe  -target mips64el-none-linux-android -gcc-toolchain C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64 --sysroot=C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/platforms/android-21/arch-mips64 -fPIC -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security -fno-exceptions -fno-rtti -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wa,--noexecstack -Wformat -Werror=format-security -fno-exceptions -fno-rtti  -O0 -fno-limit-debug-info -O0 -fno-limit-debug-info  -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -shared -Wl,-soname,libnative-lib.so -o ..\..\..\..\build\intermediates\cmake\debug\obj\mips64\libnative-lib.so CMakeFiles/native-lib.dir/src/main/cpp/native-lib.cpp.o  C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_aruco.a -llog -lm "C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/libs/mips64/libgnustl_static.a" && cd ."
C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Users/JambonSama/AppData/Local/Android/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/windows-x86_64/lib/gcc/mips64el-linux-android/4.9.x/../../../../mips64el-linux-android/bin\ld: C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a(alloc.cpp.o): Relocations in generic ELF (EM: 40)
  C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a: error adding symbols: File in wrong format
  clang++.exe: error: linker command failed with exit code 1 (use -v to see invocation)
  ninja: build stopped: subcommand failed.


* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

In particular, this error message tells me that I tried to build with the following argument :

{--build C:\Users\JambonSama\AndroidStudioProjects\customOCVtest\app\.externalNativeBuild\cmake\debug\mips64 --target native-lib}

This argument is automatically generated by AS, and I can't find the file where it is passed. I believe that if I can find it, I can build for arm (which is what I want), instead of mips. The thing is, I can't find it. (And I'm not even sure that is indeed what I'm looking for. I'm just trying to make my project build for my glasses.)

Here after is my CMakeLists.txt :

# Sets the minimum version of CMake required to build the native
# library. You should either keep the default value or only pass a
# value of 3.4.0 or lower.

cmake_minimum_required(VERSION 3.4.1)

# Creates and names a library, sets it as either STATIC
# or SHARED, and provides the relative paths to its source code.
# You can define multiple libraries, and CMake builds it for you.
# Gradle automatically packages shared libraries with your APK.

add_library( # Sets the name of the library.
             native-lib

             # Sets the library as a shared library.
             SHARED

             # Provides a relative path to your source file(s).
             # Associated headers in the same location as their source
             # file are automatically included.
             src/main/cpp/native-lib.cpp )

include_directories(C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/jni/include)

# Searches for a specified prebuilt library and stores the path as a
# variable. Because system libraries are included in the search path by
# default, you only need to specify the name of the public NDK library
# you want to add. CMake verifies that the library exists before
# completing its build.

find_library( # Sets the name of the path variable.
              log-lib

              # Specifies the name of the NDK library that
              # you want CMake to locate.
              log )

# Specifies libraries CMake should link to your target library. You
# can link multiple libraries, such as libraries you define in the
# build script, prebuilt third-party libraries, or system libraries.


target_link_libraries( # Specifies the target library.
                       native-lib
                       C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_core.a
                       C:/Libs/opencv_src/opencv/platforms/build_android_arm/install/sdk/native/libs/armeabi-v7a/libopencv_aruco.a

                       # Links the target library to the log library
                       # included in the NDK.
                       ${log-lib} )

here is my app build.gradle :

apply plugin: 'com.android.application'

android {
    compileSdkVersion 25
    buildToolsVersion "25.0.1"
    defaultConfig {
        applicationId "com.jambonsama.customocvtest"
        minSdkVersion 14
        targetSdkVersion 25
        versionCode 1
        versionName "1.0"
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
        externalNativeBuild {
            cmake {
                cppFlags ""
            }
        }
    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        }
    }
    externalNativeBuild {
        cmake {
            path "CMakeLists.txt"
        }
    }
    sourceSets { main { jni.srcDirs = ['src/main/jni', 'src/main/jniLibs/'] } }
}

dependencies {
    compile fileTree(include: ['*.jar'], dir: 'libs')
    androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.2', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })
    compile 'com.android.support:appcompat-v7:25.0.1'
    testCompile 'junit:junit:4.12'
    compile project(':openCVLibrary310dev')
}

and my openCVLibrary310dev build.gradle

apply plugin: 'com.android.library'

android {
    compileSdkVersion 25
    buildToolsVersion "23.0.2"

    defaultConfig {
        ndk {
            abiFilter("armeabi-v7a")
        }
        minSdkVersion 14
        targetSdkVersion 25
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
        }
    }
}

What I tried, mainly by just copying stuff that helped other people solve problems that seemed similar to mines :

#set(CMAKE_SYSTEM_NAME Android)

or

#set(CMAKE_TOOLCHAIN_FILE "Toolchain file" CACHE FILEPATH "C:/Libs/opencv_src/opencv/platforms/android.toolchain.cmake")

or

#set( CMAKE_CXX_COMPILER "C:/Libs/android-ndk-r13b/toolchains/arm-linux-androideabi-4.9/prebuilt/windows-x86_64/bin/arm-linux-androideabi-g++.exe" )

in the CMakeLists.txt

and

splits {
    abi {
        enable true
        reset()
        include 'armeabi-v7a'
        universalApk true
    }
}

or

tasks.getByPath(":app:linkMipsDebugRemoteDesktopSharedLibrary").enabled = false

in the build.gradle (I tried both because I didn't know which one was supposed to have it).

Except for the very last bit of code (the task.getByPath...), the sync worked, but I couldn't build. For the last one, I couldn't even sync.

I also tried to rebuild the library for mips, since the architecture seems to be the problem, but I have no idea how to do that.

Where can I find the file where the following argument :

{--build C:\Users\JambonSama\AndroidStudioProjects\customOCVtest\app\.externalNativeBuild\cmake\debug\mips64 --target native-lib}

is passed when building? And / or how do I make Android Studio build my project for arm architecture?

回答1:

So the mips architecture problem can be solved by adding the following code to the gradle.build of app, at the end of the android block :

productFlavors {
    armv7 {
        ndk {
            abiFilter "armeabi-v7a"
        }
    }
    fat
}

But then, other problems arised. Notably, the library is built with the carotene options, which Android Studio doesn't like, and then, the gzlib misses, so Android Studio can't build and / or run. So after MANY trials, I finally achieved what I wanted : cross-compilation of openCV with extra modules for Android on Windows, and building an AS project with said built library so that it works. I documented my whole proccess, from downloading the openCV sources up until running my application on my arm-architected device, I'll just copy paste it here for anyone who needs to do that too. The following instructions are mainly based on this tutorial (thank you to Zamrath Nizam who wrote it). The only problem with that tutorial is that it's probably a little old, so some options / steps need to be altered.

CROSS COMPILE OPENCV FROM SOURCE WITH EXTRA MODULES FOR ANDROID FROM WINDOWS10

* FIRST : prerequisites *

  • download OpenCV and unzip (say at '../opencv-source')
  • download OpenCV extra modules and unzip (say at '../opencv-contrib-source')
  • download Android NDK (say at '../ndk-dir')
  • download CMake (to say '../cmake-dir') and MinGW (to say '../mingw-dir')
  • install Android Studio

* SECOND : configure with CMake *

  • go to '../opencv-source/platforms' and create a folder named 'build_android_arm'
  • in CMake, fill the following first two fields with the following paths :
    1. where is the source code : '../opencv-source'
    2. where to build the binaries : '../opencv-source/platforms/build_android_arm'
  • add the following options via the 'Add Entry' :
    1. ANDROID_NDK, type 'path', value '../ndk-dir'
    2. ANDROID_NDK_HOST_X64, type 'bool', value 1
    3. CMAKE_TOOLCHAIN_FILE, type 'path', value '../opencv-source/platforms/android/android.toolchain.cmake'
  • press 'Configure'
  • choose 'MinGW Makefiles' as the compiler
  • choose 'Specify toolchain file for cross-compiling'
  • press 'Next', and 'Finish'
    • note 1 : As long as you've got no error message, everything's good. Don't worry about the warning messages (like CMake telling you you're doing deprecated stuff).
    • note 2 : If you can't configure at that step, try to read the next few points, it might help.
  • change the following additional options via the 'Search' field :

    1. EXTRA_MODULE_PATH, type 'path', value '../opencv-contrib-source/modules'
    2. WITH_CAROTENE, type 'bool', value 0
    3. BUILD_ZLIB, type 'bool', value 1
      • note : You MUST have configured already once before this step, because the variables created before are regrouped under the group 'Ungrouped entries', while the following variables are CMake automatically generated variables, and NEED to be grouped in the right groups (which is NOT 'Ungrouped entries').
  • verify that the following options are correctly set up (via the 'Search' field) :

    1. ANDROID_NDK_HOST_X64, type 'bool', value 1
    2. CMAKE_MAKE_PROGRAM, type 'path', value '../mingw-dir/bin/mingw32-make.exe'. This option, I actually DIDN'T have it in my CMake config. IF when you press 'Configure', it doesn't work AND you don't have this option, then you should try adding it (but I don't know how). IF when you press 'Configure' you don't have any problem, don't bother about that variable. This check comes from the original tutorial linked above.
    3. CMAKE_TOOLCHAIN_FILE, type 'path', value '../opencv-source/platforms/android/android.toolchain.cmake'
  • press 'Configure'

  • choose 'MinGW Makefiles' as the compiler
  • choose 'Specify toolchain file for cross-compiling'
  • press 'Next', and 'Finish'
  • press 'Generate'

* THIRD : compile with mingw *

  • go to '../mingw-dir/msys/1.0' and run 'msys' bash file
  • navigate to '../opencv-source/platforms/android_arm'
  • run 'mingw32-make' command
  • run 'mingw32-make install' command

* FOURTH : android project *

  • launch Android Studio and create a new project :
    1. select File -> New -> New Project...
    2. fill the 'Application name' with let's say 'cOCV'
    3. check the 'Include C++ Support' box
    4. click 'Next'
    5. choose your min SDK (let's say API 14: Android 4.0 (IceCreamSandwich)
    6. click 'Next', 'Next' and 'Finish'
  • go to File -> New -> Import Module...

    1. provide '../opencv-source/platforms/android_arm/install/sdk/java'
    2. click 'Next' and 'Finish'
  • change the targets in build.gradle file in openCVLibraryXXX (imported module) folder :

    1. compileSdkVersion and targetSdkVersion should be the same, greater or equal to 23
    2. minSdkVersioon should be the same as the one specified when creating the project
  • in the build.gradle file in openCVLibraryXXX (imported module) folder :

    1. replace 'apply plugin: com.android.application' as, 'apply plugin: com.android.library'
    2. remove the line 'applicationId "org.opencv"'
      • note : That last step is done in order to avoid following error : 'unspecified on project app resolves to an APK archive which is not supported as a compilation dependency'.
  • add the imported library as a dependency to the 'app' module in File->'Project Structure'

  • create a jniLibs folder in 'app/src/main' :

    1. right click 'app' in the Android view on the left menu
    2. click 'New-Folder-JNI Folder'
    3. check the 'Change Folder Location'
    4. set the 'Target Source Set as 'app/src/main/jniLibs'
  • copy libraries of OpenCV from 'opencv-source/platforms/android_arm/install/sdk/native/libs' into the newly created folder (jniLibs) inside the 'AndroidStudioProjects/cOCV/app/src/main/jniLibs' folder

    • note : For example, my AndroidStudioProjects folder is located at 'C:\Users\JambonSama\AndroidStudioProjects'.
  • in 'opencv-contrib-source/modules/module_you_desperately_need/CMakeLists.txt', change 'ocv_define_module(module_you_desperately_need opencv some other modules)' for 'ocv_define_module(module_you_desperately_need opencv some other modules WRAP java)'

    • note 1 : THIS STEP MAY BE USELESS, because WRAP java is probably already written in the file.
    • note 2 : If 'python' is written too, it's fine, you let it be written where it is.
  • in CMakeLists.txt, add :

    1. the following two lines after the 'add_library' block and before the 'find_library' block :

       include_directories(../opencv_src/opencv/platforms/build_android_armn/install/sdk/native/jni/include)
       link_directories(../AndroidStudioProjects/cOCVn/app/src/main/jniLibs/armeabi-v7a)
      
      • note : These two lines pretty straight-forwardly give the paths for the include and link directories
    2. the following lines after the 'find_library' block and before the 'target_link_libraries' block :

       file(GLOB PARTYLIBS "../opencv_src/opencv/platforms/build_android_armn/install/sdk/native/3rdparty/libs/armeabi-v7a/*.a")
       file(GLOB CVLIBS  "../opencv_src/opencv/platforms/build_android_armn/install/sdk/native/libs/armeabi-v7a/*.a")
      
      • note : These are for easier linking commands, see next point.
    3. the following paths, AS WRITTEN, in the 'target_link_libraries', after the 'native-lib' variable, and before the '${log-lib}' one :

       ${CVLIBS}
       ${PARTYLIBS}
       ${CVLIBS}
      
      • note : THAT IS THE TRICKY PART : because of cyclical dependencies, you have to write CVLIBS, PARTYLIBS, and then CVLIBS a second time, otherwise you won't stop having linking errors. By now, everything should be linked.
  • in the gradle.build of app, add the following code :

         productFlavors {
             armv7 {
                 ndk {
                     abiFilter "armeabi-v7a"
                 }
             }
             fat
        }
    

    at the end of the android block

    • note : This prevents the mips64 architecture error at build, by specifying once and for all the arm architecture
  • in the native-lib.cpp :

    1. try the following code in the 'stringFromJNI' function (this one is automatically generated by AS when the project is created)

       cv::Mat test;
       cv::VideoCapture camera;
       camera.open(0);
       cv::Ptrcv::aruco::Dictionary dict = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_250);
       cv::Mat marker;
       cv::aruco::drawMarker(dict, 25, 200, marker, 1);
       std::string hello = "Hello from C++";
       return env-NewStringUTF(hello.c_str());
      
    2. don't forget the following includes :

       #include jni.h
       #include string
       #include opencv2/aruco.hpp
       #include opencv2/videoio.hpp
      
      • note : Because the string is not generated before the end of the function, you know, when you test, that if the string is indeed displayed on screen, the function has been gone through without problem, and that you're good (since the above provided code use some modules that are not on the stable realease as of 08/12/2016 (8th of december, I write dates with a dd/mm/yyyy format).
  • time to test :

    1. sync the gradle
    2. build
    3. run on an arm device

You're good to go \ o \\ O // o /

Additional notes :

  • All the gradle.build files and the CMakeLists.txt file can be easily found on the Android View in the menu on the left of the window in AS.
  • Remember if you've got errors navigating in the command window that maybe your '/' should be '\' or the other way round. ('cd' command to navigate in a command window).
  • I run on Windows10.
  • If you're not quite sure what you should see on screen steps by steps, the tutorial I mentioned at the beginning of the answer should give you a good idea what you're supposed to see up until the AS project creation. Then, you can give a look at this tutorial, which explains how to setup OpenCV SDK in Android Studio project FOR THE OFFICIAL RELEASE. The steps I provided differs from BOTH those tutorials, but I put them here, because they can help you know what your screen / windows should look like.