Building Tensorflow Java Maven dependency for uplo

问题:

I'm doing an in-house build of Tensorflow with CPU optimizations (AVXx, SSE4.x, etc.), so I can use it in Java micro service, which so far is successful, but I have to manually copy files. I want to produce a Maven artifact, which will be uploaded to our internal Maven repo. I'm trying to compile something of these scripts(ci_build) and docs(maven, java ), but I was wondering if there is less hack-ish way? So far was able to build maven dependency, containing only Java API jar with:

git/tensorflow/tensorflow/java/maven$ mvn install:install-file \
-Dfile=../../../bazel-bin/tensorflow/java/libtensorflow.jar \
-DpomFile=../../../bazel-bin/tensorflow/java/pom.xml

The mvn command in the question is executed after building with bazel: bazel build -c opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni

Unfortunately *.so files are missing. I can copy them manually to tensorflow/tensorflow/java/maven/..., before executing mvn install but I prefer not to do that. :)

回答1:

Extract from my Docker image for building Tensorflow. After TF has been build produce a Maven artifact:

Build jar and src jar of Java API and tar archive of the native *.so binaries

    RUN bazel build -s -c opt $TF_GCC_FLAGS \
                               //tensorflow/java:libtensorflow.jar \
                               //tensorflow/java:libtensorflow-src.jar \
                               //tensorflow/tools/lib_package:libtensorflow_jni.tar.gz

WORKDIR /tf/tensorflow-${TENSORFLOW_VERSION}/tensorflow/java/maven

Copy binaries to maven folder, so that Maven can build artifacts

ENV NATIVE_DIR="/tf/tensorflow-${TENSORFLOW_VERSION}/tensorflow/java/maven/libtensorflow_jni/src/main/resources/org/tensorflow/native/linux-x86_64"
RUN mkdir -p "${NATIVE_DIR}" && \
    tar -zxvf ${TF_ROOT}/bazel-bin/tensorflow/tools/lib_package/libtensorflow_jni.tar.gz -C ${NATIVE_DIR} && \
    cd libtensorflow && \
    jar -xvf ${TF_ROOT}/bazel-bin/tensorflow/java/libtensorflow-src.jar && \
    rm -rf META-INF

Updated timestamps seem to be required to get Maven to pick up the file.

Skip PGP signing of the binaries, since we don't have PGP key.

RUN touch ${NATIVE_DIR}/* && \
    mvn versions:set -DnewVersion="${TENSORFLOW_VERSION}-cpu-optimized" && \
    mvn package -Dgpg.skip=true

CMD mvn install -Dgpg.skip=true