How to add external header files during bazel/tens

问题:

I am trying to add external header file (like OpenCL header file) for some experimentation for tensorflow. I tried to add this into BUILD file under tensorflow/core/BUILD file:

# This includes implementations of all kernels built into TensorFlow.
cc_library(
    name = "all_kernels",
    visibility = ["//visibility:public"],
    copts = tf_copts() + ["-Ithird_party/include"],    <==== this is the line I added

I have also created a softlink in this directory to the location of these header files from OpenCL driver (under tensorflow/third_party) too (like ln -s /opt/opencl/ ) but it still complains that it has not found that header file.

If I add external header file directly (like /opt/opencl/CL/) it complains that external files cannot be included (or some such thing).

I do not have root password to copy these header files into /usr/include/ too.

Can someone explain how exactly to do external header files into tensorflow for building?

Thanks for any quick help.

回答1:

I've faced with the similar problem when I built TensorFlow with Intel MKL and had to add MKL headers. My solution is the following:

1. Create symlink to your headers into third_party folder, like:

   <your tensorflow folder>/third_party/opencl/include -> /opt/OpenCL/include
   

   with command:

   ln -s /opt/OpenCL/include <your tensorflow folder>/third_party/opencl
   

2. Create simple BUILD file into <your tensorflow folder>/third_party/opencl folder:

   cc_library(
       name = "opencl",
       hdrs = glob(["include/CL/*.h"]),
       visibility = ["//visibility:public"],
   )
   

3. Add deps into target library:

   cc_library(
       name = "all_kernels",
       visibility = ["//visibility:public"],
       copts = tf_copts() + ["-Ithird_party/opencl/include"],
       deps = [
           "//third_party/opencl", 
           ...
       ],
   )
   

4. Don't forget to add compiler options either into target library as shown above or just as a flag to bazel:

    bazel build --copt="-Ithird_party/opencl/include" ...
   

回答2:

You have to add the files as a dependency of this rule.

IIUC, you have the following structure:

tensorflow/
  BUILD
  WORKSPACE
  tensorflow/
    core/
      BUILD
  third_party/
    include -> /opt/opencl/CL # or something like that

You want to expose the .h files in a way Bazel can understand/depend on, so open up the tensorflow/BUILD file and add the following:

cc_library(
    name = "opencl",
    hdrs = glob(["third_party/include/*.h"]),
    visibility = ["//visibility:public"],
)

This creates a C++ library from the .h files under third_party/include, which can be depended on from anywhere in the source tree.

Now go to your tensorflow/core/BUILD file and add a dependency to the cc_library there:

cc_library(
    name = "all_kernels",
    visibility = ["//visibility:public"],
    copts = tf_copts() + ["-Ithird_party/include"],
    deps = [
        "//:opencl", 
        # plus any other deps
    ],
)

Setting copts just changes the flags when gcc is run. Adding //:opencl to the dependencies tells Bazel to make those files available when gcc is run.

回答3:

Bazel tries to be very strict about making sure its builds only include files that it knows about, to try to make sure that they are reproducible. Unfortunately that can make it tough to experiment with. The correct way to solve the problem is to create a BUILD file and rule for the headers you want to include. You may be able to hack something by messing around with the bazel-* generated folders too, but I don't recommend it.