How to install tensorflow from source for Mac

2019-07-31 01:30发布

问题:

I have followed the steps here to install tensorflow from sources on Mac.

The latest problem on no effect is in the updated section at the bottom of this page.

The most immediate problem (solved now) I have was I can't find whl file to do sudo pip install /tmp/tensorflow_pkg/tensorflow-1.0.1-py2-none-any.whl

Before it, although I didn't get error, I have got tons of warnings when running bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package. The following is only the last part of the logging message with warnings.

arning: class 'WorkerEnv' was previously declared as a struct [-Wmismatched-tags]
class WorkerEnv;
^
./tensorflow/core/distributed_runtime/worker_env.h:37:8: note: previous use is here
struct WorkerEnv {
       ^
./tensorflow/core/distributed_runtime/worker.h:28:1: note: did you mean struct here?
class WorkerEnv;
^~~~~
struct
In file included from tensorflow/core/distributed_runtime/rpc/grpc_worker_service.cc:16:
./tensorflow/core/distributed_runtime/rpc/grpc_worker_service.h:36:8: warning: 'tensorflow::GrpcWorker::RecvTensorAsync' hides overloaded virtual function [-Woverloaded-virtual]
  void RecvTensorAsync(CallOptions* opts, const RecvTensorRequest* request,
       ^
./tensorflow/core/distributed_runtime/worker.h:72:8: note: hidden overloaded virtual function 'tensorflow::Worker::RecvTensorAsync' declared here: type mismatch at 3rd parameter ('tensorflow::TensorResponse *' vs '::grpc::ByteBuffer *')
  void RecvTensorAsync(CallOptions* opts, const RecvTensorRequest* request,
       ^
2 warnings generated.
INFO: From Linking tensorflow/core/kernels/libconv_ops.pic.lo:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/local-py3-opt/bin/tensorflow/core/kernels/_objs/conv_ops/tensorflow/core/kernels/conv_ops_using_gemm.pic.o has no symbols
INFO: From Linking tensorflow/core/kernels/libfft_ops.pic.lo:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/local-py3-opt/bin/tensorflow/core/kernels/_objs/fft_ops/tensorflow/core/kernels/fft_ops.pic.o has no symbols
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: warning for library: bazel-out/local-py3-opt/bin/tensorflow/core/kernels/libfft_ops.pic.lo the table of contents is empty (no object file members in the library define global symbols)
INFO: From Linking tensorflow/core/libsycl_runtime.pic.lo:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/local-py3-opt/bin/tensorflow/core/_objs/sycl_runtime/tensorflow/core/common_runtime/sycl/sycl_allocator.pic.o has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/local-py3-opt/bin/tensorflow/core/_objs/sycl_runtime/tensorflow/core/common_runtime/sycl/sycl_device.pic.o has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/local-py3-opt/bin/tensorflow/core/_objs/sycl_runtime/tensorflow/core/common_runtime/sycl/sycl_device_context.pic.o has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/local-py3-opt/bin/tensorflow/core/_objs/sycl_runtime/tensorflow/core/common_runtime/sycl/sycl_device_factory.pic.o has no symbols
warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: warning for library: bazel-out/local-py3-opt/bin/tensorflow/core/libsycl_runtime.pic.lo the table of contents is empty (no object file members in the library define global symbols)
INFO: From Linking tensorflow/core/libgpu_runtime.pic.lo:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/local-py3-opt/bin/tensorflow/core/_objs/gpu_runtime/tensorflow/core/common_runtime/gpu/gpu_device.pic.o has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: file: bazel-out/local-py3-opt/bin/tensorflow/core/_objs/gpu_runtime/tensorflow/core/common_runtime/gpu/gpu_device_factory.pic.o has no symbols
INFO: From Linking tensorflow/python/gen_math_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/factorization/gen_gen_clustering_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_sparse_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_user_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/nccl/gen_nccl_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_nn_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_linalg_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_parsing_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/factorization/gen_gen_factorization_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/ffmpeg/gen_decode_audio_op_py_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_random_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/tensor_forest/gen_gen_tensor_forest_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_io_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_resource_variable_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_functional_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_logging_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_script_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_set_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_cloud_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/layers/gen_bucketization_op_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_test_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_state_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_ctc_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_data_flow_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_sdca_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/input_pipeline/gen_input_pipeline_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/_pywrap_tensorflow.so:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/image/gen_image_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_string_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/layers/gen_sparse_feature_cross_op_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/cudnn_rnn/gen_cudnn_rnn_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_candidate_sampling_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_image_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/ffmpeg/gen_encode_audio_op_py_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_training_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_array_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/python/gen_control_flow_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
INFO: From Linking tensorflow/contrib/framework/gen_gen_variable_ops_py_wrappers_cc [for host]:
clang: warning: argument unused during compilation: '-pthread'
Target //tensorflow/tools/pip_package:build_pip_package up-to-date:
  bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 2177.306s, Critical Path: 2134.22s

Then I ran bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg, it seems make no error with the following output in the console:

(dlnd-tf-lab)  ->bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Wed Apr 12 16:09:42 CST 2017 : === Using tmpdir: /var/folders/gz/ch3n2mp51m9386sytqf97s6w0000gn/T/tmp.XXXXXXXXXX.WnIjTd5J
~/Downloads/tensorflow/bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles ~/Downloads/tensorflow
~/Downloads/tensorflow
/var/folders/gz/ch3n2mp51m9386sytqf97s6w0000gn/T/tmp.XXXXXXXXXX.WnIjTd5J ~/Downloads/tensorflow
Wed Apr 12 16:09:45 CST 2017 : === Building wheel
~/Downloads/tensorflow
Wed Apr 12 16:10:02 CST 2017 : === Output wheel file is in: /tmp/tensorflow_pkg

Then I could not find the whl file, do you know where can I find it?

Thanks!


Now, I found the whl file, but there was no change to tensorflow after running the following code, also I got a warning. will this warning cause the problem for not making any effect on tensorflow?

(dlnd-tf-lab)  ->sudo pip install tmp/tensorflow_pkg/tensorflow-1.0.1-cp35-cp35m-macosx_10_9_x86_64.whl
Password:
The directory '/Users/Natsume/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/Natsume/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: tensorflow==1.0.1 from file:///Users/Natsume/Downloads/tmp/tensorflow_pkg/tensorflow-1.0.1-cp35-cp35m-macosx_10_9_x86_64.whl in /Users/Natsume/miniconda2/envs/dlnd-tf-lab/lib/python3.5/site-packages
Requirement already satisfied: numpy>=1.11.0 in /Users/Natsume/miniconda2/envs/dlnd-tf-lab/lib/python3.5/site-packages (from tensorflow==1.0.1)
Requirement already satisfied: six>=1.10.0 in /Users/Natsume/miniconda2/envs/dlnd-tf-lab/lib/python3.5/site-packages (from tensorflow==1.0.1)
Requirement already satisfied: wheel>=0.26 in /Users/Natsume/miniconda2/envs/dlnd-tf-lab/lib/python3.5/site-packages (from tensorflow==1.0.1)
Requirement already satisfied: protobuf>=3.1.0 in /Users/Natsume/miniconda2/envs/dlnd-tf-lab/lib/python3.5/site-packages (from tensorflow==1.0.1)
Requirement already satisfied: setuptools in /Users/Natsume/miniconda2/envs/dlnd-tf-lab/lib/python3.5/site-packages (from protobuf>=3.1.0->tensorflow==1.0.1)

However, when I tried the code provided in the link above to check whether everything is working as expected, I still got the following warnings indicating there is no effect. Could anyone help me here? Thanks

W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
b'Hello, TensorFlow!'

回答1:

To summarize my solution

  1. follow the official guide of installing tensorflow from source with the link provided in the question;

  2. Be patient with installing brew and bazel: to install bazel, go to bazel. Make sure you have them up to the latest version. It was very slow to install or upgrade, but it will get you there, patience.

  3. follow the official instruction all the way to the step of ./configure, just use default options or no to set everything;

  4. Now the last second step: official instruction tells you to bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg. However, in my case, I can find /tmp/tensorflow_pkg in my Mac. Fortunately, it will be seen if you run bazel-bin/tensorflow/tools/pip_package/build_pip_package Your_Full_Path/tmp/tensorflow_pkg. Note, tmp folder and tensorflow folder are in the same directory or same level.

  5. go to the directory where tmp and tensorflow are stored, run pip install /tmp/tensorflow_pkg/tensorflow-1.2.0rc0-cp35-cp35m-macosx_10_9_x86_64.whl

  6. Now you did it.



回答2:

As Daniel stated in the comments, utilizing the full path after TensorFlow successfully builds worked to access and install the wheel file.