Models: ERROR: C++ compilation of rule '@tf//tensorflow/core/kernels:tensor_array_ops' failed

[ahmed-touati]

I'm trying to get SyntaxNet to run the tests. I run bazel test --linkopt=-headerpad_max_install_names \ syntaxnet/... util/utf8/... But Unfortunately I got the error below, I use mac with python 2.7, bazel 0.2.0, java 1.8.0_91 and clang-602.0.53:

``` ERROR: /private/var/tmp/_bazel_ahmedtouati/973184e4c074c538909febfe48ca7d4e/external/tf/tensorflow/core/kernels/BUILD:596:1: C++ compilation of rule '@tf//tensorflow/core/kernels:tensor_array_ops' failed: cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 -DNDEBUG ... (remaining 79 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
external/tf/tensorflow/core/kernels/tensor_array_ops.cc:244:7: error: return type 'tensorflow::Status' must match previous return type 'const ::tensorflow::Status' when lambda expression has unspecified explicit return type
return Status::OK();
^
In file included from external/tf/tensorflow/core/kernels/tensor_array_ops.cc:23:
In file included from external/tf/third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:
In file included from external/eigen_archive/eigen-eigen-aaa010b0dd40/unsupported/Eigen/CXX11/Tensor:57:
In file included from external/eigen_archive/eigen-eigen-aaa010b0dd40/unsupported/Eigen/CXX11/ThreadPool:53:
external/eigen_archive/eigen-eigen-aaa010b0dd40/unsupported/Eigen/CXX11/src/ThreadPool/EventCount.h:183:10: warning: private field 'pad_' is not used [-Wunused-private-field]
char pad_[128];
^
1 warning and 1 error generated.
INFO: Elapsed time: 421,789s, Critical Path: 401,75s
//syntaxnet:arc_standard_transitions_test NO STATUS
//syntaxnet:beam_reader_ops_test NO STATUS
//syntaxnet:graph_builder_test NO STATUS
//syntaxnet:lexicon_builder_test NO STATUS
//syntaxnet:parser_features_test NO STATUS
//syntaxnet:parser_trainer_test NO STATUS
//syntaxnet:reader_ops_test NO STATUS
//syntaxnet:sentence_features_test NO STATUS
//syntaxnet:shared_store_test NO STATUS
//syntaxnet:tagger_transitions_test NO STATUS
//syntaxnet:text_formats_test NO STATUS
//util/utf8:unicodetext_unittest NO STATUS

Executed 0 out of 12 tests: 12 were skipped. ```

[mrry]

@ebrevdo: This looks like a (perhaps compiler-specific) compiler error in tensor_array_ops.cc. Apparently some confusion between tensorflow::Status and const tensorflow::Status for the return type of a lambda. Can you take a look?

[ebrevdo]

I wish I could replicate this. For now I'll try to make the return type explicit.

[ahmed-touati]

I obtain the same error message when I tried to install tensorflow from souce, by typing bazel build -c opt //tensorflow/tools/pip_package:build_pip_package

ERROR: /Users/ahmedtouati/deeplearning/tensorflow/tensorflow/core/kernels/BUILD:596:1: C++ compilation of rule '//tensorflow/core/kernels:tensor_array_ops' failed: cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 -DNDEBUG ... (remaining 78 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
tensorflow/core/kernels/tensor_array_ops.cc:244:7: error: return type 'tensorflow::Status' must match previous return type 'const ::tensorflow::Status' when lambda expression has unspecified explicit return type
return Status::OK();
^
In file included from tensorflow/core/kernels/tensor_array_ops.cc:23:
In file included from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:
In file included from external/eigen_archive/eigen-eigen-a5e9085a94e8/unsupported/Eigen/CXX11/Tensor:57:
In file included from external/eigen_archive/eigen-eigen-a5e9085a94e8/unsupported/Eigen/CXX11/ThreadPool:53:

[ahmed-touati]

here is information about versions I used

bazel version

Build label: 0.2.0
Build target: bazel-out/local_darwin-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Feb 23 13:11:56 2016 (1456233116)
Build timestamp: 1456233116
Build timestamp as int: 1456233116

gcc version

onfigured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
Target: x86_64-apple-darwin14.0.0
Thread model: posix

[TheBlueFrog]

I'm getting this when trying to build the Docker image. This is with
the DockerFile from github today.

ERROR: /root/.cache/bazel/_bazel_root/5b21cea144c0077ae150bf0330ff61a0/external/tf/tensorflow/core/kernels/BUILD:664:1: C++ compilation of rule '@tf//tensorflow/core/kernels:padding_fifo_queue' failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 ... (remaining 79 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 4.
gcc: internal compiler error: Killed (program cc1plus)

This seems to hang the build.

Ubuntu 14.04 with Docker 1.11.1 on a Microsoft Azure virtual machine

mike collins

[ebrevdo]

There's a fix incoming. Next sync from Google.
On May 20, 2016 1:45 PM, "TheBlueFrog" [email protected] wrote:

I'm getting this when trying to setup the Docker image. This is with
the DockerFile from github today.

ERROR:
/root/.cache/bazel/_bazel_root/5b21cea144c0077ae150bf0330ff61a0/external/tf/tensorflow/core/kernels/BUILD:664:1:
C++ compilation of rule '@tf//tensorflow/core/kernels:padding_fifo_queue'
failed: gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE
'-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now
-B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object
-fno-omit-frame-pointer -g0 ... (remaining 79 argument(s) skipped):
com.google.devtools.build.lib.shell.BadExitStatusException: Process exited
with status 4.
gcc: internal compiler error: Killed (program cc1plus)

mike collins

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/tensorflow/models/issues/130#issuecomment-220714016

[dimplevador]

@ahmed-touati @TheBlueFrog @ebrevdo @mrry If anyone is still facing this issue, i solved it by updating my Mac to latest version of El Capitan - 10.11.5 (I had Yosemite) and also updated Xcode to its latest version. Just updating the OS didn't work. It worked only after i updated Xcode and re-installed gcc (from Xcode command line tool). So, gcc compiler seems to be the main problem.

Deleted the folder created by this step: git clone --recursive https://github.com/tensorflow/models.git

And did again the following steps:
git clone --recursive https://github.com/tensorflow/models.git
cd models/syntaxnet/tensorflow
./configure
cd ..
bazel test syntaxnet/... util/utf8/...
# On Mac, run the following:
bazel test --linkopt=-headerpad_max_install_names \syntaxnet/... util/utf8/...

Dimple Vador.

[yrjie]

Hi @dimplevador ,

How do you re-install gcc from Xcode command line tool? Seems like gcc is automatically installed when the command line tool is installed. I can't find an option to install it again.

Thank you.

[dimplevador]

Hi @yrjie ,

Err looks like i didn't write the steps very clearly, sorry for the confusion.

What i did:
1) Updated Mac OS to El Captian and also updated Xcode from the app store.
2) Re-start.
3) Open terminal.
4) Type: $gcc --version
5) Xcode will ask for licence agreement etc... etc...
6) Type again: $gcc --version
7) Check the version, it would have updated.

Hope that helps. Let me know, if steps are still confusing.

Dimple Vador.

[tenqyu-research]

I have the same error on a Ubuntu 16.04 LTS linode. gcc version 5.3.1 20160413

[dimplevador]

@tenqyu-research update to latest version of gcc - 6.1 and try again. Updating gcc had helped in solving my problem on Mac, should work on Ubuntu also.

[tenqyu-research]

gcc-6 is not a standard package so you have to do install via personal package archives (PPA) which might be unstable
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 60 --slave /usr/bin/g++ g++ /usr/bin/g++-6
That seems to have fixed this error.
However, after that error disappeared, I ran into this one " undeclared inclusion(s) in rule @tf//tensorflow/core:function_ops_op_lib" .
For the time being, I updated the submodule (git submodule update --init) and removed the previous installer version (bazel clean --expunge) let's see where this brings me.

[tenqyu-research]

@dimplevador looks like this approached worked. Compiling for a while now ... still 183 jobs to complete. Many thanks!

[dimplevador]

@tenqyu-research :) glad it helped.
It does take a while to complete. The Mac i was testing on had 5th gen i5, 4gb ram and no discrete graphic card. It took around 2 to 2.5 hours to finish compiling & successfully pass the 12 tests. It would finish faster if you have more ram & a solid graphic card.

[ahmed-touati]

@dimplevador I have already gcc 6.1
but It didn't work for me :/
gcc --version
Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)

[dimplevador]

@ahmed-touati your bazel version is 0.2.0, right? You need 0.2.2, be careful there is also a version 0.2.2b, but we need the one without "b" i.e. 0.2.2

I have previously mentioned the below steps but writing them again for your reference:

After that you need to delete the folder created by this step: git clone --recursive https://github.com/tensorflow/models.git

And do again the following steps:
git clone --recursive https://github.com/tensorflow/models.git
cd models/syntaxnet/tensorflow
./configure
cd ..
bazel test --linkopt=-headerpad_max_install_names \syntaxnet/... util/utf8/...

Let me know if it works out for you or you are still facing problem.

Dimple Vador.

[ahmed-touati]

@dimplevador Thank you for yout help.

I change my bazel version to 0.2.2 and I followed your instructions but I obtained the same Error.

However, Surprisingly, I succeed now to build tensorflow from source.

[dimplevador]

@ahmed-touati :) it worked, that's what matters in the end. Have fun experimenting.

[ahmed-touati]

@dimplevador : Actually, it worked only for building tensorflow but not for building syntaxnet.

[dimplevador]

@ahmed-touati Syntaxnet did build successfully on my machine after the above steps. If you need syntaxnet, let me know. It works on my machine so we can compare our versions of OSX, java, bazel, protobuf, gcc and python, hopefully that would help in figuring out why it does not work on your machine.

[calberti]

Hi Ahmed, do you have any updates on this?

[ahmed-touati]

@calberti : I have updated my osx as @dimplevador suggested but I still have the same Error :(

[dimplevador]

@ahmed-touati @calberti the problem was with gcc. You need to update gcc to latest version. For that i had to update OSX followed by XCode. If you want, we can compare different versions of our software installed to run syntaxnet. That might reveal why it finally worked on my test machine but fails on yours.

[ahmed-touati]

@dimplevador thank you a lot.
I updated my version of Xcode and it works now :)

[dimplevador]

@ahmed-touati you are most welcome. Mark the issue as closed, if possible.
@calberti it finally worked for @ahmed-touati
Give it a try. Update OSX followed by Xcode and then gcc. For detailed steps read my comments in this thread. Let me know if it still does not workout for you.