Bazel: Cannot build bazel on university clusters

[Strideradu]

I am tring to build bazel on our university clusters but seems there are errors I cannot handel

Please provide the following information. The more we know about your system and use case, the more easily and likely we can help.

Description of the problem / feature request / question:

]$ ./compile.sh
INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO: ./compile.sh compile /path/to/bazel
🍃 Building Bazel from scratch.......
🍃 Building Bazel with Bazel.
.WARNING: /tmp/bazel_G7zZtRZZ/out/external/bazel_tools/WORKSPACE:1: Workspace name in /tmp/bazel_G7zZtRZZ/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.
INFO: Found 1 target...
ERROR: /mnt/home/dunan/mysoftware/bazel-0.4.0/src/main/protobuf/BUILD:25:2: error executing shell command: 'JAR='external/local_jdk/bin/jar' OUTPUT='bazel-out/local-opt/bin/src/main/protobuf/libinvocation_policy_java_proto_srcjar.srcjar' PROTO_COMPILER='third_party/protobuf/3.0.0/protoc-3.0.0-linux-x86_6...' failed: process-wrapper failed: error executing command
(cd /tmp/bazel_G7zZtRZZ/out/bazel-sandbox/65de4ba8-87c8-4a18-8302-dd58c158e877-23/execroot/bazel-0.4.0 && \
exec env - \
LD_LIBRARY_PATH=/opt/software/R/3.2.3--GCC-4.9/lib64/R/lib:/opt/software/ClusterStudio/2013sp1/composer_xe_2013_sp1.0.080/mkl/lib/intel64:/opt/software/boost/1.58.0--GCC-4.9/lib:/opt/software/OpenMPI/1.10.0--GNU-4.9/lib:/opt/software/GCC/4.9/lib64:/opt/software/GCC/4.9/lib:/mnt/home/dunan/mysoftware/lib:/opt/software/MATLAB/R2014a/sys/opengl/lib/glnxa64:/opt/software/Python/2.7.2--GCC-4.4.5/lib/python2.7/lib-dynload:/opt/software/Python/2.7.2--GCC-4.4.5/lib \
PATH=/opt/software/R/3.2.3--GCC-4.9/bin:/opt/software/OpenMPI/1.10.0--GNU-4.9/bin:/opt/software/GCC/4.9/bin:/opt/software/JDK/jdk1.8.0_102/bin:/mnt/home/dunan/mysoftware/include:/mnt/home/dunan/mysoftware/bin:/mnt/home/dunan/mysoftware/lib:/mnt/home/dunan/anaconda/bin:/opt/software/Stata/13.0:/opt/software/MATLAB/R2014a/bin:/opt/software/Python/2.7.2--GCC-4.4.5/bin:/opt/software/cmake/2.8.5--GCC-4.4.5/bin:/usr/lib64/qt-3.3/bin:/opt/software/lmod/bin:/opt/moab/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/hpcc/bin:/opt/ibutils/bin \
/tmp/bazel_G7zZtRZZ/out/execroot/bazel-0.4.0/_bin/process-wrapper -1 5 - - /bin/bash -c 'JAR='\''external/local_jdk/bin/jar'\'' OUTPUT='\''bazel-out/local-opt/bin/src/main/protobuf/libinvocation_policy_java_proto_srcjar.srcjar'\'' PROTO_COMPILER='\''third_party/protobuf/3.0.0/protoc-3.0.0-linux-x86_64.exe'\'' SOURCE='\''src/main/protobuf/invocation_policy.proto'\'' bazel-out/host/bin/tools/build_rules/gensrcjar').
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.lang.ref.Finalizer.(Finalizer.java:226)

gensrcjar: jar failed
Target //src:bazel failed to build
INFO: Elapsed time: 11.770s, Critical Path: 7.43s

ERROR: Could not build Bazel

If possible, provide a minimal example to reproduce the problem:

Environment info

• Operating System:
  Should be CentOS 6
  java version "1.8.0_102"
  Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
  Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

• Bazel version (output of bazel info release):
  0.4.0

• If bazel info release returns "development version" or "(@non-git)", please tell us what source tree you compiled Bazel from; git commit hash is appreciated (git rev-parse HEAD):

Have you found anything relevant by searching the web? (e.g. GitHub issues, email threads in the [email protected] archive)

Anything else, information or logs or outputs that would be helpful?

(If they are large, please upload as attachment or provide link).

[kchodorow]

It looks like Bazel ran out of memory: Error occurred during initialization of VM java.lang.OutOfMemoryError: unable to create new native thread. How much memory do the university machines have?

[Strideradu]

it shouldn't be. Right now the develop node I am using has 95851092k free memory. It's weird.
And my java is:
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

$ free -m
total used free shared buffers cached
Mem: 128628 35599 93028 2 128 26314
-/+ buffers/cache: 9155 119472
Swap: 0 0 0

[michajlo]

Looks like you're not allowed to create that many threads. What's ulimit -a say?

[Strideradu]

Maybe
ulimit -a is 1024.

So is there anyway to solve the problem?

[michajlo]

You can try ulimit -u $somethinglarger, not sure if you'll have permission, or tweak flags to use less threads (don't know them off the top of my head).

[Strideradu]

I cannot modify the limit. I think may be i need a way to use less threads.

[kchodorow]

You could try adding something like -j 8 to https://github.com/bazelbuild/bazel/blob/master/scripts/bootstrap/compile.sh#L296.

Do you need to compile Bazel from source? Can you just install a binary?

[Strideradu]

I change to -j 4 and right now is successfully build! Thanks a lot for your help!