Bazel: Ubuntu 18 Arm64 (aarch64) bazel build.

[cafrank]

Feature requests: what underlying problem are you trying to solve with this feature?

Ubuntu 18 Arm64 (aarch64) bazel build. 0.19.0 was the last git build I could compile. TF 2.0 needs 0.24.0. I've been through enough stackoverflowing, googling, git'ing in an effort to build tensorflow 2.0. I am giving up. Has anyone in the world achieved this? If so, can I get a binary so that I can get back to building TF? (Yes it feels like a missing feat(ure).

[ArielleA]

Hi,
Could you please explain a little more about your environment to help me identify the problem? I have built 0.25.2-dist on my machine without problem.
Please can you include the volume of memory and any errors you are receiving in the output?

[vbatts]

I've made it further (past the OOM failures) in the aarch64 build for centos/fedora.
Now I'm finding that the build is still trying to use a bundled JDK, despite flags to use the host jdk:

+ export 'EXTRA_BAZEL_ARGS= --host_force_python=PY2 --nokeep_state_after_build --notrack_incremental_state --nokeep_state_after_build --host_javabase=@local_jdk//:jdk --verbose_failures'
+ EXTRA_BAZEL_ARGS=' --host_force_python=PY2 --nokeep_state_after_build --notrack_incremental_state --nokeep_state_after_build --host_javabase=@local_jdk//:jdk --verbose_failures'
+ ./compile.sh
Building Bazel from scratch..find: 'src/tools/xcode-common/java/com/google/devtools/build/xcode/common': No such file or directory
find: 'src/tools/xcode-common/java/com/google/devtools/build/xcode/util': No such file or directory
....
Building Bazel with Bazel.
.WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/tmp/bazel_NVE2653l/archive/libblaze.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
[...]
toValueProcessor) [for host]; 17s worker ... (10 actions, 6 running)
[1,531 / 1,589] Building src/main/java/com/google/devtools/build/lib/rules/objc/libobjc.jar (55 source files) and running annotation processors (SkylarkCallableProcessor, SkylarkConfigurationFieldProcessor, AutoCodecProcessor, OptionProce
ssor, AutoAnnotationProcessor, AutoValueProcessor) [for host]; 6s worker ... (5 actions, 4 running)
ERROR: /builddir/build/BUILD/bazel-0.27.0/scripts/BUILD:6:1: Executing genrule //scripts:bash_completion failed (Exit 37) bash failed: error executing command 
  (cd /builddir/.cache/bazel/_bazel_mockbuild/49dc368c1518f79a0495c4122d3982b5/sandbox/linux-sandbox/1518/execroot/io_bazel && \
  exec env - \
    PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/builddir/.local/bin:/builddir/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; scripts/generate_bash_completion.sh --bazel=bazel-out/host/bin/src/bazel --output=bazel-out/aarch64-fastbuild/bin/scripts/bazel-complete.bash --prepend=scripts/ba
zel-complete-header.bash --prepend=scripts/bazel-complete-template.bash')
Execution platform: @bazel_tools//platforms:host_platform

Use --sandbox_debug to see verbose messages from the sandbox: bash failed: error executing command 
  (cd /builddir/.cache/bazel/_bazel_mockbuild/49dc368c1518f79a0495c4122d3982b5/sandbox/linux-sandbox/1518/execroot/io_bazel && \
  exec env - \
    PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/builddir/.local/bin:/builddir/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; scripts/generate_bash_completion.sh --bazel=bazel-out/host/bin/src/bazel --output=bazel-out/aarch64-fastbuild/bin/scripts/bazel-complete.bash --prepend=scripts/bazel-complete-header.bash --prepend=scripts/bazel-complete-template.bash')
Execution platform: @bazel_tools//platforms:host_platform

Use --sandbox_debug to see verbose messages from the sandbox
Extracting Bazel installation...
FATAL: execv of '/tmp/generate_bash_completion.tNMdaUFh/root/install/2db87e64821badc1e549ad6d4ee55d28/_embedded_binaries/embedded_tools/jdk/bin/java' failed: (error: 8): Exec format error
Target //scripts:bazel-complete.bash failed to build
INFO: Elapsed time: 1333.806s, Critical Path: 423.53s

full build log: bazel-centos-aarch64.log

[depthwise]

Same here. My use case is CI on aarch64 (Jetson Xavier). This used to work with earlier bazel versions. It'd be good to support aarch64 at this point, ARM64 is getting fast enough to be used as a primary dev environment when developing for arm.

[1e100]

The right incantation seems to be EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh. This builds 0.29.1 on Jetson Nano.

[settle]

@1e100 I tried that build as well (see 11038), albeit for Raspberry Pi 4 (4 GB) under Ubuntu 18.04 aarch64, but it fails to build. Any thoughts as to why it worked on your Jetson Nano but not my RPI4?

[1e100]

Could be just about anything, including OOM. If you're building for aarch64 though, it could be that bazel built on another board would work on Raspberry Pi as well, as long as the same Ubuntu version is used on both. Unlike earlier ISAs, aarch64 is much more uniform across chips.

IMO the more viable strategy for Raspberry Pi is cross-build from something more capable.

There are numerous examples out there on how to plug in the toolchain, and this works well once you set it up. In my case the situation is complicated because I also need to build CUDA code, so cross-build becomes pretty horrifying because of that. But for CPU only it's pretty straightforward. In fact I'm currently pursuing CUDA cross build even for the Jetson as it simplifies CI setup etc.