Bazel: Cannot 'bazel build' master

[thundergolfer]

Description of the problem / feature request:

bazel build //src:bazel resulting in FAILED: Build did not complete successfully

Feature requests: what underlying problem are you trying to solve with this feature?

Would like to be able to compile and run a development version of bazel so that I can debug issues that I'm having with the tool.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

# checkout latest master (as of time of issue creation)
git checkout b9d80a83bb 

Get bazel (I used brew install bazel). Then do:

bazel build //src:bazel

Outputs:

INFO: Analysed target //src:bazel (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_jbelotti/f105683f3b238df100a6da5b943f82da/external/com_google_protobuf/BUILD.bazel:123:1: C++ compilation of rule '@com_google_protobuf//:protobuf_lite' failed (Exit 1) cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 37 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
external/com_google_protobuf/src/google/protobuf/arena.cc:128:1: error: unknown type name 'GOOGLE_PROTOBUF_ATTRIBUTE_NOINLINE'
GOOGLE_PROTOBUF_ATTRIBUTE_NOINLINE
^
external/com_google_protobuf/src/google/protobuf/arena.cc:129:1: error: expected unqualified-id
void ArenaImpl::SerialArena::AddCleanupFallback(void* elem,
^
external/com_google_protobuf/src/google/protobuf/arena.cc:145:1: error: C++ requires a type specifier for all declarations
GOOGLE_PROTOBUF_ATTRIBUTE_FUNC_ALIGN(32)
^
external/com_google_protobuf/src/google/protobuf/arena.cc:145:41: error: expected ';' after top level declarator
GOOGLE_PROTOBUF_ATTRIBUTE_FUNC_ALIGN(32)
                                        ^

What operating system are you running Bazel on?

macOS Mojave - Version 10.14.3

What's the output of bazel info release?

release 0.24.1

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

[email protected]:bazelbuild/bazel.git
b9d80a83bb5cd97c489b2cd7c5206e035bc85c96
b9d80a83bb5cd97c489b2cd7c5206e035bc85c96

Any other information, logs, or outputs that you want to share?

When I continuously re-run bazel build //src:bazel I keep getting:

INFO: Analysed target //src:bazel (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_jbelotti/f105683f3b238df100a6da5b943f82da/external/com_google_protobuf/BUILD.bazel:408:1: C++ compilation of rule '@com_google_protobuf//:protoc' failed (Exit 1) cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG ... (remaining 36 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
In file included from external/com_google_protobuf/src/google/protobuf/compiler/main.cc:43:
/usr/local/include/google/protobuf/compiler/js/js_generator.h:41:10: fatal error: 'google/protobuf/compiler/scc.h' file not found
#include <google/protobuf/compiler/scc.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

[philwo]

Hi @thundergolfer, sorry that you couldn't build Bazel. This should definitely work.

What's the output of these commands?

xcode-select -p
xcodebuild -version
clang -v

[thundergolfer]

• xcode-select -p -> /Library/Developer/CommandLineTools
• xcodebuild -version -> xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance
• clang -v -> clang version 5.0.2 (tags/RELEASE_502/final) Target: x86_64-apple-darwin18.2.0 Thread model: posix InstalledDir: /nix/store/d9yyrqg1gp7rjw1n0vfp06frxrdmg7wc-clang-5.0.2/bin

I'm guessing the 2nd is a problem?

[philwo]

I think the problem is 3: You're using a non-Apple toolchain (clang seems to come from NixOS?).

We're not testing against this on CI and can't make any promise that it will work :/ Can you try building against the standard Apple tools? The Xcode command-line tools should be fine (if they're reasonably up to date), although a full, recent Xcode installation is the best tested environment, as we test it on CI.

[thundergolfer]

Actually sorry I was under a Nix env when I ran the command. Late-night messup 😖.

Under ~/work/bazel (my bazel repo, not under Nix). I get:

Apple LLVM version 10.0.1 (clang-1001.0.46.3)
Target: x86_64-apple-darwin18.2.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[jschaf]

I can reproduce that Bazel doesn't build on MacOS.

$ bazel version
Build label: 0.25.0
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed May 1 21:47:49 2019 (1556747269)
Build timestamp: 1556747269
Build timestamp as int: 1556747269

$ xcode-select -p
/Library/Developer/CommandLineTools

$ clang -v
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Errors:

third_party/protobuf/3.6.1/src/google/protobuf/arena.cc:128:1: error: unknown type name 'GOOGLE_PROTOBUF_ATTRIBUTE_NOINLINE'
GOOGLE_PROTOBUF_ATTRIBUTE_NOINLINE
^
third_party/protobuf/3.6.1/src/google/protobuf/arena.cc:129:1: error: expected unqualified-id
void ArenaImpl::SerialArena::AddCleanupFallback(void* elem,
^
third_party/protobuf/3.6.1/src/google/protobuf/arena.cc:145:1: error: C++ requires a type specifier for all declarations
GOOGLE_PROTOBUF_ATTRIBUTE_FUNC_ALIGN(32)
^

[jschaf]

So, the fix for me is to:

Deleting /usr/local/Cellar/protobuf directory resolved a similar issue for me.

From https://github.com/protocolbuffers/protobuf/issues/5376

[philwo]

@hlopko This sounds like Bazel accidentally picks up header files from the system when compiling Bazel's checked in Protobuf C++ sources on macOS? Can you have a look?

[thundergolfer]

Instead of simply delete /usr/local/Cellar/protobuf I did brew uninstall protobuf and can confirm that I can now build bazel. 🎉

This sounds like Bazel accidentally picks up header files from the system

@philwo are you saying there that Bazel is exiting its build sandbox?

[philwo]

@thundergolfer Bazel's usual sandboxing does not try to protect against accidental dependencies on files from the host filesystem. This is by design, because it's not possible to give a one-size-fits-all recipe regarding which files from the host system are still OK and which are not (e.g. is it OK if an action calls /bin/cat? What about /usr/bin/zip? Is it allowed to read something in /etc or /usr/lib?). To achieve that level of isolation, you would need containerized execution (which we support on Linux via Docker) or remote execution.

However, AFAIK the C++ rules should ensure that the include files are not used from the system.

As a bonus: You can hide individual directories or files from actions using the --sandbox_block_path=/usr/local (as an example) flag, though! :)

[jschaf]

How does Bazel get the include path for /usr/local/Cellar/protobuf?

[jschaf]

Found it. usr/local/include symlinks to Homebrew headers.

$ ls -l /usr/local/include
total 0
lrwxr-xr-x  1 joe  admin  38 Apr 30 22:30 ares.h -> ../Cellar/c-ares/1.15.0/include/ares.h
lrwxr-xr-x  1 joe  admin  44 Apr 30 22:30 ares_build.h -> ../Cellar/c-ares/1.15.0/include/ares_build.h
lrwxr-xr-x  1 joe  admin  42 Apr 30 22:30 ares_dns.h -> ../Cellar/c-ares/1.15.0/include/ares_dns.h
lrwxr-xr-x  1 joe  admin  44 Apr 30 22:30 ares_rules.h -> ../Cellar/c-ares/1.15.0/include/ares_rules.h
lrwxr-xr-x  1 joe  admin  46 Apr 30 22:30 ares_version.h -> ../Cellar/c-ares/1.15.0/include/ares_version.h
lrwxr-xr-x  1 joe  admin  31 Feb 25 12:01 c++ -> ../Cellar/gcc/8.3.0/include/c++
lrwxr-xr-x  1 joe  admin  49 Apr 25 11:51 ecpg_config.h -> ../Cellar/postgresql/11.2_1/include/ecpg_config.h

[philwo]

I'm assigning this to the C++ team, because this looks like a potential bug in the C++ rules.

[hlopko]

Because by default we use system installed compiler, we also need to use system installed headers. We detect system installed headers by running $CC -E -xc++ - -v and parsing the output. I'm not sure what can we do on the rules side to prevent this issue from happening, maybe only recommend using more hermetic C++ toolchain, and not the one autoconfigured by Bazel. What do you think?

[philwo]

@hlopko I see, that makes sense! In wonder, shouldn't Bazel's own protobuf headers take precedence over headers from a system-installed protobuf though?

[rikbrown]

Because by default we use system installed compiler, we also need to use system installed headers.

I see, that makes sense! In wonder, shouldn't Bazel's own protobuf headers take precedence over headers from a system-installed protobuf though?

Was there any progress or further thoughts on this @philwo / @hlopko?

I've worked around the issue by adding build --sandbox_block_path=/usr/local to my .bazelrc per @philwo above, but it seems suboptimal that Bazel doesn't prioritise its sandboxed headers over system ones.

[evanj]

I think this is the same issue as #8984 and should plausibly be closed as a duplicate of it?

[oquenchil]

Closing as duplicate