Bazel: Missing dependency declarations for the following files included by X

Created on 9 Feb 2018 · 17Comments · Source: bazelbuild/bazel

Description of the problem / feature request:

Bazel is virtually unusable because of the constant "missing dependency declarations for the following files included by X" messages. This not only gets in the way of our own development, I can't even compile some "official" rules such as rules_sass.

Easiest way to reproduce

$ git clone https://github.com/bazelbuild/rules_sass.git rules_sass
$ cd rules_sass
$ bazel build //...

I know that it works for some people, but I get the following:

[mark@arch rules_sass]$ bazel build //...
........
INFO: Analysed 5 targets (16 packages loaded).
INFO: Found 5 targets...
ERROR: /home/mark/.cache/bazel/_bazel_mark/102e5c7ceb737dbe98e59b2e1d756774/external/sassc/BUILD.bazel:4:1: undeclared inclusion(s) in rule '@sassc//:sassc':
this rule is missing dependency declarations for the following files included by 'external/sassc/sassc.c':
  '/usr/include/stdc-predef.h'
  '/usr/include/stdio.h'
  '/usr/include/bits/libc-header-start.h'
  '/usr/include/features.h'
  '/usr/include/sys/cdefs.h'
  '/usr/include/bits/wordsize.h'
  '/usr/include/bits/long-double.h'
  '/usr/include/gnu/stubs.h'
  '/usr/include/gnu/stubs-64.h'
  '/usr/include/stddef.h'
  '/usr/include/bits/types.h'
  '/usr/include/bits/typesizes.h'
  '/usr/include/bits/types/__FILE.h'
  '/usr/include/bits/types/FILE.h'
  '/usr/include/libio.h'
  '/usr/include/_G_config.h'
  '/usr/include/bits/types/__mbstate_t.h'
  '/usr/include/stdarg.h'
  '/usr/include/bits/stdio_lim.h'
  '/usr/include/bits/sys_errlist.h'
  '/usr/include/bits/stdio.h'
  '/usr/include/bits/stdio2.h'
  '/usr/include/string.h'
  '/usr/include/bits/types/locale_t.h'
  '/usr/include/bits/types/__locale_t.h'
  '/usr/include/strings.h'
  '/usr/include/bits/strings_fortified.h'
  '/usr/include/bits/string_fortified.h'
  '/usr/include/stdlib.h'
  '/usr/include/bits/waitflags.h'
  '/usr/include/bits/waitstatus.h'
  '/usr/include/bits/floatn.h'
  '/usr/include/sys/types.h'
  '/usr/include/bits/types/clock_t.h'
  '/usr/include/bits/types/clockid_t.h'
  '/usr/include/bits/types/time_t.h'
  '/usr/include/bits/types/timer_t.h'
  '/usr/include/bits/stdint-intn.h'
  '/usr/include/endian.h'
  '/usr/include/bits/endian.h'
  '/usr/include/bits/byteswap.h'
  '/usr/include/bits/byteswap-16.h'
  '/usr/include/bits/uintn-identity.h'
  '/usr/include/sys/select.h'
  '/usr/include/bits/select.h'
  '/usr/include/bits/types/sigset_t.h'
  '/usr/include/bits/types/__sigset_t.h'
  '/usr/include/bits/types/struct_timeval.h'
  '/usr/include/bits/types/struct_timespec.h'
  '/usr/include/bits/select2.h'
  '/usr/include/sys/sysmacros.h'
  '/usr/include/bits/sysmacros.h'
  '/usr/include/bits/pthreadtypes.h'
  '/usr/include/bits/thread-shared-types.h'
  '/usr/include/bits/pthreadtypes-arch.h'
  '/usr/include/alloca.h'
  '/usr/include/bits/stdlib-bsearch.h'
  '/usr/include/bits/stdlib-float.h'
  '/usr/include/bits/stdlib.h'
  '/usr/include/getopt.h'
  '/usr/include/bits/getopt_core.h'
  '/usr/include/bits/getopt_ext.h'
  '/usr/include/stdbool.h'
INFO: Elapsed time: 4.255s, Critical Path: 1.39s
FAILED: Build did NOT complete successfully

The same sort of error manifests itself with other projects (the exact message is different, of course).

What operating system are you running Bazel on?

Arch Linux, I know there is an issue with inclusion detection in 0.10 as reported in https://github.com/bazelbuild/bazel/issues/4572, but I'm using Bazel from Nix which is 0.9.

What's the output of `bazel info release`?

[mark@arch rules_sass]$ bazel info release
release 0.9.0- (@non-git)
[mark@arch rules_sass]$ bazel version
Build label: 0.9.0- (@non-git)
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Sat Nov 2 00:00:00 +11968 (315532800000)
Build timestamp: 315532800000
Build timestamp as int: 315532800000

If `bazel info release` returns "development version" or "(@non-git)", tell us how you built Bazel.

As I said it comes from Nix.

Have you found anything relevant by searching the web?

Yes I found a few similar issues, but I the fixes are inapplicable unless I'm expected to patch every dependency (such as rules_sass) to make it work.

P3 more data needed team-Rules-CPP bug

Source

mrkkrp

❤1

Most helpful comment

Just had a similar issue on arch linux. Running bazel clean --expunge resolved it.

aszinovyev on 23 Apr 2020

👍4

All 17 comments

Curiously after some upgrading and reinstalling of gcc and glibc I get a different error:

[mark@arch rules_sass]$ bazel build //...
........
INFO: Analysed 5 targets (16 packages loaded).
INFO: Found 5 targets...
ERROR: /home/mark/.cache/bazel/_bazel_mark/102e5c7ceb737dbe98e59b2e1d756774/external/sassc/BUILD.bazel:4:1: C++ compilation of rule '@sassc//:sassc' failed (Exit 1)
In file included from external/sassc/sassc.c:7:0:
/usr/include/stdio.h:33:10: fatal error: stddef.h: No such file or directory
 #include <stddef.h>
          ^~~~~~~~~~
compilation terminated.
INFO: Elapsed time: 3.023s, Critical Path: 0.68s
FAILED: Build did NOT complete successfully

Although when I compile other projects (non-Bazel related) stddef.h is found OK, it located at:

/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include/stddef.h

mrkkrp on 9 Feb 2018

Re-installed Nix, a bit different error message but the same problem:

ERROR: /home/mark/.cache/bazel/_bazel_mark/102e5c7ceb737dbe98e59b2e1d756774/external/sassc/BUILD.bazel:4:1: undeclared inclusion(s) in rule '@sassc//:sassc':
this rule is missing dependency declarations for the following files included by 'external/libsass/src/c99func.c':
  '/usr/include/stdc-predef.h'

mrkkrp on 9 Feb 2018

This looks like a bug, but to be honest, I don't think anyone on the team has experience with Nix... 😞

@mhlopko @ulfjack Any idea?

philwo on 15 Feb 2018

I believe this is a duplicate of #4572, can you pls test if the linked fix works for you?

hlopko on 16 Feb 2018

Well, not if you're using gcc, the fix is for clang only. For gcc we should be using -fno-canonical-system-headers, but that behavior didn't change between 0.9.0 and 0.10.0. Can you pls verify that your gcc understands -fno-canonical-system-headers?

hlopko on 16 Feb 2018

According to its man page it does.

mrkkrp on 18 Feb 2018

Can you check if builtin include directories change between:

gcc -E -xc++ - -v and gcc -E -xc++ - -v -fno-canonical-system-headers
gcc -E -xc++ - -v and gcc -E -xc - -v

C include directories should be subset of c++ ones, and I assumed that -fno-canonical-system-headers doesn't affect the output. I expect that at least one of these assumptions was wrong :)

hlopko on 19 Feb 2018

Here is what I get:

[mark@arch ~]$ gcc -E -xc++ - -v
Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 7.3.0 (GCC) 
COLLECT_GCC_OPTIONS='-E' '-v' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/cc1plus -E -quiet -v -D_GNU_SOURCE - -mtune=generic -march=x86-64
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../include/c++/7.3.0
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../include/c++/7.3.0/x86_64-pc-linux-gnu
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../include/c++/7.3.0/backward
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed
 /usr/include
End of search list.
^C
[mark@arch ~]$ gcc -E -xc++ - -v -fno-canonical-system-headers
Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 7.3.0 (GCC) 
COLLECT_GCC_OPTIONS='-E' '-v' '-fno-canonical-system-headers' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/cc1plus -E -quiet -v -D_GNU_SOURCE - -mtune=generic -march=x86-64 -fno-canonical-system-headers
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../include/c++/7.3.0
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../include/c++/7.3.0/x86_64-pc-linux-gnu
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../include/c++/7.3.0/backward
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed
 /usr/include
End of search list.
^C
[mark@arch ~]$ gcc -E -xc - -v
Using built-in specs.
COLLECT_GCC=gcc
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 7.3.0 (GCC) 
COLLECT_GCC_OPTIONS='-E' '-v' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/cc1 -E -quiet -v - -mtune=generic -march=x86-64
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include
 /usr/local/include
 /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed
 /usr/include
End of search list.
^C

mrkkrp on 20 Feb 2018

Ok so everything looks fine here. It must be something else. Can you dig a .d file (it's somewhere like here: ~/.cache/bazel/_bazel_hlopko/70a58deccaeea3140ef7c823570f4916/bazel-out/k8-fastbuild/bin/external/sassc/_objs/sassc/external/libsass/src/expand.pic.d

Also changing error messages are caused by races, since bazel doesn't guarantee the order of execution of actions, and since action execution is heavily paralleled, you just get different error from different actions. But that's fine.

What's not fine is why do you get those inclusion errors. Can you isolate the problem more? E.g construct a small workspace that demonstrates the problem on a single cc_library? If you could share a dockerfile to get a identical system, that would help me a lot too.

hlopko on 22 Feb 2018

Theoretically this should reproduce it:

FROM base/archlinux

WORKDIR /home

RUN pacman --noconfirm --noprogressbar -Syu
RUN pacman --noconfirm --noprogressbar -S git gcc glibc bazel
RUN git clone https://github.com/bazelbuild/rules_sass.git rules_sass

But it doesn't. Which makes me think that my system is screwed in some way. Probably need to do a reinstall, because how can I figure out what's wrong now... :-D

mrkkrp on 24 Feb 2018

@mrkkrp Thanks for creating that repro and then also realizing that it doesn't repro ;)

I'm an Arch Linux user myself - I'd try to find any files not belonging to installed packages to rule out that this is causing the problem:

https://wiki.archlinux.org/index.php/Pacman/Tips_and_tricks#Identify_files_not_owned_by_any_package

Also see if you modified any files belonging to packages:
https://wiki.archlinux.org/index.php/Pacman/Tips_and_tricks#Listing_all_changed_files_from_packages
or https://wiki.archlinux.org/index.php/Pacman/Tips_and_tricks#Listing_changed_backup_files

Hope you can find out what's happening. If it does turn out to be a Bazel bug, we definitely want to fix it :)

philwo on 24 Feb 2018

I'll try to find out what is going on once I have some free time :)

mrkkrp on 6 Mar 2018

This might be related to #3977 and the following (partial?) fix: https://github.com/bazelbuild/bazel/commit/c122e47bd8386eb318ced9917d86e4c39378e222

judah on 30 Apr 2018

There have been many fixes in cc_configure since March, I'd try getting the latest bazel (0.21 is the latest today), run bazel clean --expunge so we are sure there is no dirty state in the output base, and everything should work.

Please reopen or fine a new issue when it is not true :)

hlopko on 11 Jan 2019

👍3 ❤1

I know this is a closed issue, but just wanted to leave a comment here, in case it is useful to others:

Faced a similar issue as above while building protobuf with bazel. My bazel was originally installed system-wide. When I reinstalled bazel locally only for the current user and deleted the bazel cache and built again, the build completed successfully.

Bazel version 0.11 (I know it's a very old version, but sadly that's what is mandated in my team)