Rust-bindgen: Document providing include paths and FAQ for missing system headers

[vks]

Sometimes there is a cryptic fatal error: 'stddef.h' file not found, presumabely when libclang is incorrectly configured:

https://github.com/servo/gecko-media/issues/71
https://github.com/bsteinb/rsmpi/issues/1
https://github.com/rust-lang-nursery/rust-bindgen/issues/242
https://github.com/ajtulloch/caffe.rs/issues/1

In the past, this issue was fixed by making sure to install the clang package or by finding the missing pathes and adding them to the includes.

I was wondering whether bindgen could work around this issue? (The second solution above can be automated.) Alternatively, it could be documented how to fix it.

[emilio]

This is probably better suited for https://github.com/KyleMayes/clang-sys. That's the crate bindgen uses for figuring out system headers.

I agree that if this is a common problem, it'd be nice to document it, though.

Thanks for the report :)

[fitzgen]

For the most part, bindgen is not in the include-path-wrangling game, and we leave that to libclang. It's full of tricky edge cases and system specific bits, and at least C compilers and their behavior are largely a known quantity, even if not always the most transparent.

I would love to merge some PRs adding more documentation / FAQs about this stuff though!

[Imirsen]

I was experiencing this issue on Ubuntu 17.10 compiling a simple C file including stddef.h. I investigated and was able to fix it by making bindgen pass the C search paths to the driver when it detects that it is compiling C. See pull request #1351

[virtualritz]

I have the 'stddef.hI' issue. I'm on macOS 10.13.6, latest command line tools installed.
When trying to generate any C++ bindings through bindgen I get:

/Library/Developer/CommandLineTools/usr/include/c++/v1/cstddef:44:15: fatal error: 'stddef.h' file not found
/Library/Developer/CommandLineTools/usr/include/c++/v1/cstddef:44:15: fatal error: 'stddef.h' file not found, err: true
thread 'main' panicked at 'Unable to generate 3Delight bindings: ()', libcore/result.rs:1009:5

Generating C bindings works fine. I tried with several different C++ headers as input for bindgen. I always get this error.
The supposedly missing header is present on my system in many locations:

$ locate stddef.h
/Library/Developer/CommandLineTools/Library/PrivateFrameworks/LLDB.framework/Versions/A/Resources/Clang/include/stddef.h
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/stddef.h
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stddef.h
/Library/Developer/CommandLineTools/usr/include/c++/v1/stddef.h
/Library/Developer/CommandLineTools/usr/lib/clang/10.0.0/include/stddef.h
/Library/Developer/CommandLineTools/usr/lib/tapi/10.0.0/include/stddef.h
/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/stddef.h
/System/Library/Frameworks/OpenCL.framework/Versions/A/lib/clang/2.0/include/stddef.h
/System/Library/PrivateFrameworks/GPUCompiler.framework/Versions/A/Libraries/lib/clang/3.5/include/stddef.h
/usr/include/stddef.h
/usr/local/Cellar/boost/1.67.0_1/include/boost/asio/detail/cstddef.hpp

bindgen 0.41.0
rustc 1.30.0-nightly (ae7fe84e8 2018-09-26)
Apple LLVM version 10.0.0 (clang-1000.10.44.2)

I tried adding some (and even all) of the above paths as clang args in build.rs but the issue remains.

[bbqsrc]

Is there a known solution to this problem?

[emilio]

Each problem is generally different depending on the setup. Could you file a separate issue pointing out which libclang are you using, and the first few lines of running bindgen with RUST_LOG=debug?

[virtualritz]

I have not managed to find a solution for this. It's basically the reason why bindgen is useless for me on macOS for certain projects that depend on system headers.
Even if there are multiple ailments – depending on setup –, could they be documented somewhere, please?

[BusyJay]

Although it may not be rust-bindgen's problem, but it hurts user experiences of rust-bindgen badly. Hope to find a way to fix it permanently.

[psporysz]

I had similar problem when building Firefox codebase and the solution was to point auto-tools script to clang installed in CommandLineTools:

--with-clang-path=/Library/Developer/CommandLineTools/usr/bin/clang
--with-libclang-path=/Library/Developer/CommandLineTools/usr/lib/

Not sure how to set this up in rust world thus i did something ugly:

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

The above "fix" is just work-around for impatient souls who ended up here and do not know rust and mac ecosystem well enough ;)

[elichai]

Any news on that? it's failing to generate because it can't find some of the included headers. how do i pass a -I/path/to/the/headers?

[virtualritz]

I had similar problem when building Firefox codebase and the solution was to point auto-tools script to clang installed in CommandLineTools:

Could you share your build.rs to see where you supplied these flags?
Thanks!

[emilio]

You can specify LIBCLANG_PATH for the same effect. If you need extra bindgen flags you can also use BINDGEN_EXTRA_CLANG_ARGS in the environment, though generally there's a better solution than that.

[virtualritz]

How do you resolve framework header errors on macOS?
For example:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A/Headers/CFBase.h:18:10:
fatal error: 'CoreFoundation/CFAvailability.h' file not found

The path in the header is CoreFoundation/CFAvailability.h. But physically, the path is [...]/CoreFoundation.framework/Headers/CFAvailability.h.
There must be a part of Clang that can resolve this. How to activate it via build.rs?

> locate CFAvailability.h
/Applications/Xcode.app/Contents/Developer/Platforms/AppleTVSimulator.platform/Developer/SDKs/AppleTVSimulator.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFAvailability.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A/Headers/CFAvailability.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A/Headers/CFAvailability.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A/Headers/CFAvailability.h

[virtualritz]

I solved this after RTF(Clang)M. The flag for finding _macOS_ frameworks is -F:

In your build.rs add sth. like:

.clang_arg("-F/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/")

[KyussCaesar]

Got stuck on this same issue trying to build code inside a Docker container. Solution was to follow the instructions in the docs (i.e: make sure clang is installed) :sweat_smile:.

Some specifics about my situation, for future reference: Ubuntu 18.04 host, debian-based container, trying to build rust-xgboost from inside (requires xgboost-sys, which is a wrapper around the main XGBoost library).

[virtualritz]

@KyussCaesar Maybe the fact that everyone here had clang installed correctly got lost in the number of replies.
Despite this you may run into odd include path/framework path resolution issues.

The doc section you quoted states about macOS you should install clang and llvm to have bindgen work.
This is simply not true, in general.
I develop on macOS and I have five crates (three of them published) that all rely on bindgen.
Curiously they all build just fine using Apple's official clang. They do not require presence of a third party clang or llvm. Go figure. :)

[KyussCaesar]

@virtualritz actually I was meaning to link to the part in the docs about Debian-based Linuxes. grepped this thread for "Docker" and found nothing, so thought it might be useful as a reminder to make sure clang is installed inside the container image.