Source-build: Fatal error: 'elf.h' file not found on macOS 10.15

[asbjornu]

In Homebrew/homebrew-core#60929 I've got dotnet to build for macOS 10.13 and 10.14, but on 10.15 the build fails with the following error:

/tmp/dotnet-20200912-76027-168kcki/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5/src/pal/src/exception/remote-unwind.cpp:68:10: fatal error: 'elf.h' file not found
#include <elf.h>
         ^~~~~~~

The end result reports something completely unrelated, but I suspect it's the missing elf.h file causing problem and not this:

  make: *** [all] Error 2
  Failed to build CoreCLR component.

  'coreclr' failed during build.
  See '/private/tmp/dotnet-20200912-4021-l77t2w/artifacts/logs/coreclr.log' for more information.
  'corefx' failed during build.
  'msbuild' failed during build.
  'known-good' failed during build.

Build FAILED.

/private/tmp/dotnet-20200912-4021-l77t2w/packages/restored/microsoft.dotnet.arcade.sdk/1.0.0-beta.19359.6/tools/ProjectLayout.props(8,3): warning MSB4011: "/private/tmp/dotnet-20200912-4021-l77t2w/packages/restored/microsoft.dotnet.arcade.sdk/1.0.0-beta.19359.6/tools/RepoLayout.props" cannot be imported again. It was already imported at "/private/tmp/dotnet-20200912-4021-l77t2w/packages/restored/microsoft.dotnet.arcade.sdk/1.0.0-beta.19359.6/tools/Build.proj (52,3)". This is most likely a build authoring error. This subsequent import will be ignored. 
/private/tmp/dotnet-20200912-4021-l77t2w/repos/Directory.Build.targets(296,5): error MSB3073: The command "/private/tmp/dotnet-20200912-4021-l77t2w/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5//build.sh x64 Release skiptests -ignoreWarnings -skipmanagedtools -skiprestore -nopgooptimize msbuildonunsupportedplatform cmakeargs -DCLR_CMAKE_USE_SYSTEM_LIBUNWIND=TRUE /bl /p:ILLinkTrimAssembly=false /p:CheckCDefines=false /p:PackagesDir=/private/tmp/dotnet-20200912-4021-l77t2w/packages/restored/ /p:ContinuousIntegrationBuild=true /p:VersionSuffix="servicing" -warnAsError:0 /v:minimal  /p:DotNetPackageVersionPropsPath=/private/tmp/dotnet-20200912-4021-l77t2w/artifacts/obj/x64/Release/PackageVersions.props /p:DotNetRestoreSourcePropsPath=/private/tmp/dotnet-20200912-4021-l77t2w/artifacts/obj/x64/Release/RestoreSources.props >> /private/tmp/dotnet-20200912-4021-l77t2w/artifacts/logs/coreclr.log 2>&1" exited with code 1. [/private/tmp/dotnet-20200912-4021-l77t2w/repos/coreclr.proj]
    1 Warning(s)
    1 Error(s)

Here's the coreclr.log file for inspection.

I don't have any good ideas how to resolve this. Since Homebrew's libelf formula does not seem to bundle elf.h, taking a dependency on libelf won't help. There's also no easy way to make dependencies conditional on the macOS version afaik.

As suggested in the same SO thread, there's a possibility to source elf.h from Apple's dtrace, but even better would be if elf.h was provided out of the box from source-build itself somehow.

[dagood]

Are you able to build dotnet/coreclr on macOS 10.15 at all? This issue should probably be raised in dotnet/runtime (since dotnet/coreclr was consolidated there), ideally without source-build infra involved so there aren't as many variables to consider.

[wfurt]

I do build runtime on 10.15 without problems. I did not try source build for a while.

[asbjornu]

I don't have a machine with 10.15 available, so I have to resort to GitHub Actions and the help from others (such as https://github.com/Homebrew/homebrew-core/pull/60929#issuecomment-691434284) and from everything I've seen, 10.15 fails consistently. I assume you've got elf.h installed somehow, @wfurt?

[dagood]

@asbjornu I wonder, does Homebrew not require Xcode to be installed, and would that be where it normally comes from?

I'm not particularly familiar with the Homebrew project, but my impression is that it builds an OSS ecosystem on top of macOS and other OSes, so perhaps avoiding using Xcode is something they're interested in.

dotnet/runtime (and dotnet/coreclr) list Xcode as a requirement: https://github.com/dotnet/runtime/blob/master/docs/workflow/requirements/macos-requirements.md

I'm sure this can be patched to work with different dependencies, but I have no idea how extensive a patch would be or if it would be reasonable to try to maintain it in the dotnet org repos.

[wfurt]

I did not find elf.h on my system, besides src/coreclr/src/pal/src/libunwind_mac/include/elf.h - but I don't think that is the one as added recently. And I realized I don't build old coreclr repo but runtime. The code looks similar but there may be some caveats.

cc: @janvorli as expert in this area.

[janvorli]

I have upgraded my macOS to the latest version last week (10.15.6) and also the xcode to the latest one (11.7 IIRC). I can still build the runtime repo without issues. Let me check where the elf.h comes from.

[dagood]

The exact commit being built above is https://github.com/dotnet/coreclr/tree/v3.1.8, FYI in case this is different in runtime/5.0.

[janvorli]

I was able to build the coreclr repo release/3.1 just fine too.

[MikeMcQuaid]

mdfind -name elf.h may help you figure out where it comes from.

[janvorli]

So the build doesn't include the elf.h on my mac at all. Neither in coreclr nor in runtime repos. Using build.sh in coreclr and src/coreclr/build-runtime.sh in the runtime repo with no arguments. And it is not supposed to include it. I wonder if the problem somehow arises from the non-apple libunwind being installed and used by accident. It looks like in the runtime repo, it would cause the HAVE_UNW_GET_ACCESSORS to be detected. This is never detected on the apple version of libunwind as it is missing the accessor functions. And looking at the coreclr.log above, it is really the case, the HAVE_UNW_GET_ACCESSORS is true:

-- Performing Test HAVE_UNW_GET_ACCESSORS - Success

I can see that the build is being passed -DCLR_CMAKE_USE_SYSTEM_LIBUNWIND=TRUE, so it seems that somehow a non-apple libunwind.h is found on the include paths before the apple one.

You can verify which libunwind.h the build is including e.g. this way (assuming you've ran the build.sh for debug configuration before, even though it has failed):

cd  bin/obj/OSX.x64.Debug/src/pal/src/
make exception/remote-unwind.cpp.i

Now you can open ./CMakeFiles/coreclrpal.dir/exception/remote-unwind.cpp.i and search for libunwind.h in it. In my case, it is
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/libunwind.h.

[SMillerDev]

I just checked inside and outside the homebrew sandbox. It works outside of it, but not inside. In the sandbox I get the following failure if I try and run make exception/remote-unwind.cpp.i @MikeMcQuaid any ideas?

Preprocessing CXX source to CMakeFiles/coreclrpal.dir/exception/remote-unwind.cpp.i
/tmp/dotnet-20200914-75099-1je1col/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5/src/pal/src/exception/remote-unwind.cpp:68:10: fatal error:
      'elf.h' file not found
#include <elf.h>
         ^~~~~~~
1 error generated.
make[1]: *** [src/pal/src/CMakeFiles/coreclrpal.dir/exception/remote-unwind.cpp.i] Error 1
make[1]: *** Deleting file `src/pal/src/CMakeFiles/coreclrpal.dir/exception/remote-unwind.cpp.i'
make: *** [exception/remote-unwind.cpp.i] Error 2

[janvorli]

@SMillerDev just delete the line that includes elf.h when generating the preprocessing file. Then the preprocessing should succeed and you should be able to see the path to the libunwind.h.

[SMillerDev]

Thanks, it's /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/libunwind.h within homebrew. Could it be that the check is based on the filepath?

[janvorli]

No, the check is based on whether the following compiles:
https://github.com/dotnet/runtime/blob/75495b773900146fd394b7b0a0e0a02d5dd86928/src/coreclr/src/pal/src/configure.cmake#L1077-L1085
The only idea I have is that the cmake configuration somehow ends up checking the include path with some other libunwind.h.
Can you please share the artifacts/obj/coreclr/OSX.x64.Debug/CMakeFiles/CMakeOutput.log file via gist?

[SMillerDev]

Can you please share the artifacts/obj/coreclr/OSX.x64.Debug/CMakeFiles/CMakeOutput.log file via gist?

What is this path relative too? I can't seem to find it.

[SMillerDev]

I have a file /tmp/dotnet-20200914-75099-1je1col/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5/bin/obj/OSX.x64.Release/CMakeFiles/CMakeOutput.log that sorta matches the path you described.

CMakeOutput.log

[janvorli]

I am sorry, I've accidentally given you path for the runtime repo, not the coreclr one. The path you have found is correct.

[janvorli]

Right, the output file proves that the cmake check has found a libunwind.h that contained definitions for unw_get_accessors.
It detects that by trying to compile the following code:

#include <libunwind.h>

int main(int argc, char **argv) {
  unw_addr_space_t as;
  unw_get_accessors(as);

  return 0;
}

This should fail to compile with the default macos libunwind. The log shows the following command line being used to compile the file:

/usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang -D_FILE_OFFSET_BITS=64  -DHAVE_UNW_GET_ACCESSORS -c -Werror=implicit-function-declaration -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -o CMakeFiles/cmTC_b34a8.dir/src.c.o -c /tmp/dotnet-20200914-75099-1je1col/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5/bin/obj/OSX.x64.Release/CMakeFiles/CMakeTmp/src.c

You can try to run it manually by creating the source file with the above mentioned context anywhere, naming it e.g. test.c and then running

/usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang -D_FILE_OFFSET_BITS=64  -DHAVE_UNW_GET_ACCESSORS -c -Werror=implicit-function-declaration -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -o test.c.o -c test.c

It should fail with:

testhaveunwgetaccessors.c:5:3: error: implicit declaration of function 'unw_get_accessors' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  unw_get_accessors(as);

[SMillerDev]

When I do that it fails as expected. Any way I can run this in a cmake context somehow to figure out if that's the problem?

[janvorli]

You could try to let the compiler at the check in cmake dump more details using additional clang command line option.
Please add the following line

set(CMAKE_REQUIRED_FLAGS "-c -H -Werror=implicit-function-declaration")

Here:
https://github.com/dotnet/runtime/blob/6ed3261974e493a54ec4a97ff281381960cf82fc/src/coreclr/src/pal/src/configure.cmake#L1076

This will dump the whole include tree the compiler has used into the CMakeOutput.log

[SMillerDev]

I did that and assumed I needed to run these commands again but nothing seems to have changed.

[janvorli]

No, you need to delete the whole bin/obj folder and run regular coreclr build. I am sorry for forgetting to mention that.

[SMillerDev]

Here's the HAVE_UNW_GET_ACCESSORS part of the log. None of the includes look weird to me.

Performing C SOURCE FILE Test HAVE_UNW_GET_ACCESSORS succeeded with the following output:
Change Dir: /tmp/dotnet-20200914-75099-1je1col/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5/bin/obj/OSX.x64.Debug/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/local/Homebrew/Library/Homebrew/shims/mac/super/gmake cmTC_c09ae/fast && /Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/cmTC_c09ae.dir/build.make CMakeFiles/cmTC_c09ae.dir/build
Building C object CMakeFiles/cmTC_c09ae.dir/src.c.o
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang -D_FILE_OFFSET_BITS=64  -DHAVE_UNW_GET_ACCESSORS -c -H -Werror=implicit-function-declaration -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -o CMakeFiles/cmTC_c09ae.dir/src.c.o -c /tmp/dotnet-20200914-75099-1je1col/artifacts/src/coreclr.f897710bc4efe6a046068fde0acf641667a8fff5/bin/obj/OSX.x64.Debug/CMakeFiles/CMakeTmp/src.c
. /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/libunwind.h
.. /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include/stdint.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdint.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_int8_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_int16_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_int32_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_int64_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types/_uint8_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types/_uint16_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types/_uint32_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types/_uint64_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_symbol_aliasing.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_posix_availability.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/i386/_types.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_pthread/_pthread_types.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_intptr_t.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/types.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/i386/types.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_u_int8_t.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_u_int16_t.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_u_int32_t.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_u_int64_t.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_intptr_t.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_uintptr_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types/_intmax_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types/_uintmax_t.h
.. /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include/stddef.h
... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include/__stddef_max_align_t.h
.. /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/mach_types.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/host_info.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/message.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/port.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/boolean.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/boolean.h
........ /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/boolean.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/vm_types.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/vm_types.h
........ /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/vm_param.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_mach_port_t.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/kern_return.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/kern_return.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/kern_return.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/appleapiopts.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/Availability.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/AvailabilityInternal.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_statistics.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/time_value.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/host_notify.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/host_special_ports.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/memory_object_types.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_prot.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_sync.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_types.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/exception_types.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/exception.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/exception.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/thread_status.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/thread_status.h
...... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/thread_status.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/_structs.h
........ /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/_structs.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/fp_reg.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/thread_state.h
....... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/i386/eflags.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/thread_state.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/mach_voucher_types.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/std_types.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_uuid_t.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/processor_info.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/processor_info.h
..... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/i386/processor_info.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/task_info.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/policy.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/task_inspect.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/task_policy.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/mach_types.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/task_special_ports.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/thread_info.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/thread_policy.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/mach_types.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/thread_special_ports.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/clock_types.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_attributes.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_inherit.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_purgable.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_behavior.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/vm_region.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/machine/vm_param.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/kmod.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/mach_types.h
... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/mach/dyld_kernel.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_fsid_t.h
.... /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_fsobj_id_t.h
Linking C executable cmTC_c09ae
/usr/local/Cellar/cmake/3.18.2/bin/cmake -E cmake_link_script CMakeFiles/cmTC_c09ae.dir/link.txt --verbose=1
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang  -DHAVE_UNW_GET_ACCESSORS -c -H -Werror=implicit-function-declaration -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names   -mmacosx-version-min=10.12  CMakeFiles/cmTC_c09ae.dir/src.c.o -o cmTC_c09ae 


Source file was:

#include <libunwind.h>

int main(int argc, char **argv) {
  unw_addr_space_t as;
  unw_get_accessors(as);

  return 0;
}

[janvorli]

This is really strange. It includes the correct libunwind.h and yet it can see the unw_get_accessors that is not defined there.
Just as a sanity check, can you do this, please?

grep --context=7 -e unw_get_accessors /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/libunwind.h

On my machine, it shows just

/*
 * traditional libuwind "remote" API
 *   NOT IMPLEMENTED on Mac OS X
 *
 * extern int               unw_init_remote(unw_cursor_t*, unw_addr_space_t, thread_t*);
 * extern unw_accessors_t   unw_get_accessors(unw_addr_space_t);
 * extern unw_addr_space_t  unw_create_addr_space(unw_accessors_t, int);
 * extern void              unw_flush_cache(unw_addr_space_t, unw_word_t, unw_word_t);
 * extern int               unw_set_caching_policy(unw_addr_space_t, unw_caching_policy_t);
 * extern void              _U_dyn_register(unw_dyn_info_t*);
 * extern void              _U_dyn_cancel(unw_dyn_info_t*);
 */

[SMillerDev]

Yeah, that shows the same thing for me.

[asbjornu]

Thanks for the time you've invested in this, @SMillerDev and @janvorli. Anything I can do to help progress this issue?

[janvorli]

@asbjornu is there an easy way how I can try to repro the issue locally? I would like to check whether is it a problem specific to certain configurations or whether it is a general problem that occurs even on my local device. I don't know much about the homebrew.

[SMillerDev]

If you have a macOS device, you'd need to:

1. install Homebrew using the link on https://brew.sh/.
2. download the file added in https://github.com/Homebrew/homebrew-core/pull/60929
3. install it using brew install --debug --keep-tmp somefile.rb

That will allow you to drop into a shell and look around in the build directorty after a build fails (that's how I created most output).
Additionally it'll also keep the temporary build directory and print the location if you quit that shell. You can then run brew sh to start a new brew shell later.

---

EDIT: if you don't have a macOS device it might also work on linux but that's not what the formula is written to support.

[janvorli]

Thank you @SMillerDev, I have a mac mini that I use for local development, so I can test it on it.

[janvorli]

I confirm that I hit the same issue when building on my device. However, I don't hit it if I just check out and build the source-build repo manually. Does brew itself introduce any environment variables that would change compiler settings?

[SMillerDev]

It mostly just removes a lot of them. If there is any specific ones you're wondering about maybe @MikeMcQuaid can help.

[MikeMcQuaid]

It avoids picking things up from inside Homebrew that haven't been specified. As mentioned: it changes a lot of things, too many to spell out in particular. We can do whatever we can to help debug these, though, and point towards logging.

[janvorli]

It would be helpful to dump all the env vars at the time the build.sh is called so that I can try to see what could possibly interfere. Is it something that I can somehow do easily?

[dagood]

binlogs keep track of all the env vars at the time of the build, and shows them up at the top in the binlog viewer. Source-build infra has a binlog, and we generate them for each repo we build. Try find . -iname '*.binlog'.

[MikeMcQuaid]

Similarly if you build with --debug or --keep-tmp you can see all the Homebrew logs in $HOME/Library/Logs/Homebrew and in the build directory.

[SMillerDev]

Additionally if you drop into the shell after brew install --debug fails you can just print the environment variables

[janvorli]

So, I've debugged it a bit and found that for some reason, the compilation of the test code during cmake configuration succeeds even if I put a call to a fake function there. For example, this still successfully compiles:

#include <libunwind.h>

int main(int argc, char **argv) {
  unw_addr_space_t as;
  unw_get_accessors(as);
  foofoofoo();
  return 0;
}

Since the C compiler was invoked with -Werror=implicit-function-declaration, it is strange it doesn't fail and it is even stranger that it behaves differently for build using homebrew and a build out of homebrew.
I've checked the env variables in the homebrew shell, but I cannot see anything that could alter the behavior of the clang compiler.

I've tried to change the check_c_source_compiles to check_cxx_source_compiles hoping that the c++ compiler would be stricter and the check started to behave as expected. We can change it that way, but I'd really like to understand why the C compiler doesn't honor the -Werror=implicit-function-declaration when running under brew.

[janvorli]

So, I've found a bug in configure.cmake, the following should not include the -c

set(CMAKE_REQUIRED_FLAGS "-c -Werror=implicit-function-declaration")

Removing the -c makes the check work correctly (the compilation of that code fails). I have not realized before that the /usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang is a quite complex ruby script. So it looks like the extra -c option has somehow confused it. I've added printing the arguments it passes to the real clang to this script and

[janvorli]

@asbjornu Is there a way to patch this in the homebrew recipe or will you need to wait for the fix in the coreclr repo?

[SMillerDev]

I think build.sh downloads that repo, so I doubt it would be possible to patch it.

[asbjornu]

Yes, we will just have to wait for a fix.

[omajid]

I think build.sh downloads that repo, so I doubt it would be possible to patch it.

Can you place a file in a repo after cloning it? If so, you can place a file at patches/coreclr/thisfix.patch inside https://github.com/dotnet/source-build.git after cloning it. That patch will be automatically applied to the coreclr repo after the coreclr repo is cloned by build.sh in the source-build.git repo.

[dagood]

Can you place a file in a repo after cloning it?

Yep, I poked at this a bit with an earlier macOS-specific issue at https://github.com/dotnet/source-build/issues/1685#issuecomment-675476767:

Or you should be able to fairly easily incorporate the fix (https://github.com/dotnet/coreclr/pull/28066) as a patch file in your own process. This might end up being a patch file (https://github.com/Homebrew/formula-patches) that in turn adds a patch file into source-build.

[dagood]

That said, if someone would like to contribute a patch file into dotnet/source-build release/3.1 directly, that seems fine to me and avoids any potential limits in the homebrew infra. Then, you could use the tip release/3.1 commit without waiting for the next release. This CoreCLR patch seems like something small that there wouldn't be much concern maintaining here until a future CoreCLR version incorporates it.

[dseefeld]

Fixed with #1789 for release/3.1. Fix is already merged in runtime master and release/5.0