Brew: pkg-config files on >=10.14 assume headers in /usr/include

[railwaycat]

Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

• [x] are reporting a bug others will be able to reproduce and not asking a question. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh
• [x] ran a brew command and reproduced the problem with multiple formulae? If it's a problem with a single, official formula (not cask) please file this issue at Homebrew/homebrew-core: https://github.com/Homebrew/homebrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
• [x] ran brew update and can still reproduce the problem?
• [x] ran brew doctor, fixed all issues and can still reproduce the problem?
• [x] ran brew config and brew doctor and included their output with your issue?

% brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: The following directories do not exist:
/usr/local/sbin

You should create these directories and change their ownership to your account.
  sudo mkdir -p /usr/local/sbin
  sudo chown -R $(whoami) /usr/local/sbin
% brew config
HOMEBREW_VERSION: 1.7.6-143-gc6acab6
ORIGIN: https://github.com/Homebrew/brew
HEAD: c6acab66feb63743fe69b38dab7dc7b90710e838
Last commit: 2 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 537f3703aa41c550edbaaf28dd280c6ec929f946
Core tap last commit: 34 minutes ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_DEV_CMD_RUN: 1
CPU: octa-core 64-bit kabylake
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 10.0 build 1000
Git: 2.17.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.54.0 => /usr/bin/curl
macOS: 10.14-x86_64
CLT: 10.0.0.0.1.1535735448
Xcode: 10.0
XQuartz: 2.7.11 => /opt/X11

To help us debug your issue please explain:

• What you were trying to do (and why)
  I tried to build a package which uses pkg-config from Homebrew.

• What happened (include command output)
  I met an error when build:

fatal error: 'libxml/tree.h' file not found
#include <libxml/tree.h>
         ^~~~~~~~~~~~~~~
1 error generated.

It looks like pkg-config from Homebrew is not providing the correct include directory of libs shipping with macOS(Xcode and CLI tools). As the release note of Xcode 10, header files which were under /usr/include no longer exists and been moved to Xcode: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include and CLI tools: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include.

output:

% which pkg-config
/usr/local/bin/pkg-config

% pkg-config --cflags --libs "libxml-2.0 > 2.6.17"
-I/usr/include/libxml2 -lxml2

% pkg-config --variable pcfiledir "libxml-2.0 > 2.6.17"
/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.14

% cat /usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.14/libxml-2.0.pc 
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
modules=1

Name: libXML
Version: 2.9.4
Description: libXML library version2.
Requires:
Libs: -L${libdir} -lxml2
Libs.private: -lz -lpthread -licucore -lm
Cflags: -I${includedir}/libxml2

I tested only with libxml2 but I believe this issue affect all libs which have pkg-config file from Homebrew Library/Homebrew/os/mac/pkgconfig/10.14, they are: libcurl.pc, libexslt.pc, libxml-2.0.pc, libxslt.pc, sqlite3.pc, zlib.pc.

• What you expected to happen

% pkg-config --cflags --libs "libxml-2.0 > 2.6.17"

should gives the correct path to the header files under Xcode or CLI tools.

• Step-by-step reproduction instructions (by running brew commands)
• brew install pkg-config
• pkg-config --cflags --libs "libxml-2.0 > 2.6.17"

[railwaycat]

Quote from release notes of Xcode 10:

The Command Line Tools package installs the macOS system headers inside the macOS SDK. Software that compiles with the installed tools will search for headers within the macOS SDK provided by either Xcode at:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
or the Command Line Tools at:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
depending on which is selected using xcode-select.

I think we can make the modification in each *.pc file under Library/Homebrew/os/mac/pkgconfig/10.14 to use one of the paths above instead of /user/include.

But the question is which one should we pick? Can we assume all Homebrew users have Xcode or have Command Line tools?

It says xcode-select will gives the path of current using, unfortunately we cannot run external command in *.pc files.

[MikeMcQuaid]

Can we assume all Homebrew users have Xcode or have Command Line tools?

We cannot assume either. It may be we need separate files for each.

[tomkrouper]

I did discover I could do this as a work around. It certainly isn't a final solution, but for people who are stuck, this adds files to /usr/include.

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

[MikeMcQuaid]

@tomkrouper Unfortunately that package isn't recommended by Apple and is likely to break things in future in Homebrew 😭

[railwaycat]

@tomkrouper Thanks for the update! Yes, this is the workaround in release notes of Xcode, and these words are also mentioned:

If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so. As a workaround, an extra package is provided which will install the headers to the base system. In a future release, this package will no longer be provided.

So even it works for now, soon or later we still need a fix..

[fxcoudert]

This is causing build failure in root: https://github.com/Homebrew/homebrew-core/issues/32949#issuecomment-431289362

[fxcoudert]

This issue is actually not limited to libxml2. All .pc files in Library/Homebrew/os/mac/pkgconfig/10.14 assume that headers are in /usr/include, which is definitely not true anymore. This is only manifesting in libxml2 because its headers are in /usr/include/libxml2, while other headers are all directly in /usr/include and pkg-config appears to filter that one out…

[fxcoudert]

I think PKG_CONFIG_SYSROOT_DIR is a possible solution here.

1. brew would set PKG_CONFIG_SYSROOT_DIR in the environment to point to the right directory
2. .pc files can be rewritten to use pc_sysrootdir
3. everybody wins

[MikeMcQuaid]

@fxcoudert Sounds good 👍

[bmlett]

Hello-

I have just run into this issue using homebrew on macOS High Sierra v10.13.4 to install root and I am wondering how I should go about fixing this error. I am fairly new with mac and do not follow the above comment enough to know what commands I'd need to run. Any suggestions would be greatly appreciated!

[MikeMcQuaid]

@bmlett The details of a possible fix are in @fxcoudert's comment above.

[gwynne]

@fxcoudert's solution is an excellent one, with one caveat - when setting the sysroot, brew should verify that the SDK in use corresponds to the current OS version. While, for example, using the 10.13 SDK to target 10.14 (as might happen if the user has Xcode 9.4 installed and selected on Mojave) will work most of the time, it's obviously not ideal. Whether an SDK/OS mismatch should be a hard error (with override possible), a warning, or a note in brew doctor, I'm not sure. (Though since the selected Xcode (and thus available SDKs) can change at any time, I imagine it's probably better as at least a warning.)

[scpeters]

I just tested this a bit and the PKG_CONFIG_SYSROOT_DIR environment variable and pc_sysrootdir variable inside .pc files, seems like a good fit. I tried the following patch:

~~~
diff --git a/Library/Homebrew/os/mac/pkgconfig/10.14/libxml-2.0.pc b/Library/Homebrew/os/mac/pkgconfig/10.14/libxml-2.0.pc
index c297c6b454..7e467b9fc6 100644
--- a/Library/Homebrew/os/mac/pkgconfig/10.14/libxml-2.0.pc
+++ b/Library/Homebrew/os/mac/pkgconfig/10.14/libxml-2.0.pc
@@ -1,7 +1,7 @@
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
-includedir=${prefix}/include
+includedir=${pc_sysrootdir}${prefix}/include
modules=1

Name: libXML
~~~

along with export PKG_CONFIG_SYSROOT_DIR=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk and got the following:

~
$ export PKG_CONFIG_SYSROOT_DIR=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
$ $ pkg-config --cflags --libs "libxml-2.0 > 2.6.17"
-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/libxml2 -lxml2
~

Are we intending to set this environment variable in super.rb? We may need to add a caveat or some other messaging for people hoping to use pkg-config outside of a brew command that they need to set this variable themselves.

[scpeters]

Also, setting PKG_CONFIG_SYSROOT_DIR globally can cause other pkg-config files to fail, as I just noticed as I moved on to another task in the same terminal that I did this testing:

~~~
CMake Error in src/CMakeLists.txt:
Imported target "ignition-common3::requested" includes non-existent path

"/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/Cellar/ossp-uuid/1.6.2_2/include/ossp"

in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include:
~~~

[scpeters]

@fxcoudert any thoughts on how the environment variable should be set? super.rb?

[MikeMcQuaid]

I don’t think setting the environment variable is a good option if it fixes things sometimes and breaks things other times.

[fxcoudert]

Apparently PKG_CONFIG_SYSROOT_DIR has two effects: it sets pc_sysrootdir (which we want), but also inserts that value as a new root in all -I and -L flags, which we definitely don't want.

Another option is to use superenv to always pass --define-variable= to define our own variable, which we will then use in Homebrew's .pc files.

[MikeMcQuaid]

Another option is to use superenv to always pass --define-variable= to define our own variable, which we will then use in Homebrew's .pc files.

This could work. Would we need a pkg-config shim?

Honestly I'm still pretty tempted by the "have duplicates" option; the overhead isn't bad, it doesn't exceed the rule of three and is the simplest to edit/debug/etc.

Thoughts?

[fxcoudert]

Would we need a pkg-config shim?

Yeah, and that's an added cost in terms of complexity, sadly.

Honestly I'm still pretty tempted by the "have duplicates" option

That work for CLT, where things are in /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk. But how would it work for non-CLT users, where things are in Xcode.app?

[MikeMcQuaid]

We assume a default Xcode location or fail the build, I think.

[gwynne]

Using a default Xcode location will definitely fail for a large number of common scenarios - the only guaranteed correct behavior is to use the result of running xcrun -sdk macosx -show-sdk-path at the time of invocation (i.e. don't cache the result). Despite the complexity, a pkg-config shim seems the best option to me; "have duplicates" runs afoul of too many common configurations.

[MikeMcQuaid]

This is only in the Xcode-only case on certain OSs so is not a widespread problem.

At this point it's pretty much "whoever can first create a PR to solve the problem(s) here will probably get their approach merged". I think we're probably reaching the end of discussion being more valuable than (any) code approach.

[MikeMcQuaid]

This is only in the Xcode-only case on certain OSs so is not a widespread problem.

Or, to be more exact, would not be a problem for the majority of our users.

[gwynne]

As said in the release notes, the CLT are going away, which means it will probably eventually be _everyone_'s problem.

[MikeMcQuaid]

@gwynne As I've said to the other Apple employees that have raised this with me previously (and feel free to email me at [email protected] if you want to be in the Slack private room for Apple employees and Homebrew maintainers to talk): I don't see a solution to problems that led Homebrew maintainers (among others) to campaign for the CLTs creation such as:

• the ability to install it non-interactively (without an unsupported tool such as mas)
• the huge size difference between the CLT and Xcode
• the ability to download it without an Apple account

If all of the above were resolved I'd be more confident in Homebrew being able to move to "Xcode-only" being a well supported configuration and eventually primary one. Even now we get backlash from both sides because some people don't want to install Xcode (see reasons above) and others claim the CLT is identical to Xcode so there's no reason they should ever have it installed (which is not the case when using Homebrew).

[MikeMcQuaid]

@gwynne As a more meta-complaint that's not specific to you but is more a comment on stuff like this in general: it'd be really great if the relevant teams in Apple could figure out how it wants to provide information and have discussions with Homebrew. If the private Slack room we currently have doesn't work: can someone suggest something that will?

[MikeMcQuaid]

For those watching: we've figured out some ways for Apple and us to talk.

---

Sounds like the pkg-config shim is now the best option.

[manchicken]

I’d be interested in helping out here if desired. It looks like y’all are well on your way to a solution, but on the off chance that you still need a hand, please let me know.

[MikeMcQuaid]

@manchicken Yes, thanks. A PR would be welcome.

[manchicken]

Is there an doc on the build process you all use that you could link me to? I can’t find it, and it seems a bad idea to install on my regular box.

~ Michael D. Stemle, Jr.

On Feb 23, 2019, at 07:50, Mike McQuaid [e-mail] wrote:

@manchicken Yes, thanks. A PR would be welcome.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[MikeMcQuaid]

Is there an doc on the build process you all use that you could link me to? I can’t find it, and it seems a bad idea to install on my regular box.

@manchicken I don't understand the question, sorry. It's fine to do Homebrew development on your normal Homebrew installation if that's what you mean.

[MikeMcQuaid]

Closing this out as we don't seem to have agreement on an approach that won't break current workflows. This will likely be handled by later macOS versions getting rid of some of these paths.

[MikeMcQuaid]

I think a/the workaround for this is that we use Homebrew dependencies for stuff that requires pkg-config files rather than creating our own for macOS/Xcode provided libraries that don't provide then.

[manchicken]

I think that makes sense, though it does mean potentially more double libs. That seems like the only way to make sure that we don’t break every time we have an update.

Not only does this break stuff for repo packages, but folks who rely on pkg-config for building modules with C bindings for dynamic language (like npm modules, or eggs, or gems, or Perl modules) also have to deal with this stuff.

~ Michael D. Stemle, Jr.

On Apr 19, 2019, at 00:29, Mike McQuaid [e-mail] wrote:

I think a/the workaround for this is that we use Homebrew dependencies for stuff that requires png-config files rather than creating our own for macOS/Xcode provided libraries that don't provide then.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[MikeMcQuaid]

Reopening as @pcr910303 opened https://github.com/Homebrew/brew/issues/6656 to address this.

The potential solutions I see:

• fix these paths so they match the command-line tools and mandate their installation
• fix these paths to mandate Xcode and mandate its installation (I'm pretty 👎 on this)
• have different pkg-config files for Xcode/CLT
• remove these pkg-config files entirely

[alanthird]

From my POV as an Emacs dev I'd be happy to have these removed as they completely break our build, which is supposed to detect the headers itself if it can't find them with pkg-config. I understand that that might break other applications, though.

Is there a good reason to prefer the Apple provided headers/libs over Homebrew's own packages?

[pcr910303]

@alanthird My understanding is that brew prefers using macOS-provided packages for everything. AFAIK it was the core differences between brew & macports.

[Bo98]

Just want to share my thoughts here. Hopefully I can perhaps try and get this moving and help towards picking a direction.

Firstly,

• fix these paths to mandate Xcode and mandate its installation (I'm pretty 👎 on this)

I am also :-1:, not necessarily because of Xcode being mandated but because this, from a technical sense, would not work correctly on anything that is not the latest OS. For example, the latest version of Xcode on Mojave does _not_ ship with Mojave system headers. This is almost certainly intentional. Though in those cases, I think Homebrew does require the CLT anyway (except if you use an older Xcode) unless I'm mistaken.

So for anything less than the latest macOS, it is perfectly fine, and expected, to mandate the CLT. As for the latest macOS, you do have a choice of what to use.

I however want to take a slight detour and raise a separate, but very much relevant, issue about pkg-config: the default search path. I'll explain why it plays a part in this dicussion in a moment.

Homebrew's pkg-config files are baked into the search path at compile time here. This works great except for one thing - when the user upgrades macOS. Homebrew does not deliver updates to pkg-config when upgrading macOS so until they reinstall pkg-config, it will continue to search at the path matching their previous macOS version. This behaviour is not clear at all and can cause some head scratches, but due to the lack of any real changes to the pkg-config files between macOS versions prior to now it hasn't really yet.

Now why is that relevant? Because this behaviour will start becoming a problem because there will be changes every macOS version. For simplicity sake, I'm going to ignore Xcode for now. If we focus on the CLT, we will have /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk for 10.14, /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk for 10.15 and so on. We will want to ensure that people using pkg-config don't end up using the wrong SDK for their system. The SDK might also not even be available. Previously it was easy as /usr/include would always point to the correct headers.

Note that /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk is _not_ sufficient. To demonstrate this, this is how the symlinks are mapped on a (non-broken) CLT 11.2 install on 10.14:

$ ls -l /Library/Developer/CommandLineTools/SDKs
total 0
lrwxr-xr-x 1 root wheel  15 Nov 15 21:28 MacOSX.sdk -> MacOSX10.15.sdk
drwxr-xr-x 7 root wheel 224 Aug 27 07:28 MacOSX10.14.sdk
drwxr-xr-x 8 root wheel 256 Oct 17 13:19 MacOSX10.15.sdk

You must change the include location for every version of macOS.

So if we go the static route, we should also have measures in place to have the pkg-config binary reinstalled if the macOS version mismatches what it is built for.

An alternative approach discussed in this issue was a dynamic-based approach, which I'll now discuss.

In terms of shims: we need to remember that the pkg-config files offered are not exclusively used in a Homebrew environment. But currently shims are pretty much exclusively for that environment. For as long as Homebrew's pkg-config files are supported outside the Homebrew envornment, any shim would need apply globally. So ideally, you would want to make it so that invoking /usr/local/bin/pkg-config would go through the shim and have the real binary elsewhere.

Alternatively you could withdraw support for these pkg-config files outside a Homebrew environment if you desire, which would avoid the need of that complication. I suppose it comes down to a balance of feasibility, time, effort and reward. Personally, it would be nice if we didn't have to, but I realise it may ultimately become the only option.

Just to put it out, with the goal of covering all possible options here, introducing the dynamic element via forking/patching pkg-config itself is also an option. An option I suspect I already know what the answer would be, but something I nevertheless wanted to make mention to (and will help out with if pursued) so that every possible direction can be said to have been considered.

I think I've covered everything. If there's a direction I forgot to mention, let me know. Regardless of which approach is taken here, I am willing to help in drafting up a proof of concept of it.

Apologies for the length of this - I've tried to be comprehensive here. Hopefully something I've said here will have been useful.

[MikeMcQuaid]

Hopefully I can perhaps try and get this moving and help towards picking a direction.

That would be great 🎉

Homebrew does not deliver updates to pkg-config when upgrading macOS so until they reinstall pkg-config, it will continue to search at the path matching their previous macOS version.

This is a good point. It's something we should probably add some sort of check to the formula for so when pkg-config is used as a dependency it can automatically reinstall. Presumably this also means we should never use or_later bottles for png-config? Alternatively, I wonder if we should specify multiple OS paths in this case and have them effectively just fall back if they don't exist 🤔

Alternatively you could withdraw support for these pkg-config files outside a Homebrew environment if you desire, which would avoid the need of that complication.

I'm not aware of our explicit support for this but, yeh, I think we should keep on the table that these files are unsupported outside of our environment.

Just to put it out, with the goal of covering all possible options here, introducing the dynamic element via forking/patching pkg-config itself is also an option.

Definitely don't want to fork but if we could have a small patch (particularly done with inreplace) and that results in a much less complicated solution over all: we should consider it.

Apologies for the length of this - I've tried to be comprehensive here. Hopefully something I've said here will have been useful.

Lots useful said, please don't apologise! Thanks for your thoughts on this and, hopefully, future PRs 😉

[Bo98]

I've seen several pull requests recently which could have found it useful to have this working so let's try progress this.

Firstly, what I propose the *.pc files look like is this:

prefix=${homebrew_sdkroot}/usr
exec_prefix=/usr
libdir=${exec_prefix}/lib
includedir=${prefix}/include

To fully solve this we need to:

• Remove this line from the formula file. Unless we implement functionality in brew which checks whether the formula was compiled on the current OS version, it has to go if we are going to point to specific SDKs.
• Supply an alternative means of supplying the *.pc path that is calculated at runtime - not at the compile time of pkg-config.
• Define homebrew_sdkroot to equal the SDK we want. Again this needs to be done at runtime (such as fetching the HOMEBREW_SDKROOT environment variable).

It turns out this isn't that complex. The approach here on depends on how much we care about pkg-config outside of the Homebrew superenv.

To make it work outside the Homebrew superenv, a patch is required. Here's one that should cover everything. All that's needed is to apply the patch, add ENV.append_to_cflags "-DHOMEBREW_LIBRARY=\\\"#{HOMEBREW_LIBRARY}\\\"" and remove the aforementioned existing *.pc path line. Nothing else required:

$ pkg-config --cflags libxml-2.0
-I/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/libxml2

Alternatively, we can avoid a patch by withdrawing the capability to use Homebrew's *.pc files outside of the Homebrew superenv. I've not fully tested this, but basically we would need to:

• Remove the aforementioned existing *.pc path line from the formula and revision bump (not technically required for the Homebrew superenv but should be done to formally drop support outside of it).
• Create a shim for pkg-config. This should supply --define-variable=homebrew_sdkroot=$HOMEBREW_SDKROOT to pkg-config.
• As for inserting the *.pc path, this is already done.

However, doing this would also be dropping support for Homebrew *.pc files for the std env (alongside outside of Homebrew altogether - though support isn't explicit there). This is because we don't shim in the std env. I know this isn't used in homebrew-core, but it could be a breaking change for some taps.

[MikeMcQuaid]

It turns out this isn't that complex. The approach here on depends on how much we care about pkg-config outside of the Homebrew superenv.

I would say: we don't care about it until we get (multiple) complaints.

However, doing this would also be dropping support for Homebrew *.pc files for the std env (alongside outside of Homebrew altogether - though support isn't explicit there). This is because we don't shim in the std env. I know this isn't used in homebrew-core, but it could be a breaking change for some taps.

I think this is reasonable given the unresolved issues in this issue.

Good call @Bo98, happy to help review a PR!

[tschoonj]

Removing support for the Homebrew supplied *.pc files in pkg-config is going to be problematic I think: I am experiencing a problem already trying to build some software manually against glib (which provides gio-2.0):

configure: error: Package requirements (gtkmm-3.0 >= 3.0.0) were not met:

Package 'zlib', required by 'gio-2.0', not found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables gtkmm_CFLAGS
and gtkmm_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

I think it's fair to say this will affect quite a few people, since lots of formulas depend on zlib.

[Bo98]

It is still available in the superenv. Is this for outside the Homebrew environment?

[tschoonj]

Yep: this is just for compiling a package manually using ./configure, make, make install using Homebrew provided packages.

[MikeMcQuaid]

I kinda consider Homebrew packages using Homebrew's superenv at runtime to be a bit of a smell.

[Bo98]

I suppose the Homebrew *.pc files never were explicitly part of the superenv. They're not even in the superenv folder. It probably predates the superenv but I haven't checked Git history.

I cannot see anything further than can be done beyond patching pkg-config, or somehow convincing Apple to ship *.pc files in the SDK (not going to happen).

Using brew sh is a workaround, but I know that's not really appropriate for all cases.

[tschoonj]

Patching pkg-config looks necessary I think. The patch you mentioned before looks good to me.

[MikeMcQuaid]

I don't want us carrying a pkg-config patch indefinitely that will never be upstreamed.

Note that Xcode-only means that all these uses of pkg-config would have already been broken. I don't think we should be worrying too/as much about how we change behaviour for stuff outside the Homebrew ecosystem. The pkg-config files we've been providing are effectively nice hacks to more easily use system libraries when you have the CLT installed. If that gets scoped to being only used for our use-case: I think the short-term pain is worth the long-term decrease in support surface.

That said: if we get large numbers of complaints/issues: we can consider another option.

[tschoonj]

I appreciate that carrying that patch around indefinitely is not ideal, but something definitely needs to happen.

Now that pkg-config no longer looks for Homebrew provided *.pc files, it will be very hard for lots of people to compile software manually using autotools, cmake, meson and others, as soon as a Homebrew library is required that drags in zlib or any of its friends.

When using pkg-config standalone, problems can be solved by passing --define-variable=homebrew_sdkroot=my-sdkroot, but to the best of my knowledge this same effect cannot be achieved with the aforementioned build tools.

Autotools lets you circumvent using pkg-config by specifying _CFLAGS and _LIBS flags but this is not very user friendly. I am not aware of similar workarounds being available for CMake and meson.

Assuming you're not willing to drop Apple provided libraries in favour of their Homebrew counterparts, the only solution I see is patching pkg-config. 😢

[Bo98]

Been thinking through this a bit. There is one alternative but would require some groundwork in place first. It would involve _all_ of the following occurring:

• We supply a default homebrew_sdkroot to the *.pc files. This would be the CLT so Xcode-only systems would not be supported.
  
  
  
  • This however can be overridden by the superenv shim so Xcode-only support can be retained there.
  
  
• We create a system where pkg-config is marked as outdated if it was built on a different major OS version than it was installed. I'm not talking about HOMEBREW_SKIP_OR_LATER_BOTTLES (though that would be a part of it), but rather if at any point a user upgrades macOS (excluding minor updates) then brew will immediately mark pkg-config as outdated. This would require to somewhere store the version of macOS used to build pkg-config - which may have been installed from source or from a bottle.
  
  
  
  • This does feel somewhat pkg-config specific though. That said, I am sure there are other formulae where this may be useful. For example, python3-config --cflags looks problematic for macOS upgrades due to the SDK path & Tk version being baked in, so python could be a potential candidate for using such feature. llvm would definitely be another one. I imagine it becomes quite broken on macOS upgrades.
  
  
• Then we can bake the Homebrew *.pc path into pkg-config as we were before (so basically partially reverting Homebrew/homebrew-core#52565).

[MikeMcQuaid]

We supply a default homebrew_sdkroot to the *.pc files. This would be the CLT so Xcode-only systems would not be supported.

• This however can be overridden by the superenv shim so Xcode-only support can be retained there.

@Bo98 I like the sound of that. Having it be CLT-only seems fine (particularly given this was already broken on Xcode-only and no-one complained).

We create a system where pkg-config is marked as outdated if it was built on a different major OS version than it was installed. I'm not talking about HOMEBREW_SKIP_OR_LATER_BOTTLES (though that would be a part of it), but rather if at any point a user upgrades macOS (excluding minor updates) then brew will immediately mark pkg-config as outdated. This would require to somewhere store the version of macOS used to build pkg-config - which may have been installed from source or from a bottle.

I think best we can probably do here is something in test do which necessitates a revision and a pour_bottle block which means the bottle isn't poured on newer versions. Beyond this we could consider an additional bottle DSL that indicates that this bottle won't work on later macOS versions.

• Then we can bake the Homebrew *.pc path into pkg-config as we were before (so basically partially reverting Homebrew/homebrew-core#52565).

Seems fine given the above 👍

[MikeMcQuaid]

That said: if we get large numbers of complaints/issues: we can consider another option.

To be explicit: I would consider us to now have "a large (enough) number of complaints".

[Bo98]

Just to be clear:

which necessitates a revision and a pour_bottle block

A pour_bottle will help new installations. A revision will be limited help as the problem isn’t the bottles themselves but prompting existing installations to reinstall at the right time - this isn’t something detectable by the CI since the concern is existing installations more than new installations. If we start adding fixed SDK locations, users would be required to brew reinstall pkg-config when upgrading macOS to prevent hard-to-debug errors. Most people will not do this.

[MikeMcQuaid]

A revision will be limited help as the problem isn’t the bottles themselves but prompting existing installations to reinstall at the right time - this isn’t something detectable by the CI since the concern is existing installations more than new installations.

You're right. The best bet here is just a brew doctor special-case, I think, which can also be run as a pre-install check (or just a pre-install check if anything in the dependency tree uses pkg-config).

[Bo98]

as a pre-install check

At that point you are one step away from making it a brew upgrade/outdated check and automating the reinstall.

special-case

Yeah, such check would probably be something like checking the output of pkg-config --variable=homebrew_sdkroot libxml-2.0, unless there's some other information somewhere that can be read which I'm not aware of.

[Bo98]

That output check actually will not work well if anyone had brew link libxml2. Argh, worst case scenario we touch some file in the install directory and check that.

[MikeMcQuaid]

At that point you are one step away from making it brew upgrade/outdated check and automating the reinstall.

Yep, I'm not opposed to that. Given that: it could be something we (ab)use the linkage checker checks to do.

Argh, worst case scenario we touch some file in the install directory and check that.

Yup. I wouldn't worry too much about the OS upgrade case yet, though, it's not as pressing (and can be pulled into a new issue).

[Bo98]

I wouldn't worry too much about the OS upgrade case yet, though, it's not as pressing

Yeah, the rest of what needs done is fairly simple.

One more thought on it though: any thoughts on adding the OS pkg-config was built on to the install receipt and using that?

[MikeMcQuaid]

One more thought on it though: any thoughts on adding the OS pkg-config was built on to the install receipt and using that?

That seems like a good idea (for all install receipts). Could also stick something like Xcode/CLT/SDK versions if you anticipate it being useful for fixing bugs in future.

[whitty]

I don't want to wade in in the wrong place, but my ticket #7292 was closed so I can't ask there.

I know this is an evolving situation, but this ticket is full of detailed discussion that is hard to parse for someone not knee deep in it. Could you perhaps synthesise a post with

• current state
• known workarounds/reverts
• expectation of duration (ie so people can decide whether to wait for a fix or try to revert)

and post it to each of the tickets you've closed, so that Google gives good results, but you don't get flooded with dumb questions on this thread.

PS:

(reverts especially welcome, because the core update seemed to break compatibility with existing bottles (eg python2), and since its discontinued (but that was hidden) I have also to fix up a whole legacy build-system in addition to pkg-config woes)

[MikeMcQuaid]

@whitty We have reverted the changes to pkg-config (brew upgrade) until we are ready with the new approach (which will continue to use Homebrew pkg-config files).

[0xTim]

Hi all,
Posting here as it seems the most relevant, but feel free to tell me to open a new issue!
TL;DR - #7277 has broken every Swift Package Manager project that relies on a system library.

So in short, that change seems to be adding the homebrew_sdkroot path to the includes directory, both in Xcode and when building from the command line when using SwiftPM. This seems to affect projects in Xcode if you open the manifest directory or use swift package generate-xcodeproj. You can see below the changes added to Xcode

This ends up causing long build errors all based around redefinition of modules as they're included by both Xcode and the command line tools. Is there a short workaround for this? I'm assuming this is a Homebrew issue as the above changed fixed it.

Thanks!

(PS there are also a couple of forum posts here and here tracking the issue.

[MikeMcQuaid]

TL;DR - #7277 has broken every Swift Package Manager project that relies on a system library.

The approach has changed multiple times since then. Additionally, Xcode and Swift don't provide or use pkg-config by default and certainly don't use Homebrew's packages. This is likely instead an issue with your specific project(s).

If you can reproduce an issue with actually running Homebrew commands or provided packages: we can investigate.

[dlbuckley]

This is likely instead an issue with your specific project(s).

Unfortunately this isn't the case, all of the projects were running fine until this (or related) change came in. One day it worked, and then after updating homebrew the exact same project with no changes, failed. This was then verified across multiple swift package projects that import system libraries.

Can I suggest this change is reverted until we get to the root of exactly what's going on as its essentially halted production of our product as there doesn't seem to be a way to downgrade homebrew to an older version to continue work. Unless you have any other suggestions @MikeMcQuaid ?

[0xTim]

@MikeMcQuaid SwiftPM relies on pkg-config to tell it what includes paths to use and what linker paths to use, as described here. So it looks whatever changes ended up being final in 2.2.12 are injecting in the CLT in the pkg-config's search path. Trying to work out how that's happening now

[MikeMcQuaid]

Firstly, drive-by 👎s of my comments without addressing my questions do nothing other than to demotivate me to help.

If you can reproduce an issue with actually running Homebrew commands or provided packages: we can investigate.

Can I suggest this change is reverted until we get to the root of exactly what's going on as its essentially halted production of our product as there doesn't seem to be a way to downgrade homebrew to an older version to continue work

If this has halted production of your product: your business needs to figure out how to better deploy your upgrades of open source projects run by volunteers.

Unless you have any other suggestions @MikeMcQuaid ?

Someone needs to provide: step-by-step reproduction instructions (with as minimal input data as possible) running brew and pkg-config commands (not SwiftPM or Xcode).

If we get that, someone can clearly demonstrate the problem and it cannot be fixed without a revert: we'll consider one.

So it looks whatever changes ended up being final in 2.2.12 are injecting in the CLT in the pkg-config's search path.

The CLT does not provide pkg-config, Homebrew provides pkg-config. If SwiftPM is relying on Homebrew's pkg-config to provide output in a format that's not documented or stated anywhere on our end: that's SwiftPM's bug to be resolved.

Come on, folks, we're all (presumably?) engineers here. Put yourselves in our shoes: someone shows up saying "X broke something and needs to be reverted" but provides no reproduction instructions outside of "my project isn't working". Be the bug report you want to see in the world!

[soonho-tri]

FYI, packages using waf + pkg-config are broken.

One example: brew install sratom --build-from-source. It generates the following error message:

Checking for program 'pkg-config'             : /usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config
Checking for 'lv2 >= 1.16.0'                  : not found
The configuration failed

My understanding is that when this line, https://github.com/Homebrew/brew/blob/master/Library/Homebrew/shims/mac/super/pkg-config#L3, is executed, HOMEBREW_OPT is not properly set (because it's called from waf?). As a result, it fails to call the proper pkg-config (which is usually at /usr/local/opt/pkg-config).

There are more examples. Please let me know if you want me to open a separate issue.

/cc @bo98 who created the file, Library/Homebrew/shims/mac/super/pkg-config.

[Bo98]

That's likely a completely separate issue from SwiftPM.

Calling Library/Homebrew/shims/mac/super/pkg-config when you run brew install is intentional. HOMEBREW_OPT is set in the superenv here: https://github.com/Homebrew/brew/blob/e12da9491d2dc557636458c265196fedde34a30a/Library/Homebrew/extend/ENV/super.rb#L54

Is there more verbose logs than "the configuration failed"? That really says nothing about _why_ it failed.

[dlbuckley]

If this has halted production of your product: your business needs to figure out how to better deploy your upgrades of open source projects run by volunteers.

@MikeMcQuaid I don't think this comment was really called for 😞 We are just trying to figure out what's going on and have narrowed it down to some change in homebrew. Comments like this push away people who are actively trying to find issues and contribute.

As for 'our business'; we haven't even reached deployment stage, this is an issue that has effected all of our local development environments.

Regarding reproduction steps; we are trying to figure out a small and simple way to reproduce it on the Vapor discord channel now. You are welcome to join if you like.

[Bo98]

Re: SwiftPM, does this work for you guys? Homebrew/homebrew-core#52820

[soonho-tri]

Is there more verbose logs than "the configuration failed"? That really says nothing about why it failed.

@Bo98, This is part of /private/tmp/sratom-20200410-9155-ehu8vz/sratom-0.6.4/build/config.log.

['/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config', 'lv2 >= 1.16.0', '--cflags', '--libs', 'lv2', '>=', '1.16.0']
not found

Still not useful? So I've modified Library/Homebrew/shims/mac/super/pkg-config to check why it generates not found. As I mentioned above, when this file is executed from waf, the HOMEBREW_OPT is not set and /usr/local/opt/pkg-config is not called. It's true that the environment variable is set correctly when waf is called from homebrew though.

[Bo98]

That's useful. I'll have a look.

[0xTim]

@Bo98 honestly I have no idea! 😅 I'm currently trying to work out how SwiftPM uses pkg-config and how pkg-config for something like /usr/local/Cellar/libressl/3.0.2/lib/pkgconfig/openssl.pc is getting it to include multiple module maps. The errors are lots of variations on:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/module.modulemap:1:8: error: redefinition of module 'AppleTextureEncoder'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/module.modulemap:1:8: error: redefinition of module 'AppleTextureEncoder'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/module.modulemap:1:8: error: redefinition of module 'AppleTextureEncoder'
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/module.modulemap:1:8: error: redefinition of module 'AppleTextureEncoder'
module AppleTextureEncoder [system] [extern_c] {
       ^
module AppleTextureEncoder [system] [extern_c] {
       ^
module AppleTextureEncoder [system] [extern_c] {
       ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/module.modulemap:1:8: note: previously defined here
module AppleTextureEncoder [system] [extern_c] {
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/module.modulemap:7:8: error: redefinition of module 'Compression'
module Compression [system] [extern_c] {
       ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/module.modulemap:7:8: note: previously defined here
module Compression [system] [extern_c] {
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/module.modulemap:13:8: error: redefinition of module 'Darwin'
module AppleTextureEncoder [system] [extern_c] {
       ^

@MikeMcQuaid apologies if I came off as accusationary in my initial comment, to be clear I'm not saying it's definitely a Homebrew error - it could well be SwiftPM using something they shouldn't be, but the change for the multiple people reporting the errors was a Homebrew upgrade and I managed to track down the inclusion of the CLTs, which matches the errors, so Homebrew was my first point to raise it

[Bo98]

honestly I have no idea!

Well I guess we'll find out when I merge the pull request.

getting it to include multiple module maps

Apple's SDK system is extremely fragile, particularly after they moved to shipping /usr/include headers in the SDK.

The way Apple originally designed SDKs and the way system headers are meant to be used are incompatible in many ways - you simply cannot use the correct system headers and the latest SDK at the same time if you are not on the latest OS. Things also break when _both_ the CLT & Xcode are used, despite the SDK contents being identical.

With that said, hopefully the linked pull request fixes the SwiftPM problems. It should have the effect that pkg-config --cflags libcurl returns nothing, but pkg-config --cflags libxml-2.0 does (since libxml isn't in a "standard" location). This should avoid the -I flag appearing in your Xcode builds and make it work out-of-the-box, at the cost that it doesn't enforce the correct SDK should you end up using system libraries like Curl (though it was like that before).

[0xTim]

@Bo98 so after more digging - swift build will run pkg-config --variable pc_path pkg-config before actually building any source files. On 2.2.11, in zlib.pc in /usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.15/zlib.pc it contains:

prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
sharedlibdir=${libdir}
includedir=${prefix}/include

Name: zlib
Description: zlib compression library
Version: 1.2.11

Requires:
Libs: -L${libdir} -L${sharedlibdir} -lz
Cflags: -I${includedir}

In 2.2.12, with the changes, the file now looks like

homebrew_sdkroot=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
prefix=${homebrew_sdkroot}/usr
exec_prefix=/usr
libdir=${exec_prefix}/lib
sharedlibdir=${libdir}
includedir=${prefix}/include

Name: zlib
Description: zlib compression library
Version: 1.2.11

Requires:
Libs: -L${libdir} -L${sharedlibdir} -lz
Cflags: -I${includedir}

That would explain why the C flags produced would now include the CLTs

[soonho-tri]

@Bo98 , it seems like my problem is my own as I can't reproduce it on another mac. Sorry for wasting your time.

[Bo98]

@soonho-tri No worries. If you figure out what happened, let me know.

[0xTim]

Additionally, it looks like SwiftPM parses the .pc files to get things like the C flags (see here). I have no experience with pkg-config so no idea if this is correct or not, but pkg-config zlib -cflags does result in -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include

[Bo98]

Yes, and when my pull request is merged, it will then return nothing:

$ pkg-config --cflags zlib

It will basically filter out <SDK>/usr/include.

Out of the ones Homebrew ship, only libxml-2.0 and uuid will return something after the upcoming change since they have slightly different include paths.

[0xTim]

@Bo98 awesome, thanks for the help!

[dlbuckley]

Thanks for the help @Bo98

[0xTim]

@Bo98 by the way, is there an easy way for me to test this locally?

[MikeMcQuaid]

@MikeMcQuaid I don't think this comment was really called for 😞 We are just trying to figure out what's going on and have narrowed it down to some change in homebrew. Comments like this push away people who are actively trying to find issues and contribute.

@dlbuckley We're a group of unpaid volunteers working on crucial software for millions of people. Comments like "as its essentially halted production of our product as there doesn't seem to be a way to downgrade homebrew to an older version to continue work" do not contribute to the discussion or finding a solution. We do not consider that a useful contribution to Homebrew.

You can stop Homebrew from autoupdating, Homebrew only upgrades when you run commands and (with a bit of research) you'd be able to figure out that you can roll back any Homebrew change if needed. If you're going to bring your business into it (rather that just talk about the implementation details): you may get feedback on how to avoid future disruption to your business.

@MikeMcQuaid apologies if I came off as accusationary in my initial comment, to be clear I'm not saying it's definitely a Homebrew error - it could well be SwiftPM using something they shouldn't be, but the change for the multiple people reporting the errors was a Homebrew upgrade and I managed to track down the inclusion of the CLTs, which matches the errors, so Homebrew was my first point to raise it

@0xTim No worries, appreciate the apology ❤️. In future it's generally worth debugging "down the stack" i.e. report this to SwiftPM and only file here if they've explicitly said it's a Homebrew issue.

[ydnar]

Hi folks, it looks like this also broke building Go packages that depend on pkg-config for CGO dependencies.

Some sample output from a Go project:

❯ go test ./...
# /usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config --cflags  -- opus opus opus opusfile opus opusfile
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: /pkg-config/bin/pkg-config: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: exec: /pkg-config/bin/pkg-config: cannot execute: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: exit status 126

Looks like HOMEBREW_OPT isn’t set. I’d love to help—how best to?

[Bo98]

Why is Go using the superenv shims outside of the Homebrew environment? The shims are strictly only for use inside the superenv.

[Bo98]

@0xTim You could maybe modify and build a version of pkg-config locally yourself. Otherwise, the fix will be out in a couple hours if all goes well.

[ydnar]

Why is Go using the superenv shims outside of the Homebrew environment? The shims are strictly only for use inside the superenv.

Great question. This is on a Homebrew-installed Go (1.14.2). What enables superenv?

[Bo98]

brew sh allows you to be in a superenv. When Homebrew builds things from source (which the CI does to create the bottles), it uses the superenv by default.

[ydnar]

Looks like Homebrew builds Go with superenv:

❯ go env PKG_CONFIG
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config

[Bo98]

Yes, using the superenv while building is correct. I'm just wondering why Go bakes the _full_ pkg-config path into its binaries and how to avoid that. I'll have a look around.

[ydnar]

Yes, using the superenv while building is correct. I'm just wondering why Go bakes the _full_ pkg-config path into its binaries and how to avoid that. I'll have a look around.

I verified that setting PKG_CONFIG at runtime to the correct path allows Go to correctly build cgo packages:

PKG_CONFIG=$(which pkg-config) go test ./...

I think this line is the culprit: https://github.com/Homebrew/brew/pull/7277/files#diff-b7261a4b3eaff4e69f0871ecfd15ca54R115

[Bo98]

I'll try removing that line and generating a new bottle. Hopefully other build systems are able to find the correct pkg-config on their own.

[ydnar]

Thanks. Might be a good idea to upstream this fix into Go, so it doesn’t bake the PKG_CONFIG variable into its binary.

https://github.com/golang/go/blob/d5e1b7ca68e2cc484d9a1517bdc0a9862936a1eb/src/cmd/dist/build.go#L216

Update: filed

[gwynne]

cmake is getting tripped up too:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.15.4
BuildVersion:   19E287
$ xcodebuild -version
Xcode 11.4
Build version 11E146
$ brew info cmake
cmake: stable 3.17.0 (bottled), HEAD
Cross-platform make
https://www.cmake.org/
/usr/local/Cellar/cmake/3.17.0_1 (6,156 files, 58.1MB) *
  Poured from bottle on 2020-03-28 at 11:15:26
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/cmake.rb
<snip>
$ cmake --find-package -DNAME=LibXml2 -DCOMPILER_ID=Clang -DLANGUAGE=C -DMODE=COMPILE
-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/libxml2
-I/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/libxml2

Obviously the latter directory doesn't exist - even if I had the CLT installed (which I make a point of not doing), they'd be for 10.15, not 10.14. CMake is intolerant of things specifying nonexistent directories and errors out during configuration.

Manually removing or renaming /usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/*/libxml2.0.pc gets rid of the extra include path and everything works again.

[Bo98]

@ydnar I'll have a look. I think other things get baked in as Go installations are not currently relocatable (which sucks for anyone not using /usr/local) so I'll try go through more of them.

@gwynne Did you upgrade to 10.15 recently? The Catalina bottles should not have 10.14 at all.

pkg-config also previously returned /usr/include/libxml2 which is also non-existent. Why did CMake not complain then?

[gwynne]

@Bo98 Oh it did complain before, I'd just added quick one-liner function to my .bash_profile that did for p in /usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/*/*.pc; do mv $p $p-DISABLE; done and put up with it on the occasions when I had to invoke it again. And no; I updated to Catalina a long, long time ago, and my cmake bottle was poured much more recently.

[gwynne]

@Bo98 Quick update - I just ran brew upgrade and now it's showing the 10.15 CLT path, which is admittedly slightly more correct in general but still fails out since I neither have nor want the (deprecated!) CLT.

[Bo98]

Given the CLT is the _only_ way to get the correct system headers on Mojave without installing an older Xcode (Mojave system headers were removed in Xcode 11), there's a long long way to go before it's phased out (which I haven't heard any concrete plans of that happening). I would not be surprised if the same situation applies to Catalina when Xcode 12 is out.

Ideally pkg-config would detect whether you have the CLT or Xcode installed but pkg-config is just incapable of doing that. It is designed to work with hardcoded paths only. pkg-config is designed for the Linux eco-system where system headers don't move all over the place like is currently the case on macOS after the removal of /usr/include.

I'm still not entirely sure how CMake breaks if you have an invalid include path. This isn't exactly a rare thing to happen under certain systems. Clang doesn't abort if a -I directory doesn't exist. It's probably also a CMake bug it returns two paths, as it does that even if they match:

$ cmake --find-package -DNAME=LibXml2 -DCOMPILER_ID=Clang -DLANGUAGE=C -DMODE=COMPILE
-I/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/libxml2 -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/libxml2

[MikeMcQuaid]

At this point I'm getting increasingly tempted to rip off the band-aid (before the wound has healed) and stop providing these pkg-config files altogether. This is an utter mess that we're basically unable to adequately fix due to the Xcode/CLT and we have way too many parts of the macOS ecosystem relying on undocumented Homebrew internals.

[gwynne]

It breaks because imported targets explicitly validate whether the paths they include make sense:

localhost:Desktop$ mkdir test && cd test
localhost:test$ touch test.c
localhost:test$ cat >CMakeLists.txt <<END
CMAKE_MINIMUM_REQUIRED( VERSION 3.17.0 )
PROJECT( test LANGUAGES C )
FIND_PACKAGE( LibXml2 REQUIRED )
ADD_EXECUTABLE( test test.c )
TARGET_LINK_LIBRARIES( test LibXml2::LibXml2 )
END
localhost:test$ cmake .
-- The C compiler identification is AppleClang 11.0.3.11030032
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LibXml2: /usr/lib/libxml2.dylib (found version "2.9.4")
-- Configuring done
CMake Error in CMakeLists.txt:
  Imported target "LibXml2::LibXml2" includes non-existent path

    "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/libxml2"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.



CMake Error in CMakeLists.txt:
  Imported target "LibXml2::LibXml2" includes non-existent path

    "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/libxml2"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.



CMake Error in CMakeLists.txt:
  Imported target "LibXml2::LibXml2" includes non-existent path

    "/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/libxml2"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.



-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.
localhost:test$

[gwynne]

@MikeMcQuaid I would be 100% behind just ditching the png-config files. It nags at me especially given I _have_ the libxml2 Homebrew formula but can't leverage that one's perfectly good libxml2.0.pc without having to manually specify PKG_CONFIG_PATH or otherwise give cmake extra my-machine-specific hints because it's keg-only (not that that's not for good reasons too...)

[MikeMcQuaid]

@gwynne With respect, we've talked about your views on the CLT in this thread (and in Slack) at length and we're now at the point where we're talking about your personal configuration. I think it's best at this point for you (or anyone else) to provide PRs for debate rather than issue comments, thanks.

[Bo98]

The SwiftPM situation should hopefully be sorted now. Please run brew upgrade pkg-config.

[0xTim]

Thanks for the quick turnaround @Bo98 and @MikeMcQuaid !

[elv-peter]

Hi folks, it looks like this also broke building Go packages that depend on pkg-config for CGO dependencies.

Some sample output from a Go project:

❯ go test ./...
# /usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config --cflags  -- opus opus opus opusfile opus opusfile
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: /pkg-config/bin/pkg-config: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: exec: /pkg-config/bin/pkg-config: cannot execute: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: exit status 126

Looks like HOMEBREW_OPT isn’t set. I’d love to help—how best to?

I don't know what is going on, but export HOMEBREW_OPT=/usr/local/opt in ~/.bash_profile worked for me

[Bo98]

Run brew upgrade go. I've just pushed new Go bottles.

[poenik]

still not working for me

[Bo98]

I'll need more information @poenik. With the latest pkg-config, pkg-config --cflags zlib returns nothing which is exactly the same as it was last week.

[poenik]

@Bo98, I have run the brew upgrade pkg-configcommand, it updated succesfully, but the errors still showed up when building. Ultimately, they were solved after following a suggestion from Vapor forum, to delete the /Library/Developer/CommandLineTools folder, as working in Xcode already, it was not needed. This solved the redefinition of modules issue.

[Bo98]

Looks like there's a significant issue with the way SwiftPM handles pkg-config. It incorrectly handles *.pc files directly so that PKG_CONFIG_SYSTEM_INCLUDE_PATH (be it the environment variable or the baked in default) is not appropriately applied. It attempts to mimic what pkg-config does but this implementation is incomplete: https://github.com/apple/swift-package-manager/blob/d433df8bfca5e3df498b3300c2b6aee222154cd4/Sources/PackageLoading/Target%2BPkgConfig.swift#L244

The SwiftPM issue has fatal effects on macOS due to systems headers no longer being at /usr/include since Mojave (SwiftPM only filters /usr/include still).

This isn't however something macOS specific. There are multiarch Linux distros that rely on this. There's also other variables commonly used in places on Linux like PKG_CONFIG_ALLOW_SYSTEM_LIBS. SwiftPM fails to recognise any of this.

There is a general issue covering the topic at https://bugs.swift.org/browse/SR-6317, filed internally as rdar://problem/40235197.

I'll look into deleting cflags entirely until the Swift issue is fixed.

[Bo98]

I think all issues mentioned here will be fixed in Homebrew 2.2.13.

[0xTim]

Thanks for all the work @Bo98 🎉

[pgrenaud]

I'm sorry to be the one that brings bad news, but I think I have an issue related to this. Since yesterday, I'm unable to build the imagick PECL extension (for PHP) that I used to be able to build without any issue. It's still unclear to me if the macOS update (or supplemental update), this Homebrew update, PHP, PECL or imagick is this cause of this issue. But everything seems to point to pkg-config.

I use a script to provision and keep up-to-date our macOS development environment and I always run it periodically in macOS VMs to be sure that nothing's broken. Here's what I got this morning when the script ran yes '' | pecl install --force imagick for PHP 7.4:

downloading imagick-3.4.4.tgz ...
Starting to download imagick-3.4.4.tgz (253,434 bytes)
.....................................................done: 253,434 bytes
19 source files, building
running: phpize
Configuring for:
PHP Api Version:         20190902
Zend Module Api No:      20190902
Zend Extension Api No:   320190902
Please provide the prefix of ImageMagick installation [autodetect] : 
Warning: Use of undefined constant name - assumed 'name' (this will throw an Error in a future version of PHP) in Builder.php on line 405
PHP Warning:  Use of undefined constant name - assumed 'name' (this will throw an Error in a future version of PHP) in /usr/local/Cellar/php/7.4.4/share/php/pear/PEAR/Builder.php on line 405

Warning: Use of undefined constant name - assumed 'name' (this will throw an Error in a future version of PHP) in /usr/local/Cellar/php/7.4.4/share/php/pear/PEAR/Builder.php on line 405
building in /private/tmp/pear/temp/pear-build-exolnetnl3Foi/imagick-3.4.4
running: /private/tmp/pear/temp/imagick/configure --with-php-config=/usr/local/opt/php/bin/php-config --with-imagick
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for a sed that does not truncate output... /usr/bin/sed
checking for pkg-config... /usr/local/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking how to run the C preprocessor... cc -E
checking for icc... no
checking for suncc... no
checking for system library directory... lib
checking if compiler supports -R... no
checking if compiler supports -Wl,-rpath,... yes
checking build system type... x86_64-apple-darwin19.4.0
checking host system type... x86_64-apple-darwin19.4.0
checking target system type... x86_64-apple-darwin19.4.0
checking for PHP prefix... /usr/local/Cellar/php/7.4.4
checking for PHP includes... -I/usr/local/Cellar/php/7.4.4/include/php -I/usr/local/Cellar/php/7.4.4/include/php/main -I/usr/local/Cellar/php/7.4.4/include/php/TSRM -I/usr/local/Cellar/php/7.4.4/include/php/Zend -I/usr/local/Cellar/php/7.4.4/include/php/ext -I/usr/local/Cellar/php/7.4.4/include/php/ext/date/lib
checking for PHP extension directory... /usr/local/Cellar/php/7.4.4/pecl/20190902
checking for PHP installed headers prefix... /usr/local/Cellar/php/7.4.4/include/php
checking if debug is enabled... no
checking if zts is enabled... no
checking for gawk... no
checking for nawk... no
checking for awk... awk
checking if awk is broken... no
checking whether to enable the imagick extension... yes, shared
checking for pkg-config... /usr/local/bin/pkg-config
checking ImageMagick MagickWand API configuration program... checking Testing /usr/local/bin/MagickWand-config... It exists
found in /usr/local/bin/MagickWand-config
checking if ImageMagick version is at least 6.2.4... found version 7.0.10 Q16 HDRI
checking for MagickWand.h or magick-wand.h header... /usr/local/Cellar/imagemagick/7.0.10-6/include/ImageMagick-7/MagickWand/MagickWand.h
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: /pkg-config/bin/pkg-config: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: exec: /pkg-config/bin/pkg-config: cannot execute: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: /pkg-config/bin/pkg-config: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: exec: /pkg-config/bin/pkg-config: cannot execute: No such file or directory
checking PHP version is at least 5.3.2... yes. found 7.4.4
libs



checking for MagickGetVersion... no
checking for __MagickGetVersion... no
checking for a sed that does not truncate output... /usr/bin/sed
checking for ld used by cc... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking for /Library/Developer/CommandLineTools/usr/bin/ld option to reload object files... -r
checking for BSD-compatible nm... /usr/bin/nm -B
checking whether ln -s works... yes
checking how to recognize dependent libraries... pass_all
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking the maximum length of command line arguments... 196608
checking command to parse /usr/bin/nm -B output from cc object... ok
checking for objdir... .libs
checking for ar... ar
checking for ranlib... ranlib
checking for strip... strip
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking if cc supports -fno-rtti -fno-exceptions... yes
checking for cc option to produce PIC... -fno-common
checking if cc PIC flag -fno-common works... yes
checking if cc static flag -static works... no
checking if cc supports -c -o file.o... yes
checking whether the cc linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin19.4.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no

creating libtool
appending configuration tag "CXX" to libtool
configure: patching config.h.in
configure: creating ./config.status
config.status: creating config.h
running: make
/bin/sh /private/tmp/pear/temp/pear-build-exolnetnl3Foi/imagick-3.4.4/libtool --mode=compile cc  -I. -I/private/tmp/pear/temp/imagick -DPHP_ATOM_INC -I/private/tmp/pear/temp/pear-build-exolnetnl3Foi/imagick-3.4.4/include -I/private/tmp/pear/temp/pear-build-exolnetnl3Foi/imagick-3.4.4/main -I/private/tmp/pear/temp/imagick -I/usr/local/Cellar/php/7.4.4/include/php -I/usr/local/Cellar/php/7.4.4/include/php/main -I/usr/local/Cellar/php/7.4.4/include/php/TSRM -I/usr/local/Cellar/php/7.4.4/include/php/Zend -I/usr/local/Cellar/php/7.4.4/include/php/ext -I/usr/local/Cellar/php/7.4.4/include/php/ext/date/lib  -DHAVE_CONFIG_H  -g -O2   -c /private/tmp/pear/temp/imagick/imagick_file.c -o imagick_file.lo
mkdir .libs
 cc -I. -I/private/tmp/pear/temp/imagick -DPHP_ATOM_INC -I/private/tmp/pear/temp/pear-build-exolnetnl3Foi/imagick-3.4.4/include -I/private/tmp/pear/temp/pear-build-exolnetnl3Foi/imagick-3.4.4/main -I/private/tmp/pear/temp/imagick -I/usr/local/Cellar/php/7.4.4/include/php -I/usr/local/Cellar/php/7.4.4/include/php/main -I/usr/local/Cellar/php/7.4.4/include/php/TSRM -I/usr/local/Cellar/php/7.4.4/include/php/Zend -I/usr/local/Cellar/php/7.4.4/include/php/ext -I/usr/local/Cellar/php/7.4.4/include/php/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /private/tmp/pear/temp/imagick/imagick_file.c  -fno-common -DPIC -o .libs/imagick_file.o
In file included from /private/tmp/pear/temp/imagick/imagick_file.c:22:
In file included from /private/tmp/pear/temp/imagick/php_imagick_file.h:24:
/private/tmp/pear/temp/imagick/php_imagick_defs.h:25:12: fatal error: 'MagickWand/MagickWand.h' file not found
#  include <MagickWand/MagickWand.h>
           ^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [imagick_file.lo] Error 1
ERROR: `make' failed

Of course, I though at first that this was an issue with imagick. But reinstalling it does not fix the issue. I have the same issue with PHP 7.2, 7.3 and 7.4, all installed with Homebrew. So far, I've reinstalled [email protected], [email protected], php, imagemagick and pkg-config. Homebrew (2.2.13-28-ga348a14) and macOS Catalina (10.15.4) are up-to-date.

Then, I found this blog post linked from this stackoverflow answer. Following the advice, I ran yes '' | C_INCLUDE_PATH=/usr/local/Cellar/imagemagick/7.0.10-6/include/ImageMagick-7 pecl install --force imagick to try installing the imagick extension again, and it worked!

My knowledge is limited, but considering the line /usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: /pkg-config/bin/pkg-config: No such file or directory from the building output above, and considering that I'm able to build imagick if I specify the include path, I assume that the issue is related to pkg-config, right?

[Bo98]

That's a ImageMagick-specific problem. Their build scripts bake in several paths based on the build environment which is not the same as the runtime environment. I'll see if there's a way to workaround that.

[mkalvas]

I'm having the same issue as @pgrenaud with the rmagick gem (which builds native extrensions with imagemagick) in a ruby on rails env. i.e.

/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: /pkg-config/bin/pkg-config: No such file or directory

Which results in rmagick failing out as Can't find the ImageMagick library or one of the dependent libraries. even though they are all there and installed correctly. Also, notably, the workaround for building through adding the C_INCLUDE_PATH and/or PKG_CONFIG_PATH doesn't work (seemingly only on this version of homebrew?).

During my investigation, I've found that however the .../super/pkg-config file is being run $HOMEBREW_OPT isn't available which is why we're getting /pkg-config as the root of the executable path. Adding HOMEBREW_OPT to the command gets it working.

e.g. the working build command

C_INCLUDE_PATH=/usr/local/Cellar/imagemagick@6/6.9.11-6/include/ImageMagick-6 PKG_CONFIG_PATH=/usr/local/Cellar/imagemagick@6/6.9.11-6/lib/pkgconfig HOMEBREW_OPT=/usr/local/opt gem install rmagick

I just wanted to add my experience and info in case it was helpful for other people here or googling in the future.

Edit: Also, I have all of these exported and everything in my .zshrc, but only including them for the command is working.

[Bo98]

@pgrenaud @mkalvas Run brew upgade imagemagick/brew upgrade imagemagick@6 and try again. Should work now.

[pgrenaud]

Wow, thanks for the quick fix @Bo98 ! 👏 I confirm that everything is working as expected on my side! 👌

[MikeMcQuaid]

This long-running issue was (finally) fixed by the incredible @Bo98. Thanks for all your work @Bo98 👏

[artob]

I think all issues mentioned here will be fixed in Homebrew 2.2.13.

@Bo98 I'm afraid the Go (specifically, cgo) issue previously encountered also by @ydnar @elv-peter and @poenik still persists with the latest.

If I've understood this long thread correctly, the problem arises because Go hardcodes the pkg-config shim path into its binaries, that shim needs HOMEBREW_OPT set, and Go upstream told us there's nothing to fix on their end?

In any case, here's a minimal reproduction of the problem with Homebrew 2.2.14, pkg-config 0.29.2_3, and go 1.14.2_1 on macOS 10.14:

$ cat main.go
package main

// #cgo pkg-config: zlib
// extern const char* zlibVersion();
import "C"
import "fmt"

func main() {
    fmt.Println(C.GoString(C.zlibVersion()))
}

Build failure:

$ go run main.go
# /usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config --cflags  -- zlib
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: /pkg-config/bin/pkg-config: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: line 6: exec: /pkg-config/bin/pkg-config: cannot execute: No such file or directory
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/pkg-config: exit status 126

Build workaround (kudos to @ydnar):

$ PKG_CONFIG=$(which pkg-config) go run main.go
1.2.11

(Alternative workaround: set HOMEBREW_OPT as demonstrated in a couple of previous replies.)

[Bo98]

I can't reproduce:

$ go run main.go
1.2.11

If you build Go from source rather than install from a bottle, make sure it was done so recently (and brew reinstall go if not).

[artob]

@Bo98 Thanks. I haven't _purposely_ built Go from source, but you are correct, it does look like it was built from source:

$ brew info go
go: stable 1.14.2, HEAD
Open source programming language to build simple/reliable/efficient software
https://golang.org
/usr/local/Homebrew/Cellar/go/1.14.2_1 (9,440 files, 424.4MB) *
  Built from source on 2020-04-11 at 13:46:44

brew reinstall go just wants to rebuild it. Probably would be preferable to use the bottle; I'll look later to see how I can force the bottle installation.

[artob]

I can't reproduce:

$ go run main.go
1.2.11

If you build Go from source rather than install from a bottle, make sure it was done so recently (and brew reinstall go if not).

@Bo98 I confirm that brew reinstall go did solve the issue, and that the problem is no longer reproducible on my laptop.