Homebrew-core: upgrade to Qt 5.10 fails to build on 10.12.6

Created on 17 Dec 2017 · 21Comments · Source: Homebrew/homebrew-core

Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

Please always follow these steps:

[X] Confirmed this is a problem with brew installing one, specific Homebrew/homebrew-core formula (not cask or tap) and not every time you run brew? If it's a general brew problem please file this issue at https://github.com/Homebrew/brew/issues/new. If it's a brew cask problem please file this issue at https://github.com/caskroom/homebrew-cask/issues/new. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
[X] Ran brew update and retried your prior step?
[X] Ran brew doctor, fixed all issues and retried your prior step?
[X] Ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link?
Error: Your HOMEBREW_GITHUB_API_TOKEN does not have sufficient scope!
[X] If brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

brew config

HOMEBREW_VERSION: 1.4.0-33-gb4d43e9
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: b4d43e950fd45c24e48d6ebfd3013357abcd21a9
Last commit: 11 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 46a295ec4c93151863775c766660708201d418bf
Core tap last commit: 3 hours ago
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_REPOSITORY: /opt/homebrew
HOMEBREW_CELLAR: /opt/homebrew/Cellar
HOMEBREW_CELLAR: /opt/homebrew/Cellar
HOMEBREW_DEV_CMD_RUN: 1
HOMEBREW_GITHUB_API_TOKEN: set
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_REPOSITORY: /opt/homebrew
CPU: octa-core 64-bit haswell
Homebrew Ruby: 2.3.3 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3/bin/ruby
Clang: 9.0 build 900
Git: 2.13.1 => /opt/homebrew/bin/git
Curl: 7.54.0 => /usr/bin/curl
Perl: /usr/bin/perl
Python: /usr/bin/python
Ruby: /usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
Java: 1.8.0_144, 1.8.0_131, 1.8.0_91, 1.8.0_72
macOS: 10.12.6-x86_64
Xcode: 9.2
CLT: 9.0.1.0.1.1506734476
X11: 2.7.11 => /opt/X11

brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: A newer Command Line Tools release is available.
Update them from Software Update in the App Store.


Warning: Putting non-prefixed coreutils in your path can cause gmp builds to fail.

Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected dylibs:
  /usr/local/lib/libprotobuf-lite.dylib
  /usr/local/lib/libprotobuf.dylib
  /usr/local/lib/libprotoc.dylib

Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected static libraries:
  /usr/local/lib/libprotobuf-lite.a
  /usr/local/lib/libprotobuf.a
  /usr/local/lib/libprotoc.a

Warning: You have unlinked kegs in your Cellar
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  [email protected]
  python

Warning: Your Homebrew's prefix is not /usr/local.
You can install Homebrew anywhere you want but some bottles (binary packages)
can only be used with a /usr/local prefix and some formulae (packages)
may not build correctly with a non-/usr/local prefix.

To help us debug your issue please explain:

What you were trying to do (and why)
brew upgrade qt # currently 5.6 is installed

What happened (include command output)

./configure -verbose -prefix /opt/homebrew/Cellar/qt/5.10.0 -release -opensource -confirm-license -system-zlib -qt-libpng -qt-libjpeg -qt-freetype -qt-pcre -nomake tests -no-rpath -pkg-config -dbus-runtime -nomake examples
==> make
Last 15 lines from /Users/vlovich/Library/Logs/Homebrew/qt/02.make:
  ^--------------------------------------
See //base/BUILD.gn:28:1: whence it was imported.
import("//build/config/sysroot.gni")
^----------------------------------
See //BUILD.gn:73:5: which caused the file to be included.
    "//base:base_unittests",
    ^----------------------
Command '['/private/tmp/qt-20171216-8466-116edvh/qt-everywhere-src-5.10.0/qtwebengine/src/3rdparty/chromium/tools/gn/out/Release/gn', 'gen', '/private/tmp/qt-20171216-8466-116edvh/qt-everywhere-src-5.10.0/qtwebengine/src/3rdparty/chromium/tools/gn/out/Release', '--args=use_qt=true is_component_build=false is_shared=true enable_nacl=false enable_remoting=false enable_web_speech=false use_allocator_shim=false use_allocator="none" v8_use_external_startup_data=false treat_warnings_as_errors=false enable_swiftshader=false use_custom_libcxx=false enable_basic_printing=true enable_print_preview=true enable_pdf=true enable_plugins=true enable_widevine=true enable_spellcheck=true enable_webrtc=true symbol_level=0 remove_webcore_debug_symbols=true remove_v8base_debug_symbols=true is_clang=true use_sysroot=false use_kerberos=false clang_base_path="/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr" clang_use_chrome_plugins=false mac_deployment_target="10.10" mac_sdk_min="10.13" toolkit_views=false use_external_popup_menu=false use_browser_spellchecker=false is_debug=false', '--script-executable=/usr/bin/python']' returned non-zero exit status 1
Building gn manually in a temporary directory for bootstrapping...
Building gn using itself to out/Release...
Project ERROR: GN build error!
make[3]: *** [sub-gn-pro-make_first] Error 3
make[2]: *** [sub-buildtools-make_first] Error 2
make[1]: *** [sub-src-make_first] Error 2
make: *** [module-qtwebengine-make_first] Error 2

02.make.log

What you expected to happen
Qt should build & install fine.
Step-by-step reproduction instructions (by running brew install commands)

Formula additions or changes

To get formulae added or changed in Homebrew please file a Pull Request
We will close issues requesting formulae changes.

Source

vlovich

Most helpful comment

This bug would only impact mac machines for Google employees.

@vlovich It seems to me that this is something Google should resolve internally, sorry. Once a patch has been accepted and merged upstream we'll more strongly consider adding it to the formula. Alternatively, we may consider accepting this patch if Google is willing to make a donation to Homebrew's Patreon (or one of the other donations methods).

MikeMcQuaid on 18 Dec 2017

👍2

All 21 comments

Not sure if the "A newer Command Line Tools release is available. Update them from Software Update in the App Store." warning is relevant, but the App Store doesn't have any updates.

vlovich on 17 Dec 2017

What's the output of

xcode-select -p
ls /Applications/ | grep -i x

ilovezfs on 17 Dec 2017

> xcode-select -p
/Applications/Xcode.app/Contents/Developer
> ls /Applications/ | grep -i x
# snip
Xcode.app
# snip
> xcrun --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk

vlovich on 17 Dec 2017

The part of the log that was interesting to me was the invocation of find_sdk.py that resulted in:

No such file or directory: '/private/tmp/qt-20171216-8466-116edvh/qt-everywhere-src-5.10.0/qtwebengine/src/3rdparty/chromium/build/mac_files/Xcode.app/Platforms/MacOSX.platform/Developer/SDKs

Very unclear how it built up that seemingly invalid path (unless chromium's build system creates a symlink in the temp directory?).

This was the invocation by find_sdk.py:

/usr/bin/python -- /private/tmp/qt-20171216-8466-116edvh/qt-everywhere-src-5.10.0/qtwebengine/src/3rdparty/chromium/build/mac/find_sdk.py --print_sdk_path --developer_dir /private/tmp/qt-20171216-8466-116edvh/qt-everywhere-src-5.10.0/qtwebengine/src/3rdparty/chromium/build/mac_files/Xcode.app 10.13

vlovich on 17 Dec 2017

Are you able to switch to /usr/local as your prefix?

ilovezfs on 17 Dec 2017

No. It's a company laptop & /usr/local permissions are managed by puppet & don't play well with homebrew.

vlovich on 17 Dec 2017

Is there anything unusual about your temporary directory?

ilovezfs on 17 Dec 2017

Warning: Putting non-prefixed coreutils in your path can cause gmp builds to fail.

That is also a potential culprit.

ilovezfs on 17 Dec 2017

No such file or directory: '/private/tmp/qt-20171216-8466-116edvh/qt-everywhere-src-5.10.0/qtwebengine/src/3rdparty/chromium/build/mac_files/Xcode.app/Platforms/MacOSX.platform/Developer/SDKs

That's even stranger than it looks at first because the path is

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/

not

/Applications/Xcode.app/Platforms/MacOSX.platform/Developer/SDKs/

I'm wondering if there's something broken with your Xcode.app

ilovezfs on 17 Dec 2017

Nothing unusual about my tmp dir AFAIK. I'll try working around the Coreutils path issue. Yeah, the path chromium build generates is clearly wrong and I don't know why.

vlovich on 17 Dec 2017

After removing coreutils from PATH the compile is still broken in the same way.

vlovich on 17 Dec 2017

Something to do with running this on a Google laptop. Chromium thinks it needs to use a hermetic Xcode installation. Not sure if the bug is in homebrew or in chromium upstream.

vlovich on 17 Dec 2017

Interesting. It's strange that it doesn't occur on our CI if it's specific to Homebrew.

ilovezfs on 17 Dec 2017

Reported upstream to chromium too in case it's an upstream issue (https://bugs.chromium.org/p/chromium/issues/detail?id=795615). The simplest solution is to hack should_use_hermetic_xcode.py to always return 0 but I'm not sure how to generate such a patch & integrate it into the recipe. This bug would only impact mac machines for Google employees.

vlovich on 17 Dec 2017

Are you able to move the directory /Library/GoogleCorpSupport/ ?

ilovezfs on 17 Dec 2017

Not really

vlovich on 17 Dec 2017

    inreplace "qtwebengine/src/3rdparty/chromium/build/mac/should_use_hermetic_xcode.py",
      "/Library/GoogleCorpSupport/", "/Library/BingCorpSupport/"

maybe?

ilovezfs on 17 Dec 2017

i.e.

iMac-TMP:homebrew-core joe$ git diff -- Formula/qt.rb 
diff --git a/Formula/qt.rb b/Formula/qt.rb
index e62050d04a..1d0f057302 100644
--- a/Formula/qt.rb
+++ b/Formula/qt.rb
@@ -55,6 +55,9 @@ class Qt < Formula
   end

   def install
+    inreplace "qtwebengine/src/3rdparty/chromium/build/mac/should_use_hermetic_xcode.py",
+      "/Library/GoogleCorpSupport/", "/Library/BingCorpSupport/"
+
     args = %W[
       -verbose
       -prefix #{prefix}

ilovezfs on 17 Dec 2017

Adding this to the recipe worked:

  patch :p1, <<~EOF
diff --git a/qtwebengine/src/3rdparty/chromium/build/mac/should_use_hermetic_xcode.py b/qtwebengine/src/3rdparty/chromium/build/mac/should_use_hermetic_xcode.py
index dc1b930..37b19f6 100755
--- a/qtwebengine/src/3rdparty/chromium/build/mac/should_use_hermetic_xcode.py
+++ b/qtwebengine/src/3rdparty/chromium/build/mac/should_use_hermetic_xcode.py
@@ -32,7 +32,7 @@ def main():
   if os.environ.get('FORCE_MAC_TOOLCHAIN') or allow_corp:
     if not mac_toolchain.PlatformMeetsHermeticXcodeRequirements(sys.argv[1]):
       return "2"
-    return "1"
+    return "0"
   else:
     return "0"
EOF

vlovich on 18 Dec 2017

🎉1

This bug would only impact mac machines for Google employees.

MikeMcQuaid on 18 Dec 2017

👍2

I made a Gist of @vlovich's patch. brew edit qt and drop this in the patch section.

  patch do
    url "https://gist.githubusercontent.com/rryan/dc4bf55663b7605994d71807c893a3a0/raw/2564489bd38df682c8db779cc2e4ff122b618d58/dont_use_hermetic_xcode.patch"
    sha256 "1921dacb812721215c4e1bbfec9183f77bdd5dfdead4c4c7aec94170e51db36d"
  end