Homebrew-core: pypy crashes with dyld: lazy symbol binding failed: Symbol not found: _clock_gettime

[irmen]

Since the formula for pypy installed pypy 5.6.0, it now crashes with the following error:

phobos:~ irmen$ pypy
Python 2.7.12 (aff251e543859ce4508159dd9f1a82a2f553de00, Nov 13 2016, 01:57:35)
[PyPy 5.6.0 with GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import time
>>>> time.clock()
dyld: lazy symbol binding failed: Symbol not found: _clock_gettime
  Referenced from: /usr/local/Cellar/pypy/5.6.0/libexec/lib/libpypy-c.dylib
  Expected in: flat namespace

dyld: Symbol not found: _clock_gettime
  Referenced from: /usr/local/Cellar/pypy/5.6.0/libexec/lib/libpypy-c.dylib
  Expected in: flat namespace

Trace/BPT trap: 5
phobos:~ irmen$ 

(edit: the above simple test no longer fails, but the problem can still be triggered with a somewhat different script. See the comment below; https://github.com/Homebrew/homebrew-core/issues/6949#issuecomment-264299095 )

I am running on OSX 10.11.6 El Capitan with command line tools 7.3 installed. I think this is related to the fact that pypy is built with tools 8.0 on OSX Sierra? I've googled and found a fair number of other projects also having issues like this.

Relevant links:
Bugreport at the pypy project itself (because I'm not sure if it is a Homebrew problem or something Pypy itself should fix?)
brew gist-logs pypy
And this is the pypy bottle that brew installed on my machine: "Downloading https://homebrew.bintray.com/bottles/pypy-5.6.0.el_capitan.bottle.tar.gz"

[tdsmith]

FWIW, the El Capitan bottle is built on El Capitan.

[mistydemeo]

I think this is related to the fact that pypy is built with tools 8.0 on OSX Sierra?

Yes, the problem is that it's built on 10.11 but using Xcode 8 with the 10.12 SDK. The 10.12 SDK includes the symbol in its libSystem, while the headers guard it so that it won't be declared with a 10.11 or older deployment target. Like you've seen here, that often isn't enough, and a binary is built for 10.11 that won't run on 10.11.

[MikeMcQuaid]

@mistydemeo @tdsmith This likely needs an upstream patch and a better test do that detects this at runtime.

[mistydemeo]

pypy3 ran into the same issue: https://github.com/Homebrew/homebrew-core/pull/6769 I wondered "Why is this working in pypy and not pypy3 when they have the same patch?" but the answer apparently is that it wasn't. 😭

pypy3 hits this in postinstall when using the built bottle; I'm surprised that this wasn't caught in CI for pypy as well.

[MikeMcQuaid]

Should maybe add the time.clock() call to both the test dos too

[mistydemeo]

Upstream simply disabled clock_gettime; we can inreplace this to True on 10.12: https://bitbucket.org/pypy/pypy/commits/91e202bbd0b983c88fa9c33b9215b0f910d1f405

[irmen]

Hi, I've reinstalled pypy and got a new build that did improve things, but the problem remains in some other places, unfortunately.

It's harder to reproduce now because it no longer occurs always, but sometimes. When running the test script below repeatedly using pypy, it will eventually show the same dyld crash again. There's some race condition going on?

Code to reproduce:

import threading

def mythread():
    print("@THREAD!")

print("@@START")
w=threading.Thread(target=mythread)
w.start()
w.join()
print("@@EXIT")

# run in bash in loop because crash is only now and then:
# while true; do pypy clockcrash.py ; done

pypy version (still running on OS X 10.11.6 el Capitan):
Python 2.7.12 (aff251e543859ce4508159dd9f1a82a2f553de00, Nov 25 2016, 00:01:55)
[PyPy 5.6.0 with GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin

I'll report this over at the pypy bug tracker as well.

edit: the upstream fix is not in the 5.6.0 branch, I had to get the nightly build from the default branch (5.7.0) to get the fix (and yes, it seems solved in that version, the script above runs without issues here). Can it be that there are more upstream patches that need to be applied, than just that single one that were done to solve the problem?

[mistydemeo]

Possibly! I'll have to dig through, and also ask them in IRC.

[MikeMcQuaid]

@mistydemeo any news?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[ilovezfs]

@irmen this seems to be fixed in pypy 5.7.1 so I'm closing this. Please let us know if it's still a problem after updating and upgrading to 5.7.1.

[irmen]

@ilovezfs well, it isn't fixed in the homebrew version of pypy. The small piece of code I posted above still crashes in the same way when ran with homebrew's pypy 5.7.1:

% pypy --version
Python 2.7.13 (1aa2d8e03cdfab54b7121e93fda7e98ea88a30bf, Apr 03 2017, 21:27:28)
[PyPy 5.7.1 with GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]

% pypy pypyclockbug.py
@@START
dyld: lazy symbol binding failed: Symbol not found: _clock_gettime
  Referenced from: /usr/local/Cellar/pypy/5.7.1/libexec/lib/libpypy-c.dylib
  Expected in: flat namespace

@THREAD!
dyld: Symbol not found: _clock_gettime
  Referenced from: /usr/local/Cellar/pypy/5.7.1/libexec/lib/libpypy-c.dylib
  Expected in: flat namespace

[1]    1366 trace trap  pypy pypyclockbug.py

Running this on the pypy build produced by pypy.org, it works without issues. For reference, this is the version and build information from that one:

% ./pypy2-v5.7.1-osx64/bin/pypy
Python 2.7.13 (1aa2d8e03cdf, Mar 31 2017, 10:20:53)
[PyPy 5.7.1 with GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin

[ilovezfs]

I was testing time.clock() from the original post but I can see this still happens with the script from https://github.com/Homebrew/homebrew-core/issues/6949#issuecomment-264299095

[ilovezfs]

As a temporary workaround, you can do this:

iMac-TMP:homebrew-core joe$ git diff -- Formula/pypy.rb 
diff --git a/Formula/pypy.rb b/Formula/pypy.rb
index 024e715..7d764c4 100644
--- a/Formula/pypy.rb
+++ b/Formula/pypy.rb
@@ -11,8 +11,8 @@ class Pypy < Formula
   bottle do
     cellar :any
     sha256 "b75849c0ee2e1cf5b412a05e9375964b29034bef926dbc416621c33f0964adbe" => :sierra
-    sha256 "e72125aaebda4ae9d4d6c3a68cb72f2b9ad9c7213bff78c9c1e0f8dabccba30f" => :el_capitan
-    sha256 "4f1d2f306dcb117825b5240ef3f4fe2b0e09266ef3df70e6971bda56da0a5e4f" => :yosemite
+    # sha256 "e72125aaebda4ae9d4d6c3a68cb72f2b9ad9c7213bff78c9c1e0f8dabccba30f" => :el_capitan
+    sha256 "4f1d2f306dcb117825b5240ef3f4fe2b0e09266ef3df70e6971bda56da0a5e4f" => :yosemite_or_later
   end

   option "without-bootstrap", "Translate Pypy with system Python instead of " \

Then brew uninstall pypy; brew install pypy

[irmen]

I'll update my original message with the link to the new script. Thanks for the work around I will try that.

[ilovezfs]

@irmen this is now fixed :)

[irmen]

That is wonderful thanks for the update