Handbrake: Issues when compiling on macOS 10.15 beta 19A512f

[HazMatt69]

I get errors while trying to download contribs:

"ERROR: download failure; : ERROR: download failure; : downloading https://download.handbrake.fr/handbrake/contrib/libogg-1.3.3.tar.gz to /Downloads/HandBrake/download/libogg-1.3.3.tar.gz
: downloading https://downloads.xiph.org/releases/ogg/libogg-1.3.3.tar.gz to /Downloads/HandBrake/download/libogg-1.3.3.tar.gz
: make[1]: * [/Downloads/HandBrake/download/libogg-1.3.3.tar.gz] Error 1
: make[1]: * Waiting for unfinished jobs....

Is it linked to this: "All TLS server certificates must comply with these new security requirements in iOS 13 and macOS 10.15"?

Thanks for your help :-)

[sr55]

My guess is either Apple broke python, or you're using a non-apple python that doesn't have it's certificate store setup.

[HazMatt69]

I'm using python 3.7.4 (installed with brew and added to path) as macOS only ships with python 2.7.16

[sr55]

Yeh, that's your problem. I'd suspect the system python works fine, but your brew version isn't configured correctly.

https://stackoverflow.com/questions/44649449/brew-installation-of-python-3-6-1-ssl-certificate-verify-failed-certificate

[HazMatt69]

I've already tried that but it still fails to get the certificate :-(

[sr55]

More a question for the brew/python community. I'm not an expert in this area I'm afraid.

[Nomis101]

FYI, I see the same issue if using Xcode 11 beta on macOS Mojave.
https://forum.handbrake.fr/viewtopic.php?f=5&t=39092
(I have not tested if the issue is fixed with the new Xcode 11 Beta 5.)

[HazMatt69]

@Nomis101 Well spotted! Issue is still on with beta 5

[HazMatt69]

Issue still on w/ Xcode 11 beta 6 (11M392q). @Nomis101 have you found a solution?

[Nomis101]

Nope, my only solution is to avoid Xcode 11. Which is more a workaround than a fix. :-(
Did you report the issue to Apple?

[bradleysepos]

Does this command work? Replace python with whatever command runs Python 3 on your system.

python -c 'from urllib.request import urlretrieve; urlretrieve("https://google.com")'

If not, then the issue is with your Python installation and Homebrew should fix it. See https://github.com/Homebrew/homebrew-core/issues/42198.

[HazMatt69]

Yeah, I suppose I'll have to wait for a fix from Homebrew...

[bradleysepos]

I'll leave this open for now with hope for a Homebrew solution. If ultimately they ignore or do not resolve the issue, we will have to close this as an upstream bug.

[bradleysepos]

@ilovezfs ^

[HazMatt69]

Ok, thanks @bradleysepos

[Nomis101]

I have the same issue, but did not install python via homebrew. I always use the official installer from the python homepage.

Did you report the issue to Apple?

Somebody did.
https://openradar.appspot.com/7111585

[bradleysepos]

Try the following:

export SSL_CERT_FILE=/etc/ssl/cert.pem
export SSL_CERT_DIR=/etc/ssl/certs
./configure && cd build && make contrib.fetch

[HazMatt69]

Unfortunately, I'm still getting the error...

[bradleysepos]

🤷‍♂

[Nomis101]

I've "fixed" this issue for me with a rather drastic workaround. I replaced the folder Xcode-beta.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7 with the official one from /Library/Frameworks/Python.framework/Versions/3.7 (installed from the official python installer), after executing Install Certificates.command.
The official one contained a directory /etc/openssl, which is missing in the Xcode one. Only moving the /etc directory to Xcode did not work.

[Nomis101]

Ok, the issue is as follow. A working installation of python 3.7 requires to install the python certificates (Install Certificates.command), which will install /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/certifi/cacert.pem which than is symlinked from /Library/Frameworks/Python.framework/Versions/3.7/etc/openssl/cert.pem

Xcode 11 now as well includes the Python.framework (Xcode 10 did not, and so the regular installed Python.framework was used from /Library), but it lacks the directory /Versions/3.7/etc/openssl, it lacks cacert.pem and the symlink. Which means, for every HTTPS request you see errors like

ERROR: download failure;

I've reported this to Apple.

[Nomis101]

I've reported this to Apple.

Apple will not fix this, because Apple does not think this is a problem. The reply from Apple is

the problem behaves as intended.

certifi is a third-party module, not part of Python itself.

urllib is a low-level library. It can handle ssl, but you must explicitly set up the SSL context with a cafile.

Try the following instead:

pip3 install requests
python -c 'import requests; print(requests.get("https://apple.com").text)'

If you only want to get cacert.pem, you can use "pip3 install certifi", but you must still explicitly pass cafile to urllib.

I don't know what a cafile is, but I will try to figure out how the comment is helpful...

[fxcoudert]

Hi, Homebrew maintainer here. If I follow correctly:

• There is an issue with the Apple-provided python3 in Catalina, which is a EWONTFIX from Apple. Probably the solution there is to install certifi as part of the Handbrake install scripts?
• There is a potential Homebrew issue (https://github.com/Homebrew/homebrew-core/issues/42198), on which we don't have enough information and we can't reproduce so far. A full bug report with all requested information would help us understand what is the problem.

[fxcoudert]

So, if you are affected by this with Homebrew python, you would need to file a first-hand report with the information requested at https://github.com/Homebrew/homebrew-core/issues/new/choose

[Nomis101]

Hi, Homebrew maintainer here. If I follow correctly:

• There is an issue with the Apple-provided python3 in Catalina, which is a EWONTFIX from Apple. Probably the solution there is to install certifi as part of the Handbrake install scripts?

It is not an issue with Catalina, it is Xcode 11 (currently in beta). There is the same issue on Mojave using Xcode 11. Xcode 11 now contains the Python.framework Xcode-beta.app/Contents/Developer/Library/Frameworks/Python3.framework (Xcode 10 and older did not). Apple will not fix this, because certifiis a third-party module and not part of Python itself.
According to Apple the solution would be to use pythons cafile (I'm not a python person, so can't give much information for that).

• There is a potential Homebrew issue (Homebrew/homebrew-core#42198), on which we don't have enough information and we can't reproduce so far. A full bug report with all requested information would help us understand what is the problem.

I don't use Homebrew at all. It is related to Xcode 11, so I don't know what Homebrew can do against it.

[fxcoudert]

It is not an issue with Catalina, it is Xcode 11

Indeed, the issue is with the Catalina SDK, present in Xcode 11.

It is related to Xcode 11, so I don't know what Homebrew can do against it.

I commented here because a comment above stated _“I'll leave this open for now with hope for a Homebrew solution.”_ and this was filed as a bug with us: https://github.com/Homebrew/homebrew-core/issues/42198
But when haven't been able to reproduce the issue with Homebrew (which does install certifi, because a bare-bones python without some basic modules is really useless).

[Nomis101]

because a bare-bones python without some basic modules is really useless).

Yes, exactly. I'm currently trying to explain this also to Apple.

[bradleysepos]

While this isn't strictly our bug, I've worked around the issue in 86e0173ec1b3c953e7a1100684caabc095e91edc by enhancing a previous check for Apple prepending its own paths to the PATH environment variable.

And updated the documentation to reflect the need to install Python 3 from python.org, since we cannot rely on Apple's distribution having certificates properly installed/referenced: https://handbrake.fr/docs/en/latest/developer/build-mac.html

[Nomis101]

Thanks a lot!