Julia: Notarize macOS app for Catalina

[staticfloat]

macOS 10.15+ is going to start requiring us to "notarize" our apps by sending them to Apple to be scanned. If we do not, we will get scary errors like the following:

"Julia-1.0" can't be opened because Apple cannot check it for malicious software.
_{This software needs to be updated. Contact the developer for more information.}

To temporarily work around this, right click on the application and select "open."

Note that this is distinct from the "cannot be verified" error when an app is not signed; notarization protects against malware by forcing developers to upload their app bundles to Apple, where Apple can scan them with some algorithm, determine there is nothing harmful in there, then pass back a cryptographic notarization. This must be done for every single build of Julia, if we want the .app to be openable. (We could conceivably not do this for nightly builds, and expect users to bypass the scary dialog, if this turns out to be a flaky process)

The workflow we'll have to adopt is:

• Build and package Julia, just like normal.

• Sign julia, but opt in to the "hardened runtime". Note that this, by default, disables things like being able to jump into writable pages, so we'll have to figure out the right set of "entitlements" to request, in order to peel back the appropriate layers of protection to get our JIT to still work. This is done by passing --options runtime to codesign.

• Submit the packaged .dmg to Apple Notary service. This should be doable by /usr/bin/xcrun altool --notarize-app --primary-bundle-id "$BUNDLE_ID" --username "$APPLE_ID_EMAIL" --password "$APPLE_ID_PASSWORD" --file "$JULIA_DMG" --output-format "xml"

• Poll Apple until the notarization is complete. We do that with something like /usr/bin/xcrun altool --notarization-info --username "$APPLE_ID_EMAIL" --password "$APPLE_ID_PASSWORD" --output-format "xml", then parse the XML. Note that this notarization can take multiple hours to complete, it is not a fast process, so I think we're going to need to run this as an asynchronous job, perhaps in parallel to testing.

• Staple notarization ticket to our app. xcrun stapler staple Julia.app. Note that this is performed on an .app, not a .dmg. We need to download the DMG, mount it as read/write (probably by using shadow mounts, like we do when needing to resign something), staple the ticket to the .app, then generate a _new_ .dmg.

References:

• https://stackoverflow.com/questions/56890749/macos-notarize-in-script

• https://developer.apple.com/documentation/security/hardened_runtime_entitlements

• https://gist.github.com/staticfloat/62369d705f265c0b35ff8888b2e1d76c

[ViralBShah]

Do we have to do this for all post 1.0 binaries?

[staticfloat]

Yes, I think we will. I'm thinking the best way to do this is to have a script that can download a premade .dmg, re-sign it with the appropriate options, submit it for notarization, then wait for it to be notarized, repackage it after stapling, and reupload.

[jeffreycunn]

This shows up when I try to run Julia in Catalina.

[KristofferC]

Right click and do open until this has been fixed.

[ViralBShah]

We probably need to note this in the platform instructions.

[ViralBShah]

The user experience for starting Julia the first time after installation on macOS 10.15 is now:

1. malicious software complaint. Click ok.
2. Unidentified developer complaint - Click open anyway in system prefs
3. Then malicious software again
4. Then the terminal access warning

The second start then seems to be ok. The main.scpt file is being modified by macOS to ignore the notarization check, which then makes it think that the application is not correctly signed.

[staticfloat]

Can I get someone to test https://julialangnightlies.s3.amazonaws.com/bin/mac/x64/1.3/julia-aee26cf525-mac64.dmg for me?

[StefanKarpinski]

You need to open up the S3 bucket.

[staticfloat]

whoops; done

[StefanKarpinski]

I get a mildly scary but not terrifying warning:

Then when I click open, the app icon bounces and I get this:

[ViralBShah]

It opened up perfectly for me.

[staticfloat]

Alright; I think I may have fixed another issue, new build: https://julialangnightlies.s3.amazonaws.com/bin/mac/x64/1.3/julia-0c659cff14-mac64.dmg

Stefan, I haven't been able to reproduce your error on any of my machines; could you try the new build and see if it still happens for you?

[ViralBShah]

I really hope apple does not introduce more hoops.

[StefanKarpinski]

It doesn't fix it, but I think it may be something broken about my setup (although everything else seems to be working fine). I'm still running Mojave, so I wonder if maybe this has to do with that.

[staticfloat]

Are you dragging the app out of the dmg before running it?

[StefanKarpinski]

I tried it both ways. Same result either way.

[staticfloat]

Okay let’s get like 5-6 more people to try this. Please report your macOS version as you do so.

[ViralBShah]

I am on Catalina.

[sterlets]

Thanks for the fix, works fine with my Catalina, too.

[enshengdong]

Thanks for the solution. I am using Catalina 10.15.1. These steps save my day: 1) drag the app onto the desktop (or any other places); 2) In System Preferences -> Security & Privacy, unlock "prevent further changes" and trust "App Store and identified developers."

[JimLynchCodes]

Hi, I can't seem to run julia at all from the command line after trying to install it via brew just now on catalina 10.15.2

[JimLynchCodes]

I was able to get it working eventually. I had messed up my Julia installation so I first deleted it:

rm -rf /Applications/Julia-1.3.app

Then installed it fresh with Brew:

brew cast install julia

Then when the popup came up I clicked "Show in Finder". Once in Finder I right-clicked on Julia, selected "Open", and then selected "Open" on the next popup. After that I was able to run julia in any directory from the command line. 👍

[conradwt]

When I download Julia from

https://julialang-s3.julialang.org/bin/mac/x64/1.3/julia-1.3.1-mac64.dmg

I'm getting the following message when attempting to use the executable:

[ViralBShah]

@staticfloat Did we notarize 1.3.1? I suspect we did 1.3.0 but not 1.3.1.

[conradwt]

@ViralBShah Is the notarization part of the CI/CD process? I guess this would be something that must be done after a release build has been created.

[ViralBShah]

Code signing etc. are done manually, I believe. Notarization is another step on Mac, but see comments above on why it is hard to automate.

[staticfloat]

@conradwt can you try this file and see if it works for you: https://julialang2.s3.amazonaws.com/bin/mac/x64/1.3/julia-1.3.1-mac64-notarized.dmg

[conradwt]

@staticfloat I'm seeing the following within the browser:

[staticfloat]

sorry, try again

[conradwt]

@staticfloat This worked and thanks for the fast turnaround on this. 🙏🏾🙏🏾

[staticfloat]

Alright, in that case I'm going to move it over to the normal download spot. I've written a new script that makes this essentially a one-shot, so we'll hopefully be better prepared in the future. :)

[Jacob640]

I think this issue may have recurred for the latest version 1.3.1-1 I'm getting the same error when trying to install

[giordano]

What version is "1.3.1-1"?

[Jacob640]

@giordano Apologies I think this relates to JuliaPro not the base Julia install

[ctkelley]

This problem is back with Julia 1.4. Installed with the new .dmg on a Mac running latest Catalina.

[ctkelley]

Bizzare. The tricks that worked with 1.3 sorta work with this one. I finally got it going but it seemed like I had to repeat the process twice.

[fredrikekre]

https://github.com/JuliaLang/julia/issues/35211

[ViralBShah]

New binaries are being uploaded now. So this should be addressed.

[ctkelley]

Thanks, I've fixed it the old way but consistently have trouble
remembering what to do and the order to do it in

— Tim

On Sun, Mar 22, 2020 at 2:07 PM Viral B. Shah [e-mail]
wrote:

New binaries are being uploaded now. So this should be addressed.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/JuliaLang/julia/issues/33331#issuecomment-602248880,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACOEX66I42K5CUAR4UNZKY3RIZHXXANCNFSM4IYQ3RAQ
.

--
C. T. Kelley
Department of Mathematics, Box 8205
SAS Hall
2311 Stinson Drive
North Carolina State University
Raleigh, NC 27695-8205
[phone], [phone] (FAX)
[email protected]
https://ctk.math.ncsu.edu

[yw-fang]

Julia 1.3 has a same problem in my Catalina. However, it got solved after upgrading to Julia 1.4.2

[ViralBShah]

Yes, we probably did not notarize older releases.

[StefanKarpinski]

If I recall correctly, notarization wasn't a thing until after 1.3. We could retroactively notarize, but since the 1.3 line isn't supported anymore, there's not really bandwidth for that.