Go: cmd/go: macOS on arm64 requires codesigning

[FiloSottile]

On the production Apple Silicon machines, Go binaries are killed at start. https://github.com/golang/go/issues/38485#issuecomment-729301832

It looks like all binaries need to be codesigned now, and indeed running codesign -s - on them lets them run correctly.

This stops go test and go run from working, and requires an extra step after go build to get a functional binary.

This also affects the bootstrapped compiler itself.

[FiloSottile]

@cherrymui indeed, binaries produced by the machine's clang are codesigned, in what looks like the same exact way as the output of codesign -s -.

filippo@Filippos-MacBook-Pro tmp % clang hello.c
filippo@Filippos-MacBook-Pro tmp % ./a.out
Hello M1!
filippo@Filippos-MacBook-Pro tmp % codesign -d -v a.out                                 
Executable=/Users/filippo/tmp/a.out
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=510 flags=0x20002(adhoc,linker-signed) hashes=13+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

[FiloSottile]

Actually, codesign outputs don't have the linker-signed flag.

[cherrymui]

Thanks, @FiloSottile !

Interesting. The C compiler on the DTK doesn't do that...

Could you try one more thing: run clang -v hello.c to see if the C compiler invokes codesign or how it signs the binary? Thanks!

[FiloSottile]

It doesn't look like the linker invocation has anything special, so I assume ld just does that (and @jedisct1 said so on Twitter as well https://twitter.com/jedisct1/status/1328862207715794946).

"/Library/Developer/CommandLineTools/usr/bin/ld" -demangle -lto_library /Library/Developer/CommandLineTools/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch arm64 -platform_version macos 11.0.0 11.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o a.out -L/usr/local/lib /var/folders/jh/3ydm4lxd71s2__g_x4hny6r00000gn/T/hello-580ca7.o -lSystem /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/lib/darwin/libclang_rt.osx.a

I expect external linking should generate codesigned binaries, but trying `go build -ldflags="-linkmode=external" fails immediately with

# tmp
loadinternal: cannot find runtime/cgo

[cherrymui]

loadinternal: cannot find runtime/cgo

This is not a failure. It prints a message but it _should_ successfully link the binary.

[FiloSottile]

You're right, I do get the binary. Interestingly, it comes out linker-signed but doesn't run, and codesign doesn't work on it.

filippo@Filippos-MacBook-Pro tmp % codesign -d -v tmp                                                       
Executable=/Users/filippo/tmp/tmp
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=7038 flags=0x20002(adhoc,linker-signed) hashes=217+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none
filippo@Filippos-MacBook-Pro tmp % ./tmp             
zsh: killed     ./tmp
filippo@Filippos-MacBook-Pro tmp % codesign -s - tmp 
tmp: the codesign_allocate helper tool cannot be found or used

[cherrymui]

Thanks!

Interesting... I'll see if it is possible to generate LC_CODE_SIGNATURE in the linker. If that's not possible, maybe shell out codesign.

Any down side for that? If the user wants to sign with a different identity, would that still be possible?

[ianlancetaylor]

I don't understand this. What is the point of code signing, if every execution of clang or the Go compiler produces a code signed binary?

[FiloSottile]

Any down side for that?

I guess the main problem would be with cross-compilation, but anyway I guess adhoc signatures only work on the same machine where they are generated.

This would be a major pain for projects distributing binaries. I wonder how Homebrew deals with it, since by default they install pre-built "Bottles".

What is the point of code signing, if every execution of clang or the Go compiler produces a code signed binary?

I don't know, but I can imagine 1) to normalize the execution path such that it always check signatures unconditionally, simplifying it and 2) to block binaries generated on other machines (and not properly signed by a trusted Developer ID) which before was sort of the role of the quarantine attributes.

[terinjokes]

You're right, I do get the binary. Interestingly, it comes out linker-signed but doesn't run, and codesign doesn't work on it.

My understanding is this is a bug already reported to Apple, see https://github.com/Homebrew/brew/issues/9082#issuecomment-727247739

I wonder how Homebrew deals with it, since by default they install pre-built "Bottles".

Per https://github.com/Homebrew/brew/issues/7857#issuecomment-727561399 I don't think they're close to code signing bottles on ARM yet.

[rolandshoemaker]

2) to block binaries generated on other machines (and not properly signed by a trusted Developer ID) which before was sort of the role of the quarantine attributes.

Looks like binaries aren't required to have a signature linked to a Developer ID if they were signed elsewhere. I was able to apply an ad-hoc signature to a binary on my Intel mac and transfer it to my M1 mac and it ran just fine. Appears like binaries just need a signature, doesn't really seem to matter where it came from.

[fxcoudert]

Homebrew maintainer here:

the codesign_allocate helper tool cannot be found or used

That's indeed a bug in codesign, and Apple is aware

My understanding is this is a bug already reported to Apple, see Homebrew/brew#9082 (comment)

Yes. If this occurs, you need to change that file's inode (so copying it someplace and back will work). Once you've done that, you can sign and it will work. See https://github.com/Homebrew/brew/pull/9102 for details.

I don't think they're close to code signing bottles on ARM yet

We do not distribute bottles yet, because our CI is not yet fully operational, but our codebase is otherwise ready. We apply ad-hoc signatures as part of formula installation: https://github.com/Homebrew/brew/pull/9102

[fxcoudert]

As for go, if Go uses its own linker (or anything other than Apple's ld) then it definitely needs to call codesign -s - on the binaries it produces: executables, shared libraries, etc.

[mislav]

@FiloSottile (and others): thank you for the information! 🙇

Do you know if it is possible to use darwin/amd64 architecture to cross-compile binaries that are able to execute on darwin/arm64, providing that the binaries are explicitly signed?

I am interested in what are the options to distribute binaries for Go projects that work on Apple Silicon without having to actually compile them on Apple Silicon.

[bwesterb]

@mislav Cross-compiling for darwin/arm64 works fine from darwin/amd64 and even linux/amd64. Obviously the resulting binaries are not signed. They will run if signed ad hoc on the same machine they will run. Binaries ad hoc signed on one machine didn't work on another machine for me as well.

[jameshartig]

Binaries ad hoc signed on one machine didn't work on another machine for me.

This comment seems to disagree: https://github.com/golang/go/issues/42684#issuecomment-729346276

I was able to apply an ad-hoc signature to a binary on my Intel mac and transfer it to my M1 mac and it ran just fine.

[bwesterb]

Ok, I tested it again and now cross-signing does work. I might've transferred the wrong file earlier. Whoops.

[FiloSottile]

Codesigning from cmd/link should be possible. https://github.com/isignpy/isign by @neilk does that for iOS.

[networkimprov]

The Zig project is also working on this here: ziglang/zig#7103 (thanks @komuw).

[cherrymui]

Thanks for the pointers. I'll look into them.

[fxcoudert]

Binaries ad hoc signed on one machine didn't work on another machine for me

I can confirm that a linker or other ad hoc signature is sufficient to run the binary (or shared library) on any machine.

[rolandshoemaker]

A quick note from trying to hack together a working toolchain today: shelling out to codesign in the linker works, but any subsequent calls to buildid to update the build ID appears to break the signature (preventing the update produces working binaries). Presumably the signature will need to be done (either magically by the linker with LC_CODE_SIGNATURE or shelling to codesign) as the last step after any changes are made to the object.

[cherrymui]

@rolandshoemaker yeah, I tried that, too, and come to the same conclusion.

I'm not sure whether signing after stamping buildid will cause any issue for the go command's staleness analysis, though.

[marcopeereboom]

This is total crap. Guess Apple does not want developers using their new hardware? Or forcing them to use the entire operating system as a very fancy ssh terminal.

[dmitshur]

@marcopeereboom Please be mindful of Gopher values when communicating in the Go issue tracker. Criticism is welcome when it's constructive and adds new information to the discussion, otherwise it does not help us get to a resolution of this issue.

[FiloSottile]

@kb091412 on Twitter found the official docs, which basically confirm everything we had deduced: https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-universal-apps-release-notes#Code-Signing

New in macOS 11 on Macs with Apple silicon, and starting in macOS Big Sur 11 beta 6, the operating system enforces that any executable must be signed before it’s allowed to run. There isn’t a specific identity requirement for this signature: a simple ad-hoc signature is sufficient. This new behavior doesn’t change the long-established policy that our users and developers can run arbitrary code on their Macs, and is designed to simplify the execution policies on Macs with Apple silicon and enable the system to better detect code modifications. This new policy doesn’t apply to translated x86 binaries running under Rosetta 2, nor does it apply to macOS 11 running on Intel-based platforms.

To reduce the impact on existing development workflows, starting with Xcode 12 beta 4, the toolchain will now automatically sign your executables whenever you build from Xcode, or use command-line utilities such as clang(1) or ld(1). Since this signing mechanism generates signatures directly at link time, and doesn’t cover any resource other than the executable, it is expected to be faster than a traditional codesign(1) invocation. If you use a custom workflow involving tools that modify a binary after linking (e.g. strip or install_name_tool) you might need to manually call codesign(1) as an additional build phase to properly ad-hoc sign your binary. These new signatures are not bound to the specific machine that was used to build the executable, they can be verified on any other system and will be sufficient to comply with the new default code signing requirement on Macs with Apple silicon. However, given that these signatures do not bear any valid identity, binaries signed this way cannot pass through Gatekeeper.

[jrick]

Interesting. Would love to see Go implement this ad-hoc signing directly instead of needing to shell out to codesign, so that cross compiling usable binaries is still possible. Would also be good to use a static key so we can keep reproducibility.

[leo60228]

An open-source ad-hoc signing tool was created for NixOS: https://github.com/thefloweringash/sigtool

[leo60228]

Using it requires Apple's codesign_allocate, but that's open-source as well: https://opensource.apple.com/source/cctools/cctools-949.0.1/misc/codesign_allocate.c.auto.html

[rolandshoemaker]

@cherrymui Oh hm, good point. I think signing after buildid would indeed break the staleness check, since adding the LC_CODE_SIGNATURE section will change the file hash, just as stamping buildid after signing breaks the signature for the same reason.

It seems like perhaps buildid on darwin would need to understand where the LC_CODE_SIGNATURE section is (or would be) in the object in order to create a hash that ignores that section and can be recalculated after the signature is added 😕.

[meme]

Found a workaround that I'm using for now. The AppleMobileFileIntegrity.kext is what handles checking for code signing, and you can see in the Console that AMFI prohibits running:

AMFI: hook..execve() killing pid 9799: Attempt to execute completely unsigned code (must be at least ad-hoc signed).

However there is an undocumented boot argument called amfi_block_unsigned_code which disables this check.

Enter recovery mode by holding down the MacBook power button when the MacBook is powered off and wait until it says that it's loading the settings. Enter settings and open a terminal:

csrutil enable --without nvram

This will leave all security features intact except disable "NVRAM Protections" and "Boot-arg Restrictions". Reboot back into macOS and set the boot argument:

sudo nvram boot-args="amfi_block_unsigned_code=0"

Now reboot once again and you'll be able to run unsigned applications. Note however that a new message will be presented at boot:

AMFI: You have used the amfi_block_unsigned_code boot-arg. This will be removed in a future release.

This will be useful for developing the Go toolchain on Apple Silicon devices and toggling this behaviour.

[kubkon]

Using it requires Apple's codesign_allocate, but that's open-source as well: https://opensource.apple.com/source/cctools/cctools-949.0.1/misc/codesign_allocate.c.auto.html

The good news here is that codesign_allocate is only used to verify the basic MachO layout of the __LINKEDIT hidden sections, and then inject an empty, 0-padded placeholder for the code signature as the very last section, plus a matching LC_CODE_SIGNATURE load command.

[kubkon]

@cherrymui Oh hm, good point. I think signing after buildid would indeed break the staleness check, since adding the LC_CODE_SIGNATURE section will change the file hash, just as stamping buildid after signing breaks the signature for the same reason.

It seems like perhaps buildid on darwin would need to understand where the LC_CODE_SIGNATURE section is (or would be) in the object in order to create a hash that ignores that section and can be recalculated after the signature is added 😕.

Do you think the code signature will always be of the same size? What I inferred about codesign is that it precalculates the size needed and then calls codesign_allocate to preallocate the space in the MachO binary before the actual SuperBlob structure is generated and inserted.

[leo60228]

Based on sigtool's source, it seems like the size of the signature is variable. I'm not a Go developer, but I'd guess that porting sigtool and codesign_allocate (both of which are open source) would make it pretty easy to add signing support to Go.

[kubkon]

@leo60228 oh for sure. I was just wondering if you knew if it was fixed; then porting codesign_allocate functionality would amount to finding the offset of string table, and padding out a fixed size slice at string table offset plus its size. But since the size of the signature can be variable, this has to be taken into account, and I guess that the staleness would indeed break because of it, unless the code signature command and the matching section are ignored from this calculation. Anyhow, we'll be figuring out something similar for Zig, so I'll be more than happy to share any exciting news of success (if any).

[cherrymui]

Thanks all above for thoughts and pointers! Here is my plan:

• Before stamping the buildid, allocate the LC_CODE_SIGNATURE load command with the right offset and size, zero-fill the signature blob. Probably done by the linker.
• Stamp the buildid. The buildid would be made to be insensitive to the LC_CODE_SIGNATURE blob. (but can be sensitive to the load command itself, which should not change)
• Fill in the signature blob. Since the buildid is insensitive to the signature blob, this shouldn't change the buildid.

I _guess_ this may work. I'll try to code up something...

[kubkon]

That's my plan for Zig as well minus the buildid step ;-) I also plan to stretch the structure a little bit in the sense I really don't like the fact that codesign_allocate bullies me into having no gaps between the __LINKEDIT hidden sections. I wonder if this is indeed strictly required for the code signature to be properly generated.

[nikitavoloboev]

You don't have to pay anyone to code sign things right? It's just about running codesign <path-to-binary> and then sharing it with everyone?

[kubkon]

You don't have to pay anyone to code sign things right? It's just about running codesign <path-to-binary> and then sharing it with everyone?

As long as we're talking about adhoc code signing, then yep, that's the case. The full command for this would actually be codesign -s - <path_to_binary>.

[nikitavoloboev]

As long as we're talking about adhoc code signing

And then the users would be still met with a warning. Like this app is from unidentified developer, you sure you want to run it? or it will be okay to run as if everything is normal?

[rfay]

I should mention that real codesigning can be bundled up several ways without leaning the whole process. https://github.com/drud/signing_tools has the tools that we use (in Catalina) for signing our golang binaries. It does require an Apple developer account. And they do break it regularly by requiring you to sign off on contract changes.

[kubkon]

As long as we're talking about adhoc code signing

And then the users would be still met with a warning. Like this app is from unidentified developer, you sure you want to run it? or it will be okay to run as if everything is normal?

They will indeed. The GateKeeper is still there. If you want to please it, you gotta code sign with an actual, valid developer identity.

A little bit more about adhoc code signing, not sure if this was mentioned here or not, but @andrewrk and myself have exchanged a couple of emails with Apple officials, and they told us that modification of an already signed file in-place will end up in SIGKILL 9 by the kernel even if you regenerate the signature. That's because the kernel apparently caches code signing info for each inode. Interestingly, you probably won't notice this when using codesign tool for instance, as it actually does a full file copy under-the-hood. This probably doesn't hurt Go as much as it does Zig where we perform incremental linking in-place.

[terinjokes]

That's because the kernel apparently caches code signing info for each inode.

I don't have an M1 to test with, so this might be a dumb question. Does modifying a codesigned executable in place invalidate this cache?

[kubkon]

That's because the kernel apparently caches code signing info for each inode.

I don't have an M1 to test with, so this might be a dumb question. Does modifying a codesigned executable in place invalidate this cache?

Yep, it unfortunately does. Then, you either have to copy the entire contents into a new inode and code sign anew (thinking here of codesigning solutions that are part of Go/Zig's toolchains), or use codesign tool which will do that for you.

[networkimprov]

This is the Apple docs on running an app from an unidentified developer. In Catalina, it's considerably more complicated than previously -- you can't just Ctrl-click the app icon and select "Open" to get a dialog with a run-anyway option.

https://support.apple.com/en-us/HT202491

Has this changed in Big Sur?

[jpap]

A little bit more about adhoc code signing, not sure if this was mentioned here or not, but @andrewrk and myself have exchanged a couple of emails with Apple officials, and they told us that modification of an already signed file in-place will end up in SIGKILL 9 by the kernel even if you regenerate the signature. That's because the kernel apparently caches code signing info for each inode. Interestingly, you probably won't notice this when using codesign tool for instance, as it actually does a full file copy under-the-hood. This probably doesn't hurt Go as much as it does Zig where we perform incremental linking in-place.

My understanding is that the inode caching only gets in the way if you first execute the binary before (re-)codesign. Does Zig execute a binary before a final codesign step?

[andrewrk]

Yes, it's part of the edit-compile-run development cycle. Programmer edits code. Compiler updates the binary in place. Programmer runs & debugs the application. Repeat.

[kubkon]

Oh no, you're right. I guess I didn't explain myself well. That's the use case I had in mind. In Zig we're working on hot code swapping and incremental linking, so you can run the binary and then recompile just bits of it. Everything is done in place.

[jpap]

Oh no, you're right. I guess I didn't explain myself well. That's the use case I had in mind. In Zig we're working on hot code swapping and incremental linking, so you can run the binary and then recompile just bits of it. Everything is done in place.

Ah that all makes perfect sense. Thanks for the reply @andrewrk and @kubkon. Go performs builds in a new $TMPDIR each time the build cache is stale, so I guess each unique binary will have a different inode, even when moved into its final directory to replace an existing executable.

[andrewrk]

Yep we will be forced to do something similar with the new code signing requirements. Most likely performing the in-place update within the zig-cache directory, and then copying the binary to the output each time. Sadly Darwin has no copy_file_range syscall.

[cherrymui]

I hacked up something that does code signing. https://go-review.googlesource.com/c/scratch/+/271866
It just a separate command line program, that signs binary after it builds, not yet integrated to the toolchain. I just wanted to see if the algorithm works.

It seems to make the "codesign" command line tool happy. Could someone with an M1 machine give it a try, see if the kernel likes the binaries signed by this program? Just do

GOARCH=arm64 GOOS=darwin go build hello.go # generate unsigned binary
./codesign hello                           # sign

Thanks!

If this works, I'll do it for real.

[rolandshoemaker]

@cherrymui looks like the kernel doesn't like it unfortunately

2020-11-19 20:13:58.195388-0800 0xbc230    Default     0x0                  0      0    kernel: CODE SIGNING: cs_invalid_page(0x100a5c000): p=48498[test] final status 0x23020200, denying page sending SIGKILL
2020-11-19 20:13:58.195416-0800 0xbc230    Default     0x0                  0      0    kernel: CODE SIGNING: process 48498[test]: rejecting invalid page at address 0x100a5c000 from offset 0x0 in file "/Users/rolandshoemaker/test" (cs_mtime:1605845619.452643497 == mtime:1605845619.452643497) (signed:1 validated:1 tainted:1 nx:0 wpmapped:1 dirty:0 depth:0)

[networkimprov]

It would be helpful to have a go tool ... command to sign objects besides Go binaries, for instance shell scripts.

I realize that exists on MacOS, but many folks assemble releases on Linux, etc.

[dmitshur]

@cherrymui I got the same result as @rolandshoemaker. In case it helps, some details:

$ export PATH="$PATH:$HOME/go116arm64/bin"
$ GOOS=darwin GOARCH=arm64 go build -o h0 hello.go
$ GOOS=darwin GOARCH=arm64 go build -o h1 hello.go && codesign -s - ./h1
$ GOOS=darwin GOARCH=arm64 go build -o h2 hello.go && /tmp/scratch/cherrycodesign ./h2

$ ./h0
zsh: killed     ./h0
$ ./h1
hello
$ ./h2
zsh: killed     ./h2

$ ls -la h*
-rwxr-xr-x  1 dmitri  staff  2042752 19 Nov 23:11 h0
-rwxr-xr-x  1 dmitri  staff  2087152 19 Nov 23:12 h1
-rwxr-xr-x  1 dmitri  staff  2058833 19 Nov 23:23 h2

$ diff -U0 <(codesign -d -vvvv ./h1 2>&1) <(codesign -d -vvvv ./h2 2>&1)
--- /dev/fd/11  2020-11-19 23:26:27.000000000 -0500
+++ /dev/fd/12  2020-11-19 23:26:27.000000000 -0500
@@ -1,2 +1,2 @@
-Executable=/private/var/folders/_0/h0671fcn4rgb5pn9c745dx2h0000gn/T/tmp.lKFaKwXh/h1
-Identifier=h1-962eb7c97571405dab96e632e60472cbd44c02f7
+Executable=/private/var/folders/_0/h0671fcn4rgb5pn9c745dx2h0000gn/T/tmp.lKFaKwXh/h2
+Identifier=./h2
@@ -4 +4 @@
-CodeDirectory v=20100 size=16124 flags=0x2(adhoc) hashes=499+2 location=embedded
+CodeDirectory v=20400 size=16061 flags=0x20002(adhoc,linker-signed) hashes=499+0 location=embedded
@@ -6,6 +6,4 @@
-CandidateCDHash sha1=44d83717b7f5d541d2dd5eee1cebfa5e49717088
-CandidateCDHashFull sha1=44d83717b7f5d541d2dd5eee1cebfa5e49717088
-CandidateCDHash sha256=710421ab654b828da6d1341fa487e4243d162103
-CandidateCDHashFull sha256=710421ab654b828da6d1341fa487e4243d162103b59185092c8759fadad50e61
-Hash choices=sha1,sha256
-CMSDigest=f2ba67a05485afc727cd4a9a2f95d1d04266eb0f7519897eeda415bcd21dae80
+CandidateCDHash sha256=c273a9f6c2be4c0bc4f040f542ea6d2c3b7057c4
+CandidateCDHashFull sha256=c273a9f6c2be4c0bc4f040f542ea6d2c3b7057c4ebb7d0b69167a44b8eeef3ff
+Hash choices=sha256
+CMSDigest=c273a9f6c2be4c0bc4f040f542ea6d2c3b7057c4ebb7d0b69167a44b8eeef3ff
@@ -14 +12 @@
-CDHash=710421ab654b828da6d1341fa487e4243d162103
+CDHash=c273a9f6c2be4c0bc4f040f542ea6d2c3b7057c4
@@ -19 +17 @@
-Internal requirements count=0 size=12
+Internal requirements=none

[rolandshoemaker]

@cherrymui in case it's useful here is the output of jtool --sig for the same binary signed using codesign -s - test and your tool, looks like codesign does a few things differently when constructing the blobs: https://gist.github.com/rolandshoemaker/edfbde70c60d1469a1ec6e5d49f70f86

[jpap]

@cherrymui, I think it's because your adhoc CD lacks a CMS encoded signature (type 0x1000) inside the embedded signature 0xfade0cc0 superblob. Apple's adhoc signature includes one with a zero-length CMS encoded signature payload.

You can see that in @rolandshoemaker's jtool dump, where the 8 bytes is the blob header (leaving no bytes for any payload):

Blob 3: Type: 10000 @26504: Blob Wrapper (8 bytes) (0x10000 is CMS (RFC3852) signature)

---

Edit: scratch the above. I created a CD without a CMS encoded signature and runs on the M1 just fine.

[cherrymui]

Could you dump a C binary built by the C toolchain? I know that codesign -s - does more, but it seems the one that ld does what I did (ld64-609), only 1 blob, which I tried to do.

Anyway, if that doesn't work, I'll try to do what codesign does...

[rolandshoemaker]

Oh interesting, yeah clang produces a very different looking blob that is similar to what you're doing:

Blob at offset: 49424 (530 bytes) is an embedded signature of 530 bytes, and 1 blobs
    Blob 0: Type: 0 @20: Code Directory (510 bytes)
        Version:     20400
        Flags:       adhoc (0x20002) (0x20002)
        CodeLimit:   0xc110
        Identifier:  a.out (0x58)
        Executable Segment: Base 0x00000000 Limit: 0x00000000 Flags: 0x00000000
        CDHash:      5bf2a837dac0d47e78f3c5ffaac83e6a0053ba15f95865f548701068d8fd82bb (computed)
        # of Hashes: 13 code + 0 special
        Hashes @94 size: 32 Type: SHA-256
            Slot   0 (File page @0x0000):   2a7d47fb7ca6d1942ec4b598da0b48420f6773223e758f890da5bf429f0b019c (OK)
            Slot   1 (File page @0x1000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   2 (File page @0x2000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   3 (File page @0x3000):   d20ec350a5e965e5c23c0382ba34e2e3704386f809bbe35979d2156520b6c904 (OK)
            Slot   4 (File page @0x4000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   5 (File page @0x5000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   6 (File page @0x6000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   7 (File page @0x7000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   8 (File page @0x8000):   7475d6c89d2de31db7ebf77586309b2ea6cf8b157fec1534bce1583f4b5cdc7f (OK)
            Slot   9 (File page @0x9000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot  10 (File page @0xa000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot  11 (File page @0xb000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot  12 (File page @0xc000):   5d7445f5164527896faba9cbb23606bb16e0b16d4e88625101f1c7aa5f992c3b (OK)

[jpap]

Ouch... codesign.go isn't fixed by padding the LC_CODE_SIGNATURE to 4k; I've deleted my previous post as a result.

It turned out that by running my own adhoc signed version of the same binary, it somehow "enabled" running of the same binary that was processed by codesign.go. (Perhaps the kernel keeps a cache of validated executables?) After a machine reboot, the codesign.go signed executable stopped working.

I'll try and dig a bit deeper into what the differences are between my tool and codesign.go, with reboots to make sure this time.

[jpap]

OK, after some digging, I think I have found the problem. You are using version 0x20400 of the CodeDirectory (CD), which has the following fields (page 8):

• exec segment base
• exec segment limit
• exec segment flags

These 64-bit fields need to match the __TEXT segment found in the LC_SEGMENT_64, which you would need to parse from the Mach-O header. You are setting them all to zero, which doesn't match the __TEXT segment.

Since an adhoc signature is as simple as you can get, you don't need the features of the most recent CD versions: an easier fix here is to just use an earlier version of the CD (see the diff below). If you use version 0x20001, you don't need the above fields, so you don't need to look for the __TEXT segment, and then your CD passes validation.

I have verified this after rebooting my M1, to be sure. :)

codesign.go diff

$ diff -Naur codesign.orig.go codesign.go
--- codesign.orig.go    2020-11-19 23:31:36.000000000 -0800
+++ codesign.go 2020-11-20 00:32:44.000000000 -0800
@@ -72,21 +72,21 @@
 }

 type CodeDirectory struct {
-   magic         uint32     // magic number (CSMAGIC_CODEDIRECTORY)
-   length        uint32     // total length of CodeDirectory blob
-   version       uint32     // compatibility version
-   flags         uint32     // setup and mode flags
-   hashOffset    uint32     // offset of hash slot element at index zero
-   identOffset   uint32     // offset of identifier string
-   nSpecialSlots uint32     // number of special hash slots
-   nCodeSlots    uint32     // number of ordinary (code) hash slots
-   codeLimit     uint32     // limit to main image signature range
-   hashSize      uint8      // size of each hash in bytes
-   hashType      uint8      // type of hash (cdHashType* constants)
-   _pad1         uint8      // unused (must be zero)
-   pageSize      uint8      // log2(page size in bytes); 0 => infinite
-   _pad2         uint32     // unused (must be zero)
-   _pad3         [11]uint32 // 64-bit extension?
+   magic   uint32 // magic number (CSMAGIC_CODEDIRECTORY)
+   length  uint32 // total length of CodeDirectory blob
+   version uint32 // compatibility version
+   //
+   flags         uint32 // setup and mode flags
+   hashOffset    uint32 // offset of hash slot element at index zero
+   identOffset   uint32 // offset of identifier string
+   nSpecialSlots uint32 // number of special hash slots
+   nCodeSlots    uint32 // number of ordinary (code) hash slots
+   codeLimit     uint32 // limit to main image signature range
+   hashSize      uint8  // size of each hash in bytes
+   hashType      uint8  // type of hash (cdHashType* constants)
+   _pad1         uint8  // unused (must be zero)
+   pageSize      uint8  // log2(page size in bytes); 0 => infinite
+   _pad2         uint32 // unused (must be zero)
    // data follows
 }

@@ -105,9 +105,6 @@
    out = put8(out, c._pad1)
    out = put8(out, c.pageSize)
    out = put32be(out, c._pad2)
-   for _, x := range c._pad3 {
-       out = put32be(out, x)
-   }
    return out
 }

@@ -130,6 +127,7 @@
 func get32le(b []byte) uint32           { return binary.LittleEndian.Uint32(b) }
 func put32le(b []byte, x uint32) []byte { binary.LittleEndian.PutUint32(b, x); return b[4:] }
 func put32be(b []byte, x uint32) []byte { binary.BigEndian.PutUint32(b, x); return b[4:] }
+func put64be(b []byte, x uint64) []byte { binary.BigEndian.PutUint64(b, x); return b[8:] }
 func put64le(b []byte, x uint64) []byte { binary.LittleEndian.PutUint64(b, x); return b[8:] }
 func put8(b []byte, x uint8) []byte     { b[0] = x; return b[1:] }
 func puts(b, s []byte) []byte           { n := copy(b, s); return b[n:] }
@@ -261,7 +259,7 @@
    cdir := CodeDirectory{
        magic:       CSMAGIC_CODEDIRECTORY,
        length:      uint32(sz) - uint32(unsafe.Sizeof(SuperBlob{})+unsafe.Sizeof(Blob{})),
-       version:     0x20400,
+       version:     0x20001,
        flags:       0x20002, // adhoc | linkerSigned
        hashOffset:  uint32(hashOff),
        identOffset: uint32(idOff),

[kubkon]

Just a heads up that I've also managed to implement adhoc code signing over in Zig's self-hosted compiler. The binaries generated with the self-hosted now are code signed out of the box, and are not greeted with SIGKILL by the kernel (tested on the DTK). If it's of any help, here's the sources: zig/src/link/MachO/CodeSignature.zig. I'll also be more than happy to help out in any way I can if anyone needs any help with this! Just lemme know ;-)

[markmentovai]

I don't understand this. What is the point of code signing, if every execution of clang or the Go compiler produces a code signed binary?

Their system has long had a mechanism to identify code by a detached (not embedded in the executable code module) ad-hoc signature. This new development gets rid of the detached aspect by requiring signatures to be explicit and attached, even if ad-hoc. It’s a simplification of their system that eliminates a funny quirk in a corner of code signing that wasn’t well-understood.

Details. Excerpting:

New in macOS 11 on Macs with Apple silicon, and starting in macOS Big Sur 11 beta 6, the operating system enforces that any executable must be signed before it’s allowed to run. There isn’t a specific identity requirement for this signature: a simple ad-hoc signature is sufficient. This new behavior doesn’t change the long-established policy that our users and developers can run arbitrary code on their Macs, and is designed to simplify the execution policies on Macs with Apple silicon and enable the system to better detect code modifications.

Interesting. The C compiler on the DTK doesn't do that...

You should be getting linker-signed code if you’re using the toolchain from Xcode 12.0b4 or newer and targeting arm64, regardless of whether you run it on arm64 (including DTK) or x86_64.

It doesn't look like the linker invocation has anything special, so I assume ld just does that

It does, since Xcode 12.0b4. ld64 has two new options, -adhoc_codesign and -no_adhoc_codesign. These can be used to overcome the defaults, which are to create an ad-hoc linker-signed signature when targeting arm64, and to not do so when targeting x86_64.

Interesting... I'll see if it is possible to generate LC_CODE_SIGNATURE in the linker. If that's not possible, maybe shell out codesign.

You _can_ do this, but codesign --sign doesn’t produce the linker-signed bit, which is somewhat valuable:

In order for this new system to adhere to existing developer expectations, it needs to be possible to codesign --sign linker output, either ad-hoc or, more interestingly, with an actual identity, according to existing workflows. If it’s already signed, codesign --sign refuses to operate on it without --force. On the other hand, if it’s signed but with the linker-signed bit, codesign --sign will happily introduce a new signature, even absent --force. (The wrinkle here is that codesign is an OS tool, not an Xcode tool, so this behavior requires codesign on macOS 11.0. Even though you can use Xcode 12.2 on macOS 10.15, you’ll still be using an older codesign in that case, and it knows nothing about the linker-signed attribute. Here’s how I dealt with that discrepancy in Chrome.)

There are other toolchain tools that modify executable modules and also need to be aware of linker-signed code and treat it differently, such as strip and install_name_tool, although possibly neither of these are very relevant to Go linker output.

It would be best if mac-arm64 Mach-O output produced by any part of Go’s toolchain came with an ad-hoc _linker-signed_ signature by default.

As some of the recent commenters above have discovered, it’s not terribly difficult to generate a usable ad-hoc signature on your own.

[networkimprov]

Re properly signing Go binaries for distribution, could we point the linker at a certificate? Or must we copy to a Mac and run codesign?

[cherrymui]

Thanks for the information!

It would be best if mac-arm64 Mach-O output produced by any part of Go’s toolchain came with an ad-hoc linker-signed signature by default.

Yeah, that's the plan.

[cherrymui]

Thanks all above for trying it out and providing informations, especially @jpap !

Just to be sure, could someone confirm that the updated version of codesign.go on https://go-review.googlesource.com/c/scratch/+/271866 works on M1? Thanks.

It is interesting that ld64 uses version 0x20400 with all later fields as zero and it worked (e.g. https://github.com/golang/go/issues/42684#issuecomment-730874626). I may dig a little more.

[jpap]

It is interesting that ld64 uses version 0x20400 with all later fields as zero and it worked (e.g. #42684 (comment)). I may dig a little more.

That's a known bug in jtool; see the example below.

Details

$ codesign -d -vvvvv hello 
Executable=hello
Identifier=hello
Format=Mach-O thin (x86_64)
CodeDirectory v=20400 size=510 flags=0x2(adhoc) hashes=13+0 location=embedded
VersionPlatform=1
VersionMin=720896
VersionSDK=720896
Hash type=sha256 size=32
CandidateCDHash sha256=b6417d252c7f06bd3e7060f41616a965c4f0b8ae
CandidateCDHashFull sha256=b6417d252c7f06bd3e7060f41616a965c4f0b8ae800389a82c5a28a89ba26327
Hash choices=sha256
CMSDigest=b6417d252c7f06bd3e7060f41616a965c4f0b8ae800389a82c5a28a89ba26327
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=16384
Executable Segment flags=0x1
Page size=4096
CDHash=b6417d252c7f06bd3e7060f41616a965c4f0b8ae
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none
$
$
$ jtool -v --sig hello 
Blob at offset: 49472 (4088 bytes) is an embedded signature of 530 bytes, and 1 blobs
    Blob 0: Type: 0 @20: Code Directory (510 bytes)
        Version:     20400
        Flags:       adhoc (0x2) (0x2)
        CodeLimit:   0xc140
        Identifier:  hello (0x58)
        Executable Segment: Base 0x00000000 Limit: 0x00000000 Flags: 0x00000000
        CDHash:      b6417d252c7f06bd3e7060f41616a965c4f0b8ae800389a82c5a28a89ba26327 (computed)
        # of Hashes: 13 code + 0 special
        Hashes @94 size: 32 Type: SHA-256
            Slot   0 (File page @0x0000):   73d0ed40a765c0efcbf4b4b1cadb65743305224e6b4c174b2c25e11248ffa524 (OK)
            Slot   1 (File page @0x1000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   2 (File page @0x2000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   3 (File page @0x3000):   e2357f1795721768662726f36224f36900802501bd8d93053f2595b3b99f0c79 (OK)
            Slot   4 (File page @0x4000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   5 (File page @0x5000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   6 (File page @0x6000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   7 (File page @0x7000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot   8 (File page @0x8000):   4586671c3ba8ead84432c824ab30f5995c0f60faa6468212836809a2ede0babb (OK)
            Slot   9 (File page @0x9000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot  10 (File page @0xa000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot  11 (File page @0xb000):   ad7facb2586fc6e966c004d7d1d16b024f5805ff7cb47c7a85dabd8b48892ca7 (OK)
            Slot  12 (File page @0xc000):   41b367d82b13868e6ceeb3fb76e7c1c57cd9a233a4550f4074a323bcffc2c6ec (OK)
$ 

[markmentovai]

Re properly signing Go binaries for distribution, could we point the linker at a certificate? Or must we copy to a Mac and run codesign?

The linker-signed signatures are ad-hoc, and aside from that one linker-signed bit, they’re essentially the same as the more traditional non-linker-signed ad-hoc signatures (codesign --sign=-). An ad-hoc signature isn’t really a signature in the strict cryptographic sense, it’s effectively just a digest.

Signing with an actual certificate (or, as Apple says, “identity”) is quite a bit different. This adds a CMS message signing (this time, in the cryptographic sense) the digest as proof of _who_ signed the object.

Certainly, golang has strong crypto support, and it may well be possible to build a suitable signature without too much additional effort, but I think that doing it here as part of the linker output would be misguided. If someone’s going to go to the trouble of writing a drop-in for Apple codesign --sign in golang, I’d certainly hope it would be available as a general-purpose tool usable on a variety of Mach-O files, not just those produced by the Go linker. (Although, once you’re at that point, sure, no reason not to give the linker the ability to do it too. Doesn’t really matter.)

As far as “signing for distribution” goes, that needs to be considered in the context of the larger packaging scheme. Notably, you’ve got a bunch of stuff to sign and notarize, then you’ve got a .pkg to build, and then you’ve got to sign, notarize, and notarization-staple that too. You might find drop-in replacements for Apple tools for some of these steps, but there are an awful lot of Apple tools to replace if you want to totally remove macOS from the official-release pipeline.

[dmitshur]

Just to be sure, could someone confirm that the updated version of codesign.go on https://go-review.googlesource.com/c/scratch/+/271866 works on M1? Thanks.

@cherrymui I tried again with Patch Set 2, it did not work on a MacBook Air with M1. Let me know if I can provide specific information or remote access to it that would help you.

[cherrymui]

@dmitshur what about PS3? I just updated the CL again.

@jpap I hex-dumped a signature generated by ld64, and those fields are actually 0.

Looking again I think I may need to round up the __LINKEDIT segment's VMSize, which I didn't.

[jpap]

@jpap I hex-dumped a signature generated by ld64, and those fields are actually 0.

That's interesting -- here's what I get when I build a simple program on the M1:

$ cat test.c
#include <stdio.h>

int main() {
        printf("hello world\n");
        return 0;
}

$ clang -arch arm64 -o test test.c
$ file test
test: Mach-O 64-bit executable arm64
$ codesign -d -vvvv test
Executable=/Users/jpap/test
Identifier=test
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=509 flags=0x20002(adhoc,linker-signed) hashes=13+0 location=embedded
VersionPlatform=1
VersionMin=720896
VersionSDK=720896
Hash type=sha256 size=32
CandidateCDHash sha256=f54d00ba0c20977c1a2a14e129ca6038500aff97
CandidateCDHashFull sha256=f54d00ba0c20977c1a2a14e129ca6038500aff971a0ef98f1ab927e2fa3fe272
Hash choices=sha256
CMSDigest=f54d00ba0c20977c1a2a14e129ca6038500aff971a0ef98f1ab927e2fa3fe272
CMSDigestType=2
Executable Segment base=0
Executable Segment limit=16384
Executable Segment flags=0x1
Page size=4096
CDHash=f54d00ba0c20977c1a2a14e129ca6038500aff97
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none
$ 

I also used a custom CD parser and got the same result... 0x0, 0x4000, and 0x1 are definitely in there. :)

[cherrymui]

Interesting. Maybe due to ld64 version? I have ld64-609. It generates the signature but the "Executable Segment" fields are 0. (And ld64-607.2 on another machine doesn't sign at all.)

Apparently it's not hard to set those fields. I could just do that, if that's what matters.

[dmitshur]

@dmitshur what about PS3? I just updated the CL again.

@cherrymui Same outcome with PS3.

[jpap]

Interesting. Maybe due to ld64 version? I have ld64-609. It generates the signature but the "Executable Segment" fields are 0. (And ld64-607.2 on another machine doesn't sign at all.)

Apparently it's not hard to set those fields. I could just do that, if that's what matters.

I've got 609.7.

Details

$ ld -version_details
{
    "version": "609.7",
    "architectures": [
        "armv6",
        "armv7",
        "armv7s",
        "arm64",
        "arm64e",
        "arm64_32",
        "i386",
        "x86_64",
        "x86_64h",
        "armv6m",
        "armv7k",
        "armv7m",
        "armv7em"
    ],
    "lto": {
        "runtime_api_version": 27,
        "static_api_version": 27,
        "version_string": "LLVM version 12.0.0, (clang-1200.0.32.27)"
    },
    "tapi": {
        "version": "12.0.0",
        "version_string": "Apple TAPI version 12.0.0 (tapi-1200.0.23.4)"
    }
}
$ 

[jpap]

Just to be sure, could someone confirm that the updated version of codesign.go on https://go-review.googlesource.com/c/scratch/+/271866 works on M1? Thanks.

@cherrymui I tried again with Patch Set 2, it did not work on a MacBook Air with M1. Let me know if I can provide specific information or remote access to it that would help you.

It works on my M1 (MacBook Air also)... @dmitshur, can you try a reboot before executing the adhoc signed binary?

Details

## Build step

GO=$HOME/Development/go/src/github.com/golang/go/bin/go

$GO clean -cache

rm -f hello-nosig hello-apple hello-jpap hello-cherry 
GOOS=darwin GOARCH=arm64 $GO build -o hello hello.go && mv hello hello-nosig
GOOS=darwin GOARCH=arm64 $GO build -o hello hello.go && codesign -s - hello && mv hello hello-apple
GOOS=darwin GOARCH=arm64 $GO build -o hello hello.go && go run ./codesign.go hello && mv hello hello-jpap
GOOS=darwin GOARCH=arm64 $GO build -o hello hello.go && go run ./codesign.new.go hello && mv hello hello-cherry

$ ls -l hello-*
-rwxr-xr-x  1 jpap  staff  2103744 Nov 20 18:59 hello-apple
-rwxr-xr-x  1 jpap  staff  2075302 Nov 20 18:59 hello-cherry
-rwxr-xr-x  1 jpap  staff  2075302 Nov 20 18:59 hello-jpap
-rwxr-xr-x  1 jpap  staff  2059136 Nov 20 18:59 hello-nosig

## After a reboot...

$ file hello-cherry
hello-cherry: Mach-O 64-bit executable arm64
$ file hello-jpap  
hello-jpap: Mach-O 64-bit executable arm64
$ file hello-nosig
hello-nosig: Mach-O 64-bit executable arm64
$ 
$ ./hello-cherry 
hello world
$ ./hello-apple
hello world
$ ./hello-jpap
hello world
$ ./hello-nosig
zsh: killed     ./hello-nosig

[dmitshur]

Thanks for double checking my result @jpap. I will try again in a bit.

[dmitshur]

I tried Cherry’s codesign again at Patch Sets 1 through 3. If I didn’t make a mistake, it seems they all work if I compress the binary and uncompress it, but not before that. The binaries were identical in all 3 cases before and after, so maybe this is related to the inode change?

[cherrymui]

Interesting... Does it work if you simply cp it? And don't try to run the unsigned version first?

If you build a fresh binary, don't try to execute it, sign it, then execute, does it work?

[dmitshur]

If you build a fresh binary, don't try to execute it, sign it, then execute, does it work?

In my previous tests, I’ve always done go build, then sign, then execute. It didn’t work. I may not have cleaned build cache though.

Does it work if you simply cp it?

Yes, it begins to work if I use cp too:

$ ./ps1  # built, then signed by codesign from PS 1
zsh: killed     ./ps1
$ ./ps2  # built, then signed by codesign from PS 2
zsh: killed     ./ps2
$ ./ps3  # built, then signed by codesign from PS 3
zsh: killed     ./ps3
$ cp ps1 ps1c
$ cp ps2 ps2c
$ cp ps3 ps3c
$ ./ps1c
hello
$ ./ps2c
hello
$ ./ps3c
hello

[cherrymui]

PS4 of https://go-review.googlesource.com/c/scratch/+/271866 sets execSeg fields, for what matters. If someone wants to give it a try, that would be great. Thanks!

[cherrymui]

I may not have cleaned build cache though.

Yeah, you may want to clean the build cache, for now, and delete the output binary before running go build. Because signing doesn't change the buildid, the go command may decide you already have the output binary, which is actually not correct.

[dmitshur]

Ok. I tried PS 4 with a modified program:

# modify p.go to be a new program, one I've definitely never built before

$ GOOS=darwin GOARCH=arm64 go build -o ps4 p.go
$ /tmp/cherrysign_ps4 ./ps4  # cherrysign_ps4 is x/scratch/cherry/codesign@d99345806828aac41e55a447c38157e0cdea1c6a
$ ./ps4
zsh: killed     ./ps4
$ cp ps4 ps4c
$ ./ps4c
hello, take 4

[cherrymui]

Thanks for trying out!

Interesting. Maybe we do need to always make a copy... I'll do that in the real thing.

[jpap]

Interesting. Maybe we do need to always make a copy... I'll do that in the real thing.

I'm not convinced you need to make a copy -- so long as you don't run it before signing it. I really doubt the kernel is reading/caching all files marked executable when they appear on the filesystem and before they're executed. :)

I got tripped up on some kind of kernel caching last night -- which is why I kept rebooting to be sure, and had the go cache -clean and rm -f pre-build steps in my test script. Fortunately the M1 boots super-fast! (In hindsight, if we run the different {no-signature, apple codesign, cherry's} tests on different-enough binaries it might've helped...)

@dmitshur, does a reboot clear things up for you?

[dmitshur]

@jpap Yes, rebooting has an effect. I tried building+signing+execing a novel binary to a non-temporary location, it was killed. After rebooting (and doing nothing else), the same binary started to run okay.

However, when I built another novel binary, signed it, and execed, it was still killed.

Edit: Apparently a non-working binary can start to work after some time passes:

$ ~/Desktop/out4
zsh: killed     ~/Desktop/out4
$ ~/Desktop/out4
zsh: killed     ~/Desktop/out4
$ ~/Desktop/out4
hello

(The only difference between 2nd and 3rd runs is that I waited a few mins/used other apps in the meantime.)

[jpap]

I have written a test program to generate a random Go or C "hello X", where X is a random number. The test program builds the random program, adds an adhoc code signature, and then runs it. If you want to try it, make sure you edit generate.go to set goBinPath to where you have the darwin/arm64 Go compiler installed.

On the M1 (without reboots):

• the C program always runs, except if you skip codesign (as you'd expect).
• the Go program only runs when you use Apple's codesign tool.
  
  
  
  • if you link the Go program with Apple's linker, it still won't run: GOOS=darwin GOARCH=arm64 go build -o test -ldflags '-linkmode=external -extldflags "-arch arm64"' ./program.go
  
  
  • before you run the executable test above, and you sign it with Apple's codesign tool, it runs.
  
  
  • it makes no difference if an empty requirement set, and/or empty CMS signature blob is added to the CD.
  
  
  • it makes no difference if you sleep between signing and execution.
  
  
  • it looks like Apple's codesign tool adds a magic hash value to the filename when it sets the CD identifier. If you sign using codesign.go with the same magic hash, it runs! (I was careful to generate an unsigned random program that I did not run: signed it using Apple's tool, noted the hash, did not run it, then signed a different copy using codesign.go (CD ver 0x20001) with the hash appended to the filename-based identifier, then ran the latter signed binary and it ran.)
  
  
  • The identifier-suffix hash appears to be tied to the contents of the executable: it looks like a sha1, but isn't a sha1 of the CDHash, the unsigned file contents, the filename, or the full path to the file (including the filename or not). I didn't see any hints in Apple's open source dump on this one, unfortunately. A random value didn't work either. :)
  
  

I have no idea what's special about the Go vs. C program and why the C program always runs despite not having the hash suffix in the CD identifier.

My guess at this stage is that if you can work out how that hash is computed, you're good to go with a simple version 0x20001 adhoc CD.

[markmentovai]

[fayep]

[markmentovai]

[randall77]

[markmentovai]

[cherrymui]

[fayep]

[jpap]

[markmentovai]

[jpap]

[gopherbot]

[gopherbot]

[gopherbot]

[gopherbot]

[cherrymui]

[cherrymui]

[fayep]

[FiloSottile]

[oxisto]

[gopherbot]

[cherrymui]

[FiloSottile]

[oxisto]

[cherrymui]

[bwesterb]

[fayep]

[dmitshur]

[matloob]

[markmentovai]

[fxcoudert]

[markmentovai]

[jimtut]

[fayep]

[dmitshur]