Go: x/mobile: gomobile build fails for iOS targets

[dlo]

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

macOS Mojave 10.14.5 (18F132)

go env Output

$ go env

GOARCH="amd64"

GOBIN=""

GOCACHE="/Users/dan/Library/Caches/go-build"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="darwin"

GOOS="darwin"

GOPATH="/Users/dan/go"

GOPROXY=""

GORACE=""

GOROOT="/usr/local/Cellar/go/1.12/libexec"

GOTMPDIR=""

GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"

GCCGO="gccgo"

CC="clang"

CXX="clang++"

CGO_ENABLED="1"

GOMOD=""

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/go-build908091193=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ go get golang.org/x/mobile/cmd/gomobile
$ go get -d golang.org/x/mobile/example/basic
$ gomobile build -bundleid basic.app -target=ios golang.org/x/mobile/example/basic

Note: the wiki instructions leave out the -bundleid parameter even though it appears to be required.

Output

gomobile: go build -tags ios -ldflags=-w -o=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/gomobile-work-749904680/arm golang.org/x/mobile/example/basic failed: exit status 2


golang.org/x/mobile/app

darwin_ios.m:139:77: warning: null passed to a callee that requires a non-null argument [-Wnonnull]

golang.org/x/mobile/example/basic

/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1

ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together

clang: error: linker command failed with exit code 1 (use -v to see invocation

What did you expect to see?

See https://github.com/golang/go/wiki/Mobile#building-and-deploying-to-ios.

The build command will build an application bundle, named basic.app.

What did you see instead?

gomobile: go build -tags ios -ldflags=-w -o=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/gomobile-work-749904680/arm golang.org/x/mobile/example/basic failed: exit status 2
# golang.org/x/mobile/app
darwin_ios.m:139:77: warning: null passed to a callee that requires a non-null argument [-Wnonnull]
# golang.org/x/mobile/example/basic
/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation

[bcmills]

CC @steeve @hyangah

[dlo]

@bcmills Should I go ahead and see if I can figure out what's going on here? Happy to chip in.

[andydotxyz]

I have the same problem on Mojave with XCode 10.2 and 10.3

[andydotxyz]

The warning can be fixed with this patch, but it does not fix the build error...

index 08bb72f..e17b150 100644
--- a/vendor/golang.org/x/mobile/app/darwin_ios.m
+++ b/vendor/golang.org/x/mobile/app/darwin_ios.m
@@ -135,8 +135,9 @@ - (void)touchesCanceled:(NSSet*)touches withEvent:(UIEvent*)event {
 @end

 void runApp(void) {
+       char * argv[] = {};
        @autoreleasepool {
-               UIApplicationMain(0, nil, nil, NSStringFromClass([GoAppAppDelegate class]));
+               UIApplicationMain(0, argv, nil, NSStringFromClass([GoAppAppDelegate class]));
        }
 }

[andydotxyz]

Aha, the problem seems to be that the go tools we are using force -headerpad and that bitcode is now enabled by default and these conflict. The following patch to gomobile fixed it for me 😎.

diff --git a/cmd/gomobile/build_iosapp.go b/cmd/gomobile/build_iosapp.go
index 9cc22ab..79b2c3f 100644
--- a/cmd/gomobile/build_iosapp.go
+++ b/cmd/gomobile/build_iosapp.go
@@ -426,6 +426,7 @@ const projPbxproj = `// !$*UTF8*$!
         SDKROOT = iphoneos;
         TARGETED_DEVICE_FAMILY = "1,2";
         VALIDATE_PRODUCT = YES;
+        ENABLE_BITCODE = NO;
       };
       name = Release;
     };
diff --git a/cmd/gomobile/env.go b/cmd/gomobile/env.go
index cb42d1c..a707965 100644
--- a/cmd/gomobile/env.go
+++ b/cmd/gomobile/env.go
@@ -140,7 +140,6 @@ func envInit() (err error) {
                default:
                        panic(fmt.Errorf("unknown GOARCH: %q", arch))
                }
-               cflags += " -fembed-bitcode"
                if err != nil {
                        return err
                }

[andydotxyz]

I don't know how to create a PR for gomobile, so here is a consolidated patch file instead...

gomobile.diff.txt

[rob-deutsch]

I just ran into the exact same problem. @andydotxyz solution fixed it!

@andydotxyz why did you decide to turn off bitcode instead of turning off headerpad?

I ask because I understand that Apple seems to prefer that bitcode is turned on.

[andydotxyz]

That is a great question @rob-deutsch - it was not my preferred approach either. Unfortunately headerpad is coded into the Go tools which gomobile delegates to (as far as I can tell) - this means that the patch would have to go upstream to the main project which seemed much harder!

[rob-deutsch]

Ha, that's a great reason! Thanks for the info @andydotxyz .

I know its unlikely, but would you happen to remember where exactly headerpad is hardcoded? I'm happy to take this upstream.

[hyangah]

@rob-deutsch I see in ld and some hints in macho_combine_dwarf.go

[andydotxyz]

Sorry I don't recall @rob-deutsch - I just used grep for "headerpad" I think...

[hyangah]

For people who didn't follow the cl -
@eliasnaur commented in the cl.

This CL effectively reverts https://golang.org/cl/168062, potentially re-opening https://golang.org/issue/22395. What changed?

Does anyone have an answer?

[bcmills]

@hyangah, which CL? (I don't see one linked here — it's possible that GopherBot missed a cross-reference.)

[eliasnaur]

@bcmills I believe @hyangah refers to https://golang.org/cl/189857.

[eliasnaur]

I did some digging and found that the answer to my own question is: nothing changed, but gomobile build mode broke with my bitcode-enabling CL168062. Presumably I tested gomobile bind but failed to test gomobile build when preparing that CL.

gomobile build builds an iOS executable, while gomobile bind builds a C archive (using buildmode=c-archive). Building an iOS executable with -fembed-bitcode is not compatible with the flags cmd/link pass to the host linker; I've found -headerpad, -pagezero_size and -fno_pie to be incompatible.

It seems to me the fix is to figure out why cmd/link needs to pass the incompatible flags, and if it doesn't, remove or replace them with bitcode compatible flags. Removing the incomptable flags succeeded in producing a binary but I haven't run it nor tried to upload it to the App Store for verification.

The next best fix is to work around the issue and switch gomobile build to use the buildmode=c-archive like gomobile bind. As a side effect, gomobile build will be much more similar to gomobile bind, decreasing the likelyhood of future breaks.

The easiest workaround is for gomobile build to omit -fembed-bitcode. The downside is that executables built this way will not have bitcode enabled. Since bitcode is not (yet) a requirement for iOS, I don't think missing bitcode matters.

CL 189857 implements the third option, except that it also omits the flag for gomobile bind which is wrong.

[eliasnaur]

I asked on golang-dev for the purpose of the incompatible flags:

https://groups.google.com/forum/#!topic/golang-dev/U1jK3xmmGAk

It seems possible that my first suggestion could work: remove -headerpad, -fno_pie, -pagezero_size and see whether gomobile build programs still work on iOS, and whether the App Store will accept them.

I won't have much time before GoLab Florence, and we're approaching the freeze. If someone else could do the experiment, I'd be very grateful.

[eliasnaur]

With https://github.com/golang/go/issues/34501#issuecomment-542535507 we can now target iOS builders with the TRY= directive.

[gopherbot]

Change https://golang.org/cl/201358 mentions this issue: cmd/link/internal/ld: remove flags incompatible with -fembed-bitcode

[eliasnaur]

https://golang.org/cl/201358 seems to complete all.bash (on at least darwin/arm64). Can someone using gomobile build check whether App Store still accepts binaries with bitcode and CL 201358 applied? Thanks.
@hajimehoshi @andydotxyz @rob-deutsch @dlo

[andydotxyz]

Testing on the latest macOS with gomobile build I get a new error:

ld: -no_pie and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[eliasnaur]

On Sat Oct 26, 2019 at 10:49 AM Andy Williams wrote:

Testing on the latest macOS with gomobile build I get a new error:

ld: -no_pie and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I didn't make it clear, but the CL only works for darwin/arm64, not (yet) for darwin/arm.

Please use

$ gomobile build -target ios/arm64 ...

to build a binary for arm64 only and test with that. You may have to disable arm in your Xcode
as well.

[andydotxyz]

Ah yes, apologies.
It compiles and runs on device - I will attempt an upload to app store shortly.
First I need to figure how to manage different certification profiles using gomobile ;)

[andydotxyz]

OK, I have attempted the upload and it seems to have no bitcode related errors :)

[eliasnaur]

Nice, thanks for checking.

>

[eliasnaur]

@andydotxyz could you be persuaded to check out Cherry's https://golang.org/cl/205060? It should fix the darwin/arm build as well.

[andydotxyz]

I would be happy to but am only at a Mac on Wednesday this week - I’ll do my best to get it checked then :)

[gopherbot]

Change https://golang.org/cl/205060 mentions this issue: cmd/link: enable PIE on darwin/arm

[andydotxyz]

With both patches applied I get:

/Users/andy/Code/Go/goroot/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: warning: OS version (7.0.0) too small, changing to 8.0.0
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[eliasnaur]

Both patches are in go tip, where headerpad is guarded by

      if !ctxt.Arch.InFamily(sys.ARM, sys.ARM64) {
          // -headerpad is incompatible with -fembed-bitcode.
          argv = append(argv, "-Wl,-headerpad,1144")
      }

so -headerpad shouldn't be emitted on darwin/arm nor darwin/arm64. Can you try again with Go tip?

[andydotxyz]

arm and arm64 are both working, however the "ios" target is not - so it's still not building for simulator.

works:

• gomobile build -target ios/arm
• gomobile build -target ios/arm64

does not:

• gomobile build -target ios

[eliasnaur]

Thank you, I had forgotten about the simulators. Let me know whether golang.org/cl/205340 works, which fixes the problem for:

$ gomobile build -target ios -bundleid <...> golang.org/x/mobile/example/flappy

[gopherbot]

Change https://golang.org/cl/205340 mentions this issue: cmd/link/internal/ld: omit bitcode-incompatible flags on iOS simulator

[andydotxyz]

Excellent, thanks - that fixes "-target ios"

$ file myapp.app/main
Mach-O universal binary with 4 architectures: [i386:Mach-O executable i386] [x86_64] [arm_v7] [arm64]
(for architecture i386):    Mach-O executable i386
(for architecture x86_64):  Mach-O 64-bit executable x86_64
(for architecture armv7):   Mach-O executable arm_v7
(for architecture arm64):   Mach-O 64-bit executable arm64

[gopherbot]

Change https://golang.org/cl/206337 mentions this issue: cmd/link/internal/ld: set MachO platform to PLATFORM_IOS on iOS

[andydotxyz]

I guess this can be closed when 1.13.5 comes out?

[eliasnaur]

The fixes will be in Go 1.14 and are not going to be backported. I left this issue open in case someone wanted to work on a gomobile workaround for Go 1.13. If not, go ahead and close it.

[hajimehoshi]

What would the workaround in gomobile be?

[andydotxyz]

Probably something like: https://github.com/fyne-io/mobile/commit/9aa562349ac805f7ffcfa97f808fe2bf198fa75e

[eliasnaur]

Yes, but activated only with Go < 1.14.

[jvburnes]

Hey peeps! I'm trying to use go mobile for an ios target and I ran into the same problem. It seems like the patch submitted above still hasn't been accepted in the master branch. I clone'd from github, applied the patch manually but now I'm trying to figure out how to rebuild go mobile from source. I did see the docs on fetching subrepositories like mobile, but I couldn't figure out how to rebuild from a local copy. Bear in mind I'm relatively new to go, but an experienced dev. Any help greatly appreciated.

[hajimehoshi]

I'm trying to create a CL.

[gopherbot]

Change https://golang.org/cl/214899 mentions this issue: cmd/gomobile: disable bitcode on Go 1.13 or older

[gopherbot]

Change https://golang.org/cl/214957 mentions this issue: cmd/gomobile: enable the test for gomobile-build on iOS