Go: plugin: link error building with plugin package on darwin

[captncraig]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version devel +5d1c601 Tue Nov 29 05:58:06 2016 +0000 darwin/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/craig/gopath"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-build527038733=/tmp/go-build -gno-record-gcc-swi
tches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

Trying to add plugin functionality to https://github.com/mholt/caddy

go get github.com/mholt/caddy

simply adding import _ "plugin" to any go file is sufficient to cause the issue for me.

What did you see instead?

go build github.com/mholt/caddy/caddy

# github.com/mholt/caddy/caddy
/usr/local/go/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
duplicate symbol __expand_key_128 in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-417611788/go.o
duplicate symbol __expand_key_192a in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-417611788/go.o
duplicate symbol __expand_key_192b in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-417611788/go.o
duplicate symbol __expand_key_256a in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-417611788/go.o
duplicate symbol __expand_key_256b in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-417611788/go.o
duplicate symbol _andMask in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-417611788/go.o
duplicate symbol _bswapMask in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-417611788/go.o
duplicate symbol _gcmPoly in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-417611788/go.o
ld: 8 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

some output of go build -x -ldflags "-extldflags -v" github.com/mholt/caddy/caddy:

/usr/local/go/pkg/tool/darwin_amd64/link -o $WORK/github.com/mholt/caddy/caddy/_obj/exe/a.out -L $WORK -L /Users/craig/gopath/pkg/darwin_amd64 -extld=clang -buildmode=exe -buildid=97514994bbed6389d6e33305ba860096
4fc235cc -extldflags -v $WORK/github.com/mholt/caddy/caddy.a
# github.com/mholt/caddy/caddy
/usr/local/go/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
Apple LLVM version 8.0.0 (clang-800.0.38)
Target: x86_64-apple-darwin16.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -dynamic -arch x86_64 -macosx_version_min 10.12.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Plat
forms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -o $WORK/github.com/mholt/caddy/caddy/_obj/exe/a.out -headerpad 1144 -flat_namespace -pagezero_size 4000000 /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go
-link-040389860/go.o /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/000000.o /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/000001.o /var/folders/mb/4rxpmz611m773qkhcgz_rx_
m0000gn/T/go-link-040389860/000002.o /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/000003.o -framework CoreFoundation -framework Security -lpthread -lSystem /Applications/Xcode.app/Contents/D
eveloper/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.0.0/lib/darwin/libclang_rt.osx.a
duplicate symbol __expand_key_128 in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/go.o
duplicate symbol __expand_key_192a in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/go.o
duplicate symbol __expand_key_192b in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/go.o
duplicate symbol __expand_key_256a in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/go.o
duplicate symbol __expand_key_256b in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/go.o
duplicate symbol _andMask in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/go.o
duplicate symbol _bswapMask in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/go.o
duplicate symbol _gcmPoly in:
    /var/folders/mb/4rxpmz611m773qkhcgz_rx_m0000gn/T/go-link-040389860/go.o
ld: 8 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

It may be an environment issue on my end, but a simpler program that imports plugin seems to compile fine.

[crawshaw]

Those particular symbols are duplicates in caddy. They are defined both in a stdlib package and a third-party package:

src/github.com/lucas-clemente/aes12/asm_amd64.s:TEXT _expand_key_128<>(SB),NOSPLIT,$0
go/src/crypto/aes/asm_amd64.s:TEXT _expand_key_128<>(SB),NOSPLIT,$0

Looks like cmd/asm doesn't mangle the names of <> symbols. In its typical build mode cmd/link is fine gets by, but when DynlinkingGo is true something goes wrong.

[bradfitz]

Plugin support was removed from Darwin for Go 1.8. Moving this bug to Go 1.9.

[kris-nova]

Has anyone started looking at this? Maybe @crawshaw?

I would like to help out here and help code in a solution if at all possible, but don't want to duplicate efforts. Thoughts?

[pjebs]

Any news on this?

[bradfitz]

Nobody is working on it. There are no improvements in Go 1.9 and no planned improvements for Go 1.10 unless an owner steps up or somebody starts sending fixes.

[bradfitz]

Actually, the one change in Go 1.9 is ac29f30dbb35d8812c865aed6a6ae738a480def3

[crawshaw]

This has been fixed at HEAD. It appears the duplicate symbols are still in the caddy binary (produced on darwin with import _ "plugin"), but it builds without complaint now:

$ go build github.com/mholt/caddy/caddy
$ objdump -t caddy | grep __expand_key_128
00000000040fd710 l       __TEXT,__text  __expand_key_128
000000000442df40 l       __TEXT,__text  __expand_key_128
$

I'm not sure what CL did this. And whatever the fix is, it's not what I would have done (which is to change cmd/asm to prefix unexported symbols with the package name). But as it's fixed, I'll close this. Please comment if you can reproduce it at HEAD.

[benesch]

I wanted a little more closure, so I went digging.

I'm not sure what CL did this. And whatever the fix is, it's not what I would have done (which is to change cmd/asm to prefix unexported symbols with the package name).

Kind of funny you say this, @crawshaw. Git bisect's pointing the finger at https://github.com/golang/go/commit/96414ca39f4a63d04ddc5cea6d4c63237f2a5fd4 as the commit that fixed the duplicate symbols.