Go: cmd/cgo: undefined reference for C function with -buildmode=c-archive

[Andari85]

What version of Go are you using (go version)?

go version go1.10.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/Andari/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/Andari/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.10.3/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.10.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lw/5pzww_8n38g1lv_kqqmqfvqr0000gn/T/go-build228325343=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I was experimenting with C interfacing:

// int cadd(int a, int b);
import "C"
...
// using the function somewhere

I built it with go build -buildmode=c-archive GoAdd.go

What did you expect to see?

The build to be successful.

What did you see instead?

Undefined symbols for architecture x86_64:
"_cadd", referenced from:
__cgo_ece3f0761be9_Cfunc_cadd in _x002.o

Why would linking be neccessary, if I am generating an archive which I link together with the cadd implementing object file later?

[bcmills]

CC @ianlancetaylor

[ianlancetaylor]

The cgo tool works this way because it needs to know the type of the C function cadd. It currently determines it by linking the program against the C libraries and examining the debug info. It would be nice if it could simply parse the declaration in the "C" comment, but, well, that requires a C parser. Examining the debug info seems to work reliably enough.

You are doing something rather complicated: you are building a C archive, which is Go code callable from C. And you want that code, written in Go, to itself call a C function. So you have C calling Go calling C. I agree that in principle this ought to work. But in practice it's kind of complicated.

Which is to say: yes, let's fix this if we can. But no promises on when or if anybody will look into it.

[Andari85]

Ok, I understand – I thought, it already parsed the comment abowe the import "C". Note, that I can get it working by compiling-only (clang -c) the C code first and using #cgo LDFLAGS: CAdd.o. The resulted CAdd.o and GoAdd.a can be linked or archived together either with each other or for example a C++ code using extern "C", or an Objective C / Swift code, and possible many other languages. (Actually, my final goal is to write a rendering engine in Go by calling its exported functions from native languages of each platform.)

[Andari85]

Alternative possibility to allow declaring only C function would be something like this:

import "C"

//import cadd
func cadd(a C.int, b C.int) C.int

Like extern "C" in C++ or extern(C) in D.
This way a C parser would be not required.

[ianlancetaylor]

You might be able to do that today using go:linkname. I haven't tried, though.

[Andari85]

I had no success with go:linkname to get the C function imported.

[NickNaso]

Hi everyone,
I have the same problem. I have C calling Go calling C and it should be great have a chance to do that without the linker error.

[Andari85]

Until there is a better option, you can get it working by compiling the C code with -c and use that to compile the Go code and than link together those. For that, you have to have the declarations of the functions exported from the Go source. I did that by hand. I was even lazy enough to simply use int on both sides, which works on most architectures.

I would suggest to create a feature request to be able to use this:

import "C"

//import cadd
func cadd(a C.int, b C.int) C.int

// for use in other packages with Go types
func Cadd(a int, b int) int {
  return int(cadd(C.int(a), C.int(b)))
}

[NickNaso]

Hi @Andari85 ,
I did as you proposed and all worked fine. Thanks for the advice. It could be good have a better option because in my case compiling with the -c option requested a lot of work. Thanks again.

[Andari85]

So, is there a specific way to create a feature request, or will this be picked up by someone?

[ianlancetaylor]

@Andari85 This issue is still open, so it is in effect a feature request. However, as I said above, it's kind of complicated, let's fix it if we can, but no promises on when or if anybody will look into it.

Go is an open source project, so: volunteers welcome. I've already marked the issue as "help wanted".

[Andari85]

Okay, thanks. Well, I'm not much of a compiler guru, one day I might try to learn into the Go compiler.

[ianlancetaylor]

For what it's worth, I don't think that any fix to this will involve the compiler itself at all. I think all the work will be in the go tool and/or the cgo tool.

[zelbrium]

I was able to achieve the requested result (a Go static library which can be invoked by a C main program and can, in turn, call exposed C functions in the main program) by doing the following:

In the Go code, export functions as normal with the //export directive:

//export myGoFunc
func myGoFunc() {
    C.myCFunc()
}

Import cgo, and in the import declarations, declare the external C functions you want to call from Go.

// extern void myCFunc();
import "C"

Then, the important part: the cgo code needs to be compiled with the linker flag -unresolved-symbols=ignore-all (for Linux GCC, on Mac the equivalent flag is -undefined,dynamic_lookup). You could probably do this at the build command somehow, but its more clear to add it to the cgo declaration like this:

// #cgo linux LDFLAGS: -Wl,-unresolved-symbols=ignore-all
// #cgo darwin LDFLAGS: -Wl,-undefined,dynamic_lookup
// extern void myCFunc();
import "C"

These linker flags allow the Go code to successfully compile without having the external C functions defined anywhere, as they get looked up later when the C program they're in gets linked with the Go library.

Then, finally, build the Go static library with -buildmode=c-archive (specifically, my build command is go build -o mylib.a -buildmode=c-archive src.go) and link it to a C program in which the extern function(s) are defined.

So far this seems to work correctly for me, but I'd like to know from Go experts if there is anything wrong with this solution. Will certain features of Go, like goroutines/scheduling or memory management, not work correctly? They appear to be working, but I don't know enough about the technical details of Go to be sure...

[ianlancetaylor]

@zelbrium I don't think there is anything wrong with your solution. There are some oddball cases where it might fail if you want to use internal linking (-ldflags=-linkmode=internal), but that is not the default, and there is no reason for you to use it.

[NickNaso]

Hi everybody,
how the LDFLAGS need to be set for Windows?

[zelbrium]

@NickNaso I don't know for sure -- I'm not very experienced with MSVC -- but perhaps try the /FORCE:UNRESOLVED argument instead of the -unresolved-symbols=ignore-all argument. It appears to do something similar, at least by what the documentation says.

Alternatively, you might be able to use MinGW + GCC on Windows to compile with GCC instead of MSVC. Then the LDFLAGS should be the same as on Linux.

Good luck!

[swq123459]

omg, finally i foud

// export

is wrong
but the follow is rigth

//export

just don't add space after "//"

[zelbrium]

@swq123459 From Go's Hidden Pragmas: "[No space after //] is partly an accident of history, but it also makes it less likely to conflict with a regular comment." But the fact that Go even uses // for pragmas and Cgo directives in the first place is absolutely crazy....