Go: runtime/cgo: macro expansion producing 'defined' has undefined behavior

[Sr0121]

What version of Go are you using (go version)?

$ go version
go version go1.14.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

OS: macOS Catalina 10.15.4

go env Output

$ go env


GO111MODULE=""

GOARCH="amd64"

GOBIN=""

GOCACHE="/Users/sr/Library/Caches/go-build"

GOENV="/Users/sr/Library/Application Support/go/env"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="darwin"

GOINSECURE=""

GONOPROXY=""

GONOSUMDB=""

GOOS="darwin"

GOPATH="/Users/sr/go"

GOPRIVATE=""

GOPROXY="https://proxy.golang.org,direct"

GOROOT="/usr/local/go"

GOSUMDB="sum.golang.org"

GOTMPDIR=""

GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"

GCCGO="gccgo"

AR="ar"

CC="clang"

CXX="clang++"

CGO_ENABLED="1"

GOMOD=""

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/7t/mh0c8z9x6rn72vfl7jxrqh4r0000gn/T/go-build284435102=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Try to run this code:

package main

import (
    "fmt"
    "log"
    "net/http"
)

func main() {
    http.HandleFunc("/", indexHandler)
    log.Fatal(http.ListenAndServe(":9999", nil))
}

// handler echoes r.URL.Path
func indexHandler(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "URL.Path = %q\n", req.URL.Path)
}

What did you expect to see?

Expect the program to run.

What did you see instead?

# runtime/cgo
In file included from gcc_darwin_amd64.c:6:
/usr/local/include/pthread.h:331:6: error: macro expansion producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:2: note: expanded from macro '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
/usr/local/include/pthread.h:331:6: error: macro expansion producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:34: note: expanded from macro '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
/usr/local/include/pthread.h:332:5: error: declaration of built-in function 'pthread_create' requires inclusion of the header <pthread.h> [-Werror,-Wbuiltin-requires-header]
/usr/local/include/pthread.h:540:6: error: macro expansion producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:2: note: expanded from macro '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
/usr/local/include/pthread.h:540:6: error: macro expansion producing 'defined' has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:34: note: expanded from macro     ####'_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'

[ALTree]

This is a dup of #38552 but you provided a reproducer so let's keep this one and I'll close the other thread.

[ianlancetaylor]

What do you see if you create a file foo.c containing

#include <string.h>
#include <pthread.h>

and then run

cc -c -Wall -Werror foo.c

?

[Sr0121]

What do you see if you create a file foo.c containing

#include <string.h>
#include <pthread.h>

and then run

cc -c -Wall -Werror foo.c

?

@ianlancetaylor

In file included from foo.c:2:
/usr/local/include/pthread.h:331:6: error: macro expansion producing 'defined'
      has undefined behavior [-Werror,-Wexpansion-to-defined]
#if !_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT
     ^
/usr/local/include/pthread.h:200:2: note: expanded from macro
      '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
        defined(SWIFT_CLASS_EXTRA) && (!defined(SWIFT_SDK_OVERLAY_PTHREA...
        ^
/usr/local/include/pthread.h:331:6: error: macro expansion producing 'defined'
      has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:34: note: expanded from macro
      '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
        defined(SWIFT_CLASS_EXTRA) && (!defined(SWIFT_SDK_OVERLAY_PTHREA...
                                        ^
/usr/local/include/pthread.h:332:5: error: declaration of built-in function
      'pthread_create' requires inclusion of the header <pthread.h>
      [-Werror,-Wbuiltin-requires-header]
int pthread_create(pthread_t _Nullable * _Nonnull __restrict,
    ^
/usr/local/include/pthread.h:540:6: error: macro expansion producing 'defined'
      has undefined behavior [-Werror,-Wexpansion-to-defined]
#if !_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT
     ^
/usr/local/include/pthread.h:200:2: note: expanded from macro
      '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
        defined(SWIFT_CLASS_EXTRA) && (!defined(SWIFT_SDK_OVERLAY_PTHREA...
        ^
/usr/local/include/pthread.h:540:6: error: macro expansion producing 'defined'
      has undefined behavior [-Werror,-Wexpansion-to-defined]
/usr/local/include/pthread.h:200:34: note: expanded from macro
      '_PTHREAD_SWIFT_IMPORTER_NULLABILITY_COMPAT'
        defined(SWIFT_CLASS_EXTRA) && (!defined(SWIFT_SDK_OVERLAY_PTHREA...
                                        ^

[ianlancetaylor]

Thanks. Seems to me that your C compiler is broken, or at least that it doesn't agree with your library.

Is there some option that we can pass to the compiler to disable this pointless error? Does it work if you run

cc -Wall -Werror -Wno-expansion-to-defined foo.c

?

[Sr0121]

Thanks. Seems to me that your C compiler is broken, or at least that it doesn't agree with your library.

Is there some option that we can pass to the compiler to disable this pointless error? Does it work if you run

cc -Wall -Werror -Wno-expansion-to-defined foo.c

?

A new error has occurred.

In file included from foo.c:2:
/usr/local/include/pthread.h:332:5: error: declaration of built-in function
      'pthread_create' requires inclusion of the header <pthread.h>
      [-Werror,-Wbuiltin-requires-header]
int pthread_create(pthread_t _Nullable * _Nonnull __restrict,
    ^
1 error generated.

[ianlancetaylor]

I really don't understand that one.

That said, why it is including from /usr/local/include? Is that where standard library header files live on Darwin? Or is there another that it should be using instead? What happens if you temporarily remove /usr/local/include/pthread.h and compile foo.c?

[victorneuret]

Hi!
I already encountered an error like that one.
I've had to add CGO_CPPFLAGS=-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined -Wbuiltin-requires-header to my env to make it work.
I think that it's just removing the compilation flags.

My program is the same than yours and I was able to make it work using this.
I'm using MacOs Catalina 10.15.4

[Sr0121]

Thank you all for your help. I finally found out that my Xcode command line tools can not work as expectation. What I did is to run these lines in Terminal.

sudo rm -rf /Library/Developer/CommandLineTools
xcode-select --install

[stephanGarland]

FWIW I had the same issue (different codebase, tried using OP's code and reproduced issue), and reinstalling xcode did NOT fix it.

Installing gcc with brew and then explicitly setting CC and CXX to the symlinked gcc binaries did fix it.

[erikterwiel]

Hi!
I already encountered an error like that one.
I've had to add CGO_CPPFLAGS=-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined -Wbuiltin-requires-header to my env to make it work.
I think that it's just removing the compilation flags.

My program is the same than yours and I was able to make it work using this.
I'm using MacOs Catalina 10.15.4

This works for me except -Wbuiltin-requires-header -> -Wno-builtin-requires-header.
Then, what add to env is: CGO_CPPFLAGS=-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-builtin-requires-header

[codeBehindMe]

So i noticed this issue on MacOs Catalina 10.15.4 as well, out of the blue. I tried the solution that @Sr0121 suggested, didn't work, I also tried the reinstalling gcc as per @stephanGarland's comment, to no avail.

Finally I upgraded 10.15.5 and did the above as well, still didn't change anything. I had to add compiler flags as per @victorneuret .

Kind of annoying how this came out of the blue.
I also tried different go version as well.

[sharpSteff]

@erikterwiel That worked for me as well!
Thank you!

[sharpSteff]

found out.. I recently symlinked gcc headers into usr/local/include to get some other project running.
After removing these symlinks from usr/local/include, everything works fine again

[codeBehindMe]

Yea I think I did something similar to get it working.

[kangkang59812]

Hi!
I already encountered an error like that one.
I've had to add CGO_CPPFLAGS=-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined -Wbuiltin-requires-header to my env to make it work.
I think that it's just removing the compilation flags.
My program is the same than yours and I was able to make it work using this.
I'm using MacOs Catalina 10.15.4

This works for me except -Wbuiltin-requires-header -> -Wno-builtin-requires-header.
Then, what add to env is: CGO_CPPFLAGS=-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-builtin-requires-header

where to add and how?

[kangkang59812]

symlinked

how to do it?

[emil-j-olsson]

@kangkang59812
export CGO_CPPFLAGS="-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-builtin-requires-header"

Then check your environment via go env.

[vietvudanh]

One more problem is that I got permission denied after using the CGO_CPPFLAGS flag:

go build runtime/cgo: copying /Users/hrbc/Library/Caches/go-build/63/63a132c80210b5c7b4f4ed9a902aaec712c42b15e23aa2c1b1778e2f85165d04-d: open /usr/local/go/pkg/darwin_amd64/runtime/cgo.a: permission denied

I need to grant permission to /usr/local/go for it to work. I think this should be handled by the installer?