Terraform: crash on macOS Mojave with go1.14

[maurorappa]

My setup:
System Version: macOS 10.14
go version: go1.14 darwin/amd64
checkout latest HEAD commit: bc7b03e0edfd8b669b542cf89cfb7ad6f34dbf5e

I run go install .
go: downloading github.com/mattn/go-colorable v0.1.1
.....
go: downloading github.com/Azure/go-autorest/autorest/to v0.3.0
all good so far!

I now run:
Mauros-MacBook-Pro:terraform maurorappa$ ../go/bin/terraform
runtime: netpoll: break fd ready for -2
fatal error: runtime: netpoll: break fd ready for something unexpected

runtime stack:
runtime.throw(0x29c482a, 0x39)
/usr/local/Cellar/go/1.14/libexec/src/runtime/panic.go:1112 +0x72
runtime.netpoll(0x176414, 0xc00004c001)
/usr/local/Cellar/go/1.14/libexec/src/runtime/netpoll_kqueue.go:140 +0x30a
runtime.findrunnable(0xc00004e800, 0x0)
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:2323 +0x72b
runtime.schedule()
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:2520 +0x2fc
runtime.park_m(0xc000182600)
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:2690 +0x9d
runtime.mcall(0x80000)
/usr/local/Cellar/go/1.14/libexec/src/runtime/asm_amd64.s:318 +0x5b

goroutine 1 [runnable, locked to thread]:
runtime.doInit(0x3bd8c40)
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5398 +0xd8
runtime.doInit(0x3bd1120)
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5409 +0x57
runtime.doInit(0x3bdacc0)
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5409 +0x57
runtime.doInit(0x3be1060)
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5409 +0x57
runtime.doInit(0x3bf1e40)
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5409 +0x57
runtime.doInit(0x3be7a40)
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:5409 +0x57
runtime.main()
/usr/local/Cellar/go/1.14/libexec/src/runtime/proc.go:190 +0x1ce
runtime.goexit()
/usr/local/Cellar/go/1.14/libexec/src/runtime/asm_amd64.s:1373 +0x1

goroutine 19 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc0001b7220)
/Users/maurorappa/go/pkg/mod/go.opencensus.[email protected]/stats/view/worker.go:154 +0x100
created by go.opencensus.io/stats/view.init.0
/Users/maurorappa/go/pkg/mod/go.opencensus.[email protected]/stats/view/worker.go:32 +0x57

is there anything obvious I missed?
Thanks

[jbardin]

Hi @maurorappa,

Thanks for the notice, though this isn't something we've encountered yet. If this happens on program initialization, it's likely to be a Go problem, and something unique to your system configuration.

Does the same commit work with go1.13? (you may need to modify the go version in go.mod)

Are there any logs output from the binary at all? Is there any output if you run with TF_FORK=0?

If it's a go problem, you may be able to reproduce it by fetching the official go1.14 distribution, and running all.bash which will rebuild and run all builtin tests. If that works, does building terraform with this new toolchain fix the issue?

[Bo98]

We're seeing the same thing over at Homebrew, using 1.12.23. It does not happen on macOS Catalina - only Mojave and High Sierra (we don't test anything earlier than that). I tested locally using both Homebrew's build of Go and Go's official 1.14 binaries and had the same issue with both.

Go 1.13 has no issues, though that's because this error didn't exist in that version.

TF_FORK=0 outputs nothing extra.

Go toolchain builtin test results:

ALL TESTS PASSED

[jbardin]

Thanks for the extra info @Bo98!

This definitely seems like a Go issue, since the binary doesn't even get initialized before crashing.
I'll see if we can find a way to replicate this so we can file an issue.

Note that the 0.12 releases are only supported on go1.12. Only the Terraform master branch has been switched to go1.14, but that is only for development of the 0.13 release as of now.

[Bo98]

I'll see if we can find a way to replicate this so we can file an issue.

That'll be great. I did wonder if it was a Go issue, but with a large codebase like Terraform I wouldn't know where to start to narrow it down to a small reproducible sample.

[maurorappa]

Hi all,
I just tested with GO_VERSION=1.13.8 and the built terraform works, with 1.14 it fails as above.
This time I didn't use Go from Brew but the official (https://storage.googleapis.com/golang/go1.14.darwin-amd64.tar.gz).
I am checking other opensource project if they have similar issues now...

[jbardin]

Upstream issue: https://github.com/golang/go/issues/37726

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.