Go: cmd/go: runtime: epollwait on fd 4 failed with 38 (on ancient CentOS 5.11)

[lukpank]

What version of Go are you using (go version)?

go version go1.10.1 linux/amd64

Does this issue reproduce with the latest release?

Yes (on go1.10.1).

But previous go1.10 works without the issue.

What operating system and processor architecture are you using (go env)?

It is (an ancient) CentOS 5 VM:

$ uname -a
Linux localhost.localdomain 2.6.18-398.el5 #1 SMP Tue Sep 16 20:50:52 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux

From go env (~90% runs crashed with no stdout output, longest output below):

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/username/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"

What did you do?

I run go version/go env (on the Centos5 VM) or go build hello_world.go (on the Centos5 VM or on some real machine running Centos5). Where hello_world.go comes from https://golang.org/ (almost, I used English word "world").

What did you expect to see?

I expect no crash and proper output of go version / go env and successful compilation with go build.

Or (may be) a note in release notes if such old kernel is not supported but even than I would not expect it to change in a point release (from 1.10 to 1.10.1).

Actually for go1.10 I see what was expected (on both the Centos 5 VM and the real machine running Centos 5).

What did you see instead?

On the real CentOS 5 machine go version works fine but go build has similar issue as described below.

On the CentOS 5 VM:

1. go version, 1 of 5 runs displays expected output but almost always crashes as below (with or without proper output).
2. go env, ~90% runs crashes with no output, ~10% display some output and crashes (similar to below).
3. go build crashes (similar to below).

Example crash of go version:

$ go version
runtime: epollwait on fd 4 failed with 38
fatal error: runtime: netpoll failed

runtime stack:
runtime.throw(0x8b4145, 0x17)
        /usr/local/go/src/runtime/panic.go:616 +0x81
runtime.netpoll(0x76ecc806f00, 0x2b2300000000)
        /usr/local/go/src/runtime/netpoll_epoll.go:75 +0x216
runtime.sysmon()
        /usr/local/go/src/runtime/proc.go:4265 +0x424
runtime.mstart1(0x0)
        /usr/local/go/src/runtime/proc.go:1227 +0xe7
runtime.mstart()
        /usr/local/go/src/runtime/proc.go:1193 +0x76

goroutine 1 [chan receive, locked to thread]:
text/template/parse.(*lexer).nextItem(...)
        /usr/local/go/src/text/template/parse/lex.go:195
text/template/parse.(*Tree).peek(...)
        /usr/local/go/src/text/template/parse/parse.go:95
text/template/parse.(*Tree).parse(0xc4200b6100)
        /usr/local/go/src/text/template/parse/parse.go:280 +0xb8f
text/template/parse.(*Tree).Parse(0xc4200b6100, 0x8c94da, 0x8f3, 0x0, 0x0, 0x0, 0x0, 0xc420198510, 0xc420051f00, 0x2, ...)
        /usr/local/go/src/text/template/parse/parse.go:233 +0x226
text/template/parse.Parse(0x8aa77d, 0x4, 0x8c94da, 0x8f3, 0x0, 0x0, 0x0, 0x0, 0xc420051f00, 0x2, ...)
        /usr/local/go/src/text/template/parse/parse.go:55 +0x108
text/template.(*Template).Parse(0xc420171940, 0x8c94da, 0x8f3, 0xc420051ef0, 0x1fad, 0x8a9d7a)
        /usr/local/go/src/text/template/template.go:198 +0x118

goroutine 5 [runnable]:
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:20
created by os/signal.init.0
        /usr/local/go/src/os/signal/signal_unix.go:28 +0x41

goroutine 6 [runnable]:
text/template/parse.(*lexer).run(0xc4201adc70)
        /usr/local/go/src/text/template/parse/lex.go:226
created by text/template/parse.lex
        /usr/local/go/src/text/template/parse/lex.go:221 +0x15f

[kardianos]

From https://github.com/golang/go/wiki/MinimumRequirements

We don't support CentOS 5. The kernel is too old (2.6.18).

[lukpank]

Ok, I see.

But I searched from https://golang.org/doc/go1.10#ports down to https://golang.org/doc/go1.3 and I could not find a note on this decision for linux kernel (but there are many notes on unsupported versions of other OSes). I only found kernel 3.19 mentioned in context of syscall package on https://golang.org/doc/go1.5#minor_library_changes

[mikioh]

IIRC, the decision was made in pre-Go 1 releases; in one of weekly-2012-mm-dd releases without explicit notification.

[ianlancetaylor]

CentOS 5 has not been updated in over 4 years. CentOS 6 was released 6 1/2 years ago. It doesn't seem like an unreasonable decision. We can't support everything.

[darren]

@lukpank

For those who still want to try go1.10.1 on CentOS5, check this commit 1188eb9c5bc67fa7d8dc97f01905e9e364eeb793

errno 38 is:

define ENOSYS 38 /* Function not implemented */

just change epoll_pwait back to epoll_wait will do

diff --git a/src/runtime/sys_linux_amd64.s b/src/runtime/sys_linux_amd64.s
index a5db5cba23..8e945add07 100644
--- a/src/runtime/sys_linux_amd64.s
+++ b/src/runtime/sys_linux_amd64.s
@@ -41,6 +41,7 @@
 #define SYS_sched_getaffinity  204
 #define SYS_epoll_create       213
 #define SYS_exit_group         231
+#define SYS_epoll_wait         232
 #define SYS_epoll_ctl          233
 #define SYS_openat             257
 #define SYS_faccessat          269
@@ -664,8 +665,7 @@ TEXT runtime·epollwait(SB),NOSPLIT,$0
        MOVQ    ev+8(FP), SI
        MOVL    nev+16(FP), DX
        MOVL    timeout+20(FP), R10
-       MOVQ    $0, R8
-       MOVL    $SYS_epoll_pwait, AX
+       MOVL    $SYS_epoll_wait, AX
        SYSCALL
        MOVL    AX, ret+24(FP)
        RET

[OlegKorchagin]

P.S. i have some antient version of telegraf, so i had try it

rhel 5 x86_64:

• old telegraf release ( telegraf 1.6.1 ) works fine
• telegraf 1.8.1 does not work

rhel 5 i686:

• telegraf 1.8.3 works fine

so, if you need telegraf on old rhel5 x86_64, you need old version of telegraf ( ~ 1.6.1 )

Unfortunately, I did not find where you can get archive version of telegraf.
Of couse, I would copy rpm to googledrive, but you obviously can't trust me ;)

[chakri-stark]

I am trying to compile the latest usql(https://github.com/xo/usql) on RedHat 2.6.18-274.el5 but still I am facing the same error after applying the patch that @darren proposed.

[darren]

@chakri-stark

First of all , centos5 is too old, please upgrade.

I have prebuilt go binary for centos5 https://github.com/darren/golang-on-centos5/releases
maybe you can have a try. I built usql with it and a simple run gives no error.

[chakri-stark]

@darren , You are a life saver!! You just unblocked me
BTW, the centos5 is one of my clients server and I had to get this done on this particular machine.
Again, thanks for the binary man. :)

[frjufvjn]

@lukpank

For those who still want to try go1.10.1 on CentOS5, check this commit 1188eb9

errno 38 is:

define ENOSYS 38 /* Function not implemented */

just change epoll_pwait back to epoll_wait will do

diff --git a/src/runtime/sys_linux_amd64.s b/src/runtime/sys_linux_amd64.s
index a5db5cba23..8e945add07 100644
--- a/src/runtime/sys_linux_amd64.s
+++ b/src/runtime/sys_linux_amd64.s
@@ -41,6 +41,7 @@
 #define SYS_sched_getaffinity  204
 #define SYS_epoll_create       213
 #define SYS_exit_group         231
+#define SYS_epoll_wait         232
 #define SYS_epoll_ctl          233
 #define SYS_openat             257
 #define SYS_faccessat          269
@@ -664,8 +665,7 @@ TEXT runtime·epollwait(SB),NOSPLIT,$0
        MOVQ    ev+8(FP), SI
        MOVL    nev+16(FP), DX
        MOVL    timeout+20(FP), R10
-       MOVQ    $0, R8
-       MOVL    $SYS_epoll_pwait, AX
+       MOVL    $SYS_epoll_wait, AX
        SYSCALL
        MOVL    AX, ret+24(FP)
        RET

Very useful. I experienced the same epollwait issue and fixed it after changing the build environment from go1.13 to go1.9. Thank you.