Go: runtime: expose morestack (or equivalent) to make calling into JIT'ed code easier (and less hacky)

[twitchyliquid64]

What version of Go are you using (go version)?

$ go version
go version go1.11.2 linux/amd64

Does this issue reproduce with the latest release?

N/A

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

GOARCH="amd64"

GOBIN=""

GOCACHE="/home/jsonp/.cache/go-build"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOOS="linux"

GOPATH="/home/jsonp/go"

GOPROXY=""

GORACE=""

GOROOT="/usr/local/go"

GOTMPDIR=""

GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"

GCCGO="gccgo"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD=""

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build594946674=/tmp/go-build -gno-record-gcc-switches"

Context

Existing JITs (where the program generates machine code and then executes it on-the-fly) use a bunch of hacks to get it working from Go (without using cgo - the overhead from cgo kinda defeats the purpose of partially JIT'ing code). One of these hacks is ensuring there is enough stack space for the generated code. It would be nice for the runtime package to expose a method to ensure the stack has at least x bytes remaining for the duration of the current calling frame.

(The existing hacks for managing stack size are rather nasty, and they range from trying to allocate unnecessary slices to grow the stack, to searching ELF symbols to locate runtime.morestack).

There are other issues with calling into JITed code, including setting up the stack maps, ABI etc but those hacks are less subtle (and easier to test IMHO).

Feature request

Create a runtime.EnsureMinStack(bytes uint64) method or similar, that would ensure the current frame has at least x bytes spare stack space (growing it if necessary).

[randall77]

The problem with EnsureMinStack is that although it can guarantee that upon return there's some free stack space available, there's no guarantee that that much space will be available later. The runtime can take unused stack space back at any time. (Especially with non-cooperative preemption coming.)

[twitchyliquid64]

I'll have to look into that. Regardless of the specific details, I don't see why we couldn't implement a way to reserve that stack space for the duration of the calling frame (the naive approach would be to mark the calling frame such that unused stack space would not be reclaimed).

As a worst case, this hack could continue to work as long as the compiler doesnt optimize it away or allocate it on the heap (Those are the unspecified behaviours I'm hoping not to be dependent on with EnsureMinStack).

var buf [16 << 10] byte
for i := range buf {
    buf[i] = byte(i)
}

[FiloSottile]

This would enable external FFI (à la rustgo) to take different tradeoffs than cgo by not creating system stacks. Notice how in that blog post I had to lie to the assembler about the frame size of the trampoline.

[twitchyliquid64]

I am researching this and what it would look like ATM. I plan to come back to this issue with a few ideas for moving forward.

[twitchyliquid64]

Some thoughts/questions:

1. Each stack has a buffer of around 720 bytes, called StackGuard, seemingly intended for routines running on top of the goroutines' stack like deferproc. Is it a fair assumption that small, hot JITs (ie: ~100 instructions, 16-80 bytes stack or so) could run in a frame (with no defers), and safely eat into this buffer?

2. It looks like the reflect package is doing some magic with stack frame & gc structures. Could we do a similar thing, and mint a new frame + gc bitvector etc, to trick any sweeps that come along to leave our stack alone (for the duration of the frame)?

Instead of runtime.EnsureMinStack, we could have runtime.CallWithMinStackSize (or even put it in the reflect package).