Go: all: remove nacl port

[bradfitz]

We propose to remove the nacl ports (for amd64p32, 386, and arm).

Nacl is deprecated in favor of WebAssembly and @rsc made the point that it's no longer even a great sandbox in light of all the speculative execution problems of late.

The only current user of nacl is the playground (https://play.golang.org/) which we'd need to move to something else. Perhaps gVisor's runsc (https://github.com/google/gvisor#gvisor).

Related: #30324 (for faking time, network)

/cc @randall77 @ianlancetaylor @rsc @aclements @davecheney @josharian @andybons @dmitshur

[dmitshur]

Related issue is #25224.

The only current user of nacl is the playground (https://play.golang.org/) which we'd need to move to something else. Perhaps gVisor's runsc (https://github.com/google/gvisor#gvisor).

To expand on that, another option to consider is WebAssembly.

WebAssembly use for playground details

If using WebAssembly, there exist choices for where the playground snippet code is compiled, and where it's executed. It can be compiled either on the backend server, or in the frontend client (user's browser). The compiled WebAssembly module (application/wasm) can be executed either on the backend sever, or in the frontend client. Some of these options are more difficult to implement than others and have trade-offs.

[martisch]

I think nacl is the only GOOS for GOARCH amd64p32. Can we therefore also remove all support for amd64p32 when removing nacl?

[bradfitz]

@martisch, yes. That's one of the main motivations.

[ianlancetaylor]

I'll note that gccgo uses GOARCH=amd64p32 for x32 mode (https://en.wikipedia.org/wiki/X32_ABI). In general I think we'll want to keep it if we ever choose to do an x32 mode port. Though I don't know how relevant x32 is for Go.

[bradfitz]

@kortschak brought up golang.org/x/tools/cmd/present too.

Currently the present tool has flags to either use play.golang.org or run locally. And when run locally, it has an option for whether to use nacl or not. It defaults to non-nacl, local execution (not using play.golang.org).

Adding gvisor support in present is probably onerous and wouldn't help e.g. people presenting on Mac or Windows laptops.

So, I think present should probably use WebAssembly. Then perhaps we could remove one of the two current flags and then the only choice would be remote play.golang.org vs local WebAssembly.

/cc @neelance @adg

[neelance]

There are two options for WebAssembly: Run the wasm binary on the client (in browser) or on the server (with Node.js or some other compatible wasm runtime).

When running it on the client the main issue is browser compatibility. I think for present it is not really a problem, because you usually can choose which browser to use for your presentation. For the playground however it might not be good to have it fail on incompatible browsers. On #28360 we said that (for now) wasm only needs to support the latest browsers. This requirement might not be suitable for the playground.

Running on the server however might be good for the playground, because we can take care of properly setting up the wasm runtime.

[bradfitz]

For the playground (play.golang.org), we'll run it on the server and not use WebAssembly, because we control the 1 server and we can use gvisor (or most likely: just use Google Cloud Functions, which uses gvisor)

For present, there are many user machines, so running WebAssembly is the best bet, and running it in the client is best so users don't need to worry about having nodejs installed. I don't imagine many people are going to be presenting on Blackberry's browser: https://caniuse.com/#feat=wasm

Re #28360 and browser compatibility, I believe my most recent comment is still what we want to do: https://github.com/golang/go/issues/28360#issuecomment-432853069 ... that is, support the current stable releases of all evergreen browsers by default and only use experimental WebAssembly features behind a compiler-side flag like GOWASM=foo. Alternatively, if we really had to, we could go the other way and make present tell the compiler to do GOWASM=super-portable-at-the-cost-of-speed.

[neelance]

Re browser compatibility: Are we saying different things? I feel like we're saying the same but it sounds like you're disagreeing. Feel free to answer on #28360 so we don't clutter this thread.

[kortschak]

When running it on the client the main issue is browser compatibility. I think for present it is not really a problem, because you usually can choose which browser to use for your presentation.

This is part of the story. I use present for lectures with runnable code examples including code from outside the standard library. The present server is accessible to the students during the lectures and after. I have close to zero control over the machines they use.

[adg]

It would make sense to me to run code snippets using WebAssembly on the backend for both play.golang.org and present. One supported code path, the only external dependency would be some kind of JS runtime. gVisor/runsc means Docker, right? I investigated running short-lived processes on Cloud inside gVisor and it wasn't fast or easy to provision (weird permissions hacks starting Docker containers from inside Docker containers), but maybe I missed something.

[bradfitz]

@adg, App Engine and Google Cloud Functions use gvisor. So all we have to do is deploy a Google Cloud Function that runs the compiler & resulting binary. That part's trivial.

For the present case, though, we don't want a requirement for present users to be that they have nodejs installed. Hence server-side compilation to WebAssembly & client-side execution of WebAssembly.

[adg]

So all we have to do is deploy a Google Cloud Function that runs the compiler & resulting binary. That part's trivial.

Ah so you can deploy Cloud Functions that run code with restricted permissions (no network access)?

we don't want a requirement for present users to be that they have nodejs installed

That's fair. Although I'm curious to see how it goes, shipping megabytes of webasm to the browser on each 'run'.

[bradfitz]

Ah so you can deploy Cloud Functions that run code with restricted permissions (no network access)?

Yeah, I think that between porting nacl's faketime support to linux/amd64 (which @aclements has in progress) and restricting outbound network access via modifications to the net package, we could pretty much just use the resulting linux/amd64 binaries unmodified under gvisor: the real network stack (restricted to localhost) and the real filesystem (I'll have to check whether there are magic auth token files that would cause a problem). Worst case we also port over the nacl fake filesystem behind a build tag.

[bradfitz]

That's fair. Although I'm curious to see how it goes, shipping megabytes of webasm to the browser on each 'run'.

It's 14% smaller as of https://go-review.googlesource.com/c/go/+/167801 and 1.5% smaller with https://go-review.googlesource.com/c/go/+/167937 ... but megabytes isn't terrible terrible.

[neelance]

Gzip compression gives a size of 650 KB for "hello world" with fmt.

[aclements]

Ah so you can deploy Cloud Functions that run code with restricted
permissions (no network access)?

Yeah, I think that between porting nacl's faketime support to linux/amd64
(which @aclements https://github.com/aclements has in progress) and
restricting outbound network access via modifications to the net package,
we could pretty much just use the resulting linux/amd64 binaries unmodified
under gvisor: the real network stack (restricted to localhost) and the real
filesystem (I'll have to check whether there are magic auth token files
that would cause a problem). Worst case we also port over the nacl fake
filesystem behind a build tag.

If it's just a linux/amd64 binary, I worry that it could break out to the
real network too easily. I wouldn't want to rely on language-level
protection like modifying net and disabling "unsafe" alone to ensure
something like this.

>

[aclements]

That's fair. Although I'm curious to see how it goes, shipping megabytes
of webasm to the browser on each 'run'.

It's 14% smaller as of https://go-review.googlesource.com/c/go/+/167801
and 1.5% smaller with https://go-review.googlesource.com/c/go/+/167937
... but megabytes isn't terrible terrible.

A while back we floated the idea of doing modular wasm builds, so the
runtime and std could be shipped as a cacheable wasm module. I assume
nothing has happened on that because I probably would have heard. :)

>

[bradfitz]

Porting nacl's fake network was the original plan so I'm still fine with that too.

I forgot to mention child processes. If we use the Google App Engine/GCF/etc existing gvisor sandbox we'll also need to block child processes. Otherwise you could just write out a binary to do network access and run that. Or we could just use runsc in a VM with nested virt and define our own gvisor sandbox, and then not worry about faking network or disk. I'll try that first.

[ianlancetaylor]

To be conservative, I think we should announce removal of NaCl in the 1.13 release, and actually remove it in the 1.14 release.

[bradfitz]

I tweeted about this here: https://twitter.com/bradfitz/status/1108072470320304128

One potential user from the tweet replies: https://github.com/tinygo-org/tinygo/pull/211#issuecomment-473270115

[andybons]

Per discussion with @golang/proposal-review, 1.13 will be the last release to support it, with it being removed in 1.14.

[aykevl]

One potential user from the tweet replies: tinygo-org/tinygo#211 (comment)

FYI (now also in this issue): we have decided not to use nacl after all, so from our perspective it can be removed.

[dmitshur]

Per discussion with @golang/proposal-review, 1.13 will be the last release to support it, with it being removed in 1.14.

I've filed a release-blocking issue #32948 to document this, so we don't forget.

On another note, I saw on r/golang that @ccbrown has built a prototype of the playground via WebAssembly at https://github.com/ccbrown/go-web-gc.

[ccbrown]

On another note, I saw on r/golang that @ccbrown has built a prototype of the playground via WebAssembly at https://github.com/ccbrown/go-web-gc

My prototype is doing something I don’t think any of the comments here had in mind: Running the compiler in the browser. So that eliminates the several megabyte download on each run that was mentioned above. You just do a one-time, highly cacheable download, and running doesn’t hit the network at all. From a bandwidth perspective I really don’t think it’s too bad at all.

That said, the increased bandwidth usage is still subpar, Chrome is crashy, Safari is slow, and the sandboxing challenge seems solveable without negatively impacting the UX.

[dmitshur]

My prototype is doing something I don’t think any of the comments here had in mind: Running the compiler in the browser.

I mentioned that possibility in the details of https://github.com/golang/go/issues/30439#issuecomment-467991126.

Chrome is crashy, Safari is slow, and the sandboxing challenge seems solveable without negatively impacting the UX.

This is useful to know.

I suspect that at first we'll want to do the compilation and execution on the backend where we control the environment and can be more confident it'll provide a good user experience. When more time passes and WebAssembly clients in all browsers become much more mature, consistent and reliable, we can consider moving it in the frontend then.

[agnivade]

Another cheap win might be to just use gofmt and goimports at the frontend and compile at backend. And having a goal of eventually moving the /compile endpoint to the frontend. Those have a fairly straightforward algorithm and should^TM work nicely in all browsers. Related https://github.com/golang/go/issues/32331.

[Merovius]

I have some concerns about making the playground compile and execute code on the client:

1. How does this interact with the recently acquired ability to use third-party packages on the playground? ISTM that's not easily done with the same-origin policy and it also would increase traffic more, even if the source is proxied through the backend.
2. Running playground examples would mean giving code execution to untrusted third parties. It means we rely on the browser sandbox for safety and security. Given the variety in quality of browser implementations, that doesn't seem a super manageable risk. With server-side execution, the code only runs in one well-defined sandbox.
3. We lose determinism and cross-client caching of results. I would assume that there are going to be differences in the behavior of the WebAssembly runtime of clients, meaning playground examples might produce different results for different clients. Playground code can also be computationally relevant and while it's probably not a huge deal in the great scheme of things, I would prefer not to burn CPU cycles unnecessarily (this latter part is only a small concern though).
4. Even if highly cacheable, I feel that transferring multiple Megabytes for the playground should be prohibitive. Even for my domestic dataplan, 10MB would already be 1% of my monthly budget. Not to speak of roaming, or more expensive data plans in the global south. My experience with web caches is also not super great. For example, even though my blog is exclusively static files and extremely cacheable, cloudflare tells me that they only serve 4% from caches. Now, that's of course also due to misconfiguration probably and it not getting enough traffic to be worth keeping cached. And it doesn't speak to how much of the requests don't even get there, because they are served from caches closer to the clients. But it makes me seriously doubt my intuition about how well web caches work.

I get that it's cool to run the compiler and code in the browser. And I also understand if the Go team doesn't want to carry the computational cost of running it on the backend (though AIUI, the actual cost is pretty minuscule). But from a practical perspective, it doesn't seem like the greatest choice to me.

[gopherbot]

Change https://golang.org/cl/185537 mentions this issue: doc/go1.13: document removal of NaCl targets in Go 1.14

[fjl]

Not sure if this should be its own issue, but it'd be really useful to keep the runtime.faketime hack alive even without NaCl. I've been using it for simulation purposes occasionally and it's a really neat feature to have.

[gopherbot]

Change https://golang.org/cl/192537 mentions this issue: dashboard: remove nacl builders from the master branch

[gopherbot]

Change https://golang.org/cl/192740 mentions this issue: runtime: platform-independent faketime support

[gopherbot]

Change https://golang.org/cl/192738 mentions this issue: runtime: wrap nanotime, walltime, and write

[gopherbot]

Change https://golang.org/cl/192739 mentions this issue: syscall: redirect writes to runtime.write in faketime mode

[gopherbot]

Change https://golang.org/cl/192620 mentions this issue: internal/cpu: simplify x86 by removing NaCl and amd64p32 support

[gopherbot]

Change https://golang.org/cl/195058 mentions this issue: doc/go1.13: start doc; note end of NaCl platform support

[gopherbot]

Change https://golang.org/cl/195983 mentions this issue: sandbox: add gvisor runsc-based sandbox

[fjl]

@aclements, thank you so much for introducing general faketime support.

[aclements]

@fjl, you're very welcome.

[gopherbot]

Change https://golang.org/cl/199499 mentions this issue: all: remove the nacl port

[gopherbot]

Change https://golang.org/cl/199500 mentions this issue: go/analysis/passes/asmdecl: remove amd64p32 support

[bradfitz]

@ianlancetaylor, in the CLs I just mailed out, I just deleted all the GOARCH=amd64p32 support. I don't see how to keep it in any halfway functional form that doesn't bitrot in a short period of time.

Do you think that's okay or do you have some idea of which parts you want kept around?

I suppose one requirement is that cmd/go must still be able to use gccgo to build linux x32 binaries, but I'm not sure how I'd test that. Is that just GOOS=linux GOARCH=amd64p32 go install ... with gccgo somehow?

[ianlancetaylor]

@bradfitz Ah, sorry, I replied on the CL before I saw your comment here.

You're right about testing. If it's not too much of a pain, I suggest that we remove NaCl in two CLs: one that removes NaCl proper, and one that removes the amd64p32 support. The latter is the one we can use as a resurrection guide if we ever decide to add x32 support.

[bradfitz]

I can do two CLs.

Do I need to keep anything in the go tree to not break gccgo's x32 support?

[ianlancetaylor]

No. gccgo supports lots of CPUs that the gc toolchain does not. This will just be one more.

[gopherbot]

Change https://golang.org/cl/200077 mentions this issue: all: remove the nacl port (part 2, amd64p32 + toolchain)

[gopherbot]

Change https://golang.org/cl/200318 mentions this issue: all: remove nacl (part 3, more amd64p32)

[gopherbot]

Change https://golang.org/cl/200739 mentions this issue: runtime: move sighandler into signal_unix.go

[gopherbot]

Change https://golang.org/cl/200765 mentions this issue: dashboard: disable nacl builders for the dev.link branch

[gopherbot]

Change https://golang.org/cl/200941 mentions this issue: all: remove some nacl and amd64p32 stuff

[gopherbot]

Change https://golang.org/cl/201377 mentions this issue: cmd/internal/obj/arm: remove NaCl related DATABUNDLE

[gopherbot]

Change https://golang.org/cl/201380 mentions this issue: cmd/compile: remove amd64p32 related SSA rules

[gopherbot]

Change https://golang.org/cl/202159 mentions this issue: cmd/compile: remove overflow pointer padding for nacl

[gopherbot]

Change https://golang.org/cl/204600 mentions this issue: cmd/compile: remove amd64p32 rules

[bradfitz]

Go 1.14 is out so I'm calling this done: https://golang.org/doc/go1.14#nacl

@toothrot will likely keep improving the gVisor-based playground but other bugs can/do track that.

[gopherbot]

Change https://golang.org/cl/260200 mentions this issue: dashboard, env/linux-x86-nacl: remove nacl builders and supporting files