Go: x/build: add playground to build.golang.org

Created on 12 Apr 2018 · 10Comments · Source: golang/go

Currently running trybots for x/playground produces this:
https://storage.googleapis.com/go-build-log/2d26a65f/linux-amd64_e35969e6.log

Add Playground to build.golang.org.
Make builders be able to test Playground in the its docker container.

Builders NeedsInvestigation Testing

Source

ysmolsky

Most helpful comment

Rough outline, though:

make a new misc-playground builder (similar to misc-compile-*).
make a new host type in x/build/dashboard/builders.go etc that's VM-based, not container-based. Probably can resurrect docker2boot, which we used to use before Kubernetes. (we don't want to use Kubernetes here because docker build on Kubernetes remains a mess: https://github.com/kubernetes/kubernetes/issues/1806 etc)
make a "go test" friendly test that runs exec.Command("docker", "build") etc if os.Getenv("GO_BUILDER_NAME") == "misc-playground"
add some special cases for which tests to run in dashboard/builders.go and cmd/coordinator, like some of the others clarifying which repos run on which builders.

bradfitz on 13 Apr 2018

👍4

All 10 comments

You should also add it to build.golang.org :smile:

andybons on 12 Apr 2018

🎉1 👍1

@andybons, is there any good doc describing how trybots work besides https://go.googlesource.com/build/ ?

ysmolsky on 12 Apr 2018

Good question. @bradfitz ?

andybons on 12 Apr 2018

Nope, not besides reading the coordinator code. But this is not a good starter project.

bradfitz on 12 Apr 2018

Something is not clear. We have to test Playground via it's docker container. But where should we run that? I think it does not make much sense to run it on many platforms. Also I fail to find repos tests in similar fashion, but running not a go test ./... but rather docker container which will take more than 10 minutes to build from scratch. I will dig further a bit later...

ysmolsky on 13 Apr 2018

Yes, we have no way to do this right now. That's why it's not a good starter project. This will be a lot of work and require a number of changes across our build system.

bradfitz on 13 Apr 2018

Rough outline, though:

make a new misc-playground builder (similar to misc-compile-*).
make a new host type in x/build/dashboard/builders.go etc that's VM-based, not container-based. Probably can resurrect docker2boot, which we used to use before Kubernetes. (we don't want to use Kubernetes here because docker build on Kubernetes remains a mess: https://github.com/kubernetes/kubernetes/issues/1806 etc)
make a "go test" friendly test that runs exec.Command("docker", "build") etc if os.Getenv("GO_BUILDER_NAME") == "misc-playground"
add some special cases for which tests to run in dashboard/builders.go and cmd/coordinator, like some of the others clarifying which repos run on which builders.

bradfitz on 13 Apr 2018

👍4

We just open sourced a tool today to make docker build on kubernetes less of a mess: github.com/GoogleCloudPlatform/kaniko

dlorenc on 16 Apr 2018

👍1

Resolving this issue should be easier now compared to 2018 for at least one reason: thanks to the module mirror, additional non-golang.org/x dependencies will no longer be a problem for the builders/trybots to fetch.

I'm not if something Docker-related also needs to be done, but @toothrot may have more context.

/cc @toothrot @findleyr

dmitshur on 20 Apr 2020

The integration tests changed in #25224 to require two docker containers running in a docker network, and also require the ability to run runsc, which requires root. I have a few ideas on how we can get good coverage, but in the current state the TryBots and the Playground's test setup will require a little work.