Go: cmd/dist: no obvious way to disable test caching

[mundaym]

There doesn't appear to be a way to disable the new test caching behaviour when running go tool dist test. This is a bit inconvenient when trying to recreate/resolve intermittent issues.

What version of Go are you using (go version)?

go version devel +7781fed24e Thu Nov 16 02:24:37 2017 +0000 linux/s390x

Does this issue reproduce with the latest release?

No, 1.9.x does not have test caching.

What did you do?

go tool dist test

What did you expect to see?

##### Testing packages.
ok      archive/tar 0.033s
ok      archive/zip 0.749s
ok      bufio   0.126s
ok      bytes   0.934s
ok      compress/bzip2  0.068s
ok      compress/flate  0.833s
ok      compress/gzip   0.010s
ok      compress/lzw    0.020s
ok      compress/zlib   0.100s
ok      container/heap  0.035s
ok      container/list  0.015s
ok      container/ring  0.014s
ok      context 1.008s
...

What did you see instead?

##### Testing packages.
ok      archive/tar (cached)
ok      archive/zip (cached)
ok      bufio   (cached)
ok      bytes   (cached)
ok      compress/bzip2  (cached)
ok      compress/flate  (cached)
ok      compress/gzip   (cached)
ok      compress/lzw    (cached)
ok      compress/zlib   (cached)
ok      container/heap  (cached)
ok      container/list  (cached)
ok      container/ring  (cached)
ok      context (cached)
...

The output of go tool dist test -h is currently:

usage: go tool dist test [options]
  -banner string
        banner prefix; blank means no section banners (default "##### ")
  -compile-only
        compile tests, but don't run them. This is for some builders. Not all dist tests respect this flag, but most do.
  -k    keep going even when error occurred
  -list
        list available tests
  -no-rebuild
        overrides -rebuild (historical dreg)
  -race
        run in race builder mode (different set of tests)
  -rebuild
        rebuild everything first
  -run string
        run only those tests matching the regular expression; empty means to run all. Special exception: if the string begins with '!', the match is inverted.
  -v    verbosity

The -rebuild flag does not affect the caching.

[mundaym]

/cc @rsc

[laboger]

Did you try GOCACHE=off.

[mundaym]

Did you try GOCACHE=off.

Ah I wasn't aware of that. Thanks, yes that does the trick.

If GOCACHE is going to be the general solution to this it might be nice to add to the dist help output.

[rsc]

Why are you running go tool dist test at all instead of go test? Just curious. Normally go tool dist test is basically only for all.bash/run.bash/builders.

[mundaym]

Normally I just use all.bash and go test when I'm testing. I sometimes use go tool dist test when I want to run the cgo and link mode tests without rebuilding everything. I could use run.bash instead, I just use go tool ... out of habit.

In this particular case I was trying to recreate a 'too many open files' failure on the builder and was changing ulimit -n in between runs to see what failed.

I guess the only other reason to use go tool dist test directly rather than run.bash would be to avoid the -rebuild flag. Useful when testing an install of Go without write access.

Since run.bash is just a thin wrapper around go tool dist test -rebuild it does have the same issue. I sometimes (rarely these days) need to run the tests a few times to be confident I've fixed an intermittent issue. In that case I don't want to see cached results.

GOCACHE=off is fine for me. Happy for this issue to be closed if you think this an obscure use case.

[laboger]

I think there needs to be some clarification of the -rebuild option.
Because IMO it does not behave the way it used to, i.e., it doesn't rebuild
everything if the cache is active.

On Tue, Nov 21, 2017 at 4:41 AM, Michael Munday [e-mail]
wrote:

Normally I just use all.bash and go test when I'm testing. I sometimes
use go tool dist test when I want to run the cgo and link mode tests
without rebuilding everything. I could use run.bash instead, I just use go
tool ... out of habit.

In this particular case I was trying to recreate a 'too many open files'
failure on the builder and was changing ulimit -n in between runs to see
what failed.

I guess the only other reason to use go tool dist test directly rather
than run.bash would be to avoid the -rebuild flag. Useful when testing an
install of Go without write access.

Since run.bash is just a thin wrapper around go tool dist test -rebuild
it does have the same issue. I sometimes (rarely these days) need to run
the tests a few times to be confident I've fixed an intermittent issue. In
that case I don't want to see cached results.

GOCACHE=off is fine for me. Happy for this issue to be closed if you
think this an obscure use case.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/22758#issuecomment-345986943, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AI_wjDQpHvqikxA1ptWHYy9aAKYAl3TVks5s4qjpgaJpZM4QgaPN
.

[rsc]

@laboger, I believe that using -rebuild _does_ rebuild everything even when the cache is active. cmd/dist says:

if t.rebuild {
    t.out("Building packages and commands.")
    // Force rebuild the whole toolchain.
    goInstall("go", append([]string{"-a", "-i"}, toolchain...)...)
}

The -a should be forcing a rebuild of everything. If not, that's a bug in -a.

[rsc]

@mundaym, thanks for elaborating. I will send a CL that uses -count=1 on all go test invocations from cmd/dist, so that those results are never cached.

[gopherbot]

Change https://golang.org/cl/80735 mentions this issue: cmd/dist: disable test caching during run.bash

[laboger]

Sorry I misunderstood Michael's comment above where he said "The -rebuild
flag does not affect the caching". I have had no personal experience of the
-rebuild option not working as expected.

On Wed, Nov 29, 2017 at 11:20 AM, Russ Cox [e-mail] wrote:

@laboger https://github.com/laboger, I believe that using -rebuild
does rebuild everything even when the cache is active. cmd/dist says:

if t.rebuild {
t.out("Building packages and commands.")
// Force rebuild the whole toolchain.
goInstall("go", append([]string{"-a", "-i"}, toolchain...)...)
}

The -a should be forcing a rebuild of everything. If not, that's a bug in
-a.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/22758#issuecomment-347931731, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AI_wjJLPfP2FMwBcKI-iBjrQRgCPXj5fks5s7ZJOgaJpZM4QgaPN
.