Go: cmd/go: add summary at end of go test results

[markvincentcaro]

Currently, there is no way to quickly tell if a test failure occurred after running go test.

You have to scroll up and look for the word FAIL or search for it in a text editor.

Printing a summary of the test results at the bottom (e.g. Tests run: 10, Failures: 2, Errors: 0, Skipped: 0) would make it easy to tell at a glance if any test failures occurred or not.

An answer to Issue https://github.com/golang/go/issues/27755 suggested using the -json flag, but I haven't seen any good documentation or examples on how to use it. Also, this approach seems too convoluted for something that other languages have for tests out of the box.

_EDIT:_ I'm on Windows 10.

[mvdan]

Currently, there is no way to quickly tell if a test failure occurred after running go test

The very last line should tell you. Even if you're testing many packages, each package that succeeded should only output a single line, so I don't see how this would be a problem unless you were testing a thousand packages at once.

Or, if you simply want to tell if any test failed, just check go test's exit code.

[markvincentcaro]

@mvdan

Even if you're testing many packages, each package that succeeded should only output a single line, so I don't see how this would be a problem unless you were testing a thousand packages at once.

Actually, the fact that you need to manually check if any package failed can be subject to human errors, however rare they may be. And you shouldn't be discouraged from decoupling your code into organized packages and writing tests for them just because you want to keep your package count low to make test verification easier. The compiler's tools should be doing that for you.

Or, if you simply want to tell if any test failed, just check go test's exit code.

How exactly do you do this?

[ianlancetaylor]

Or, if you simply want to tell if any test failed, just check go test's exit code.

How exactly do you do this?

What system are you running on? GNU/Linux? Windows?

[ianlancetaylor]

Programmatically you check the exit code by checking whether os/exec's Run method return an error or not.

[mvdan]

On unix-y shells, you can do go test whatever-packages; echo $? to see the last command's exit status. I'm sure Windows and all other systems will have similar mechanisms.

[markvincentcaro]

@mvdan
What info do you need? If it's my system, I added it in the description.

[mvdan]

Well, @ianlancetaylor asked a question, and we both showed that it's generally simple to check an exit status code. You were asking how to quickly tell if a go test run had any failures - that's the simplest way to check. We're waiting for info because your entire proposal seems to be based around that problem.

[markvincentcaro]

@mvdan

The answer from this SO thread (https://stackoverflow.com/a/334890/7824245) says the errorlevel environment variable is how to check it on Windows, but to be safe, it also needs set errorlevel= to clear any previous values.

I'll try it out and see how that works.

[markvincentcaro]

@mvdan @ianlancetaylor

Nevermind, I found a better way to quickly detect test failures from this SO thread:

go test ./... -v -short > test-results.txt & findstr "FAIL _test" test-results.txt

Though I still kinda wish that the Go team would officially add this feature in the future.

[rsc]

Note that #27755 was about a per-package summary. This issue is about a per-run summary over many packages. In general I'm skeptical of summaries: totaling numbers across tests is misleading in various ways, and people won't agree about the level of detail. But I do sympathize with seeing the end of output and not even knowing whether the overall list succeeded.

I propose that in 'package list mode' (see 'go help test' for 'local directory mode' vs 'package list mode'), the overall 'go test' could end by printing FAIL when there are failures (but stay silent for all "ok"/"skip").

[markvincentcaro]

@rsc
Now _this_ I can get behind. It would also be very helpful to put all failed tests, the specific line number, and t.Error messages at the bottom so we don't have to hunt for them at the top.

[andybons]

Per discussion with @golang/proposal-review, let's move forward with @rsc's idea.

[gopherbot]

Change https://golang.org/cl/170948 mentions this issue: cmd/go: print finally FAIL if a test has failed in package list mode

[manikesh]

what is the resolution to this issue?? do we have any command which gives result in asked manner..

[ianlancetaylor]

This issue is not yet resolved.

[marco-m]

@markvincentcaro check out https://github.com/gotestyourself/gotestsum