Vstest: Test Explorer Should Show Test Durations in ms by default

[davkean]

_From @ardalis on January 18, 2017 0:27_

The new format of 0:00:00.000 for test times takes up way too much space for most unit tests which run in under 10ms. See the screenshot below, showing VS2017 above and VS2015 below:

I'd much rather see 3 ms than 0:00:00.003 for test duration, and I'd much rather see more of the name of my test than a bunch of useless 0s.

This is rejected as "by design" in user voice, but I'd like to see this reconsidered. The reasoning behind this change is "consistency with the summary" but I don't see how that's valuable. The format used to display the duration should match the context, not necessarily be a one-size-fits-all decision.

_Copied from original issue: dotnet/roslyn-project-system#1232_

[davkean]

_From @sbaid on January 18, 2017 4:28_

Thanks for feedback. We are looking at it. + @pvlakshm

[brianlagunas]

Please, ms by default. I do not want to see all those zeros cluttering up my screen. Just need to know at a quick glance how long it took to run. Not sure who is making these "by deign" choices, but these choices need to be looked at again. This time with some common sense.

[dotNZ]

Agree with Brian, too many zeros. Please reconsider this or make it configurable.

[bradwilson]

The test list view window should only show the most significant unit of precision (it can include a small fraction as appropriate, f.e. 35ms as well as 4.2s or god forbid 1.3h 😛). A more precise version should be available in the details when selecting the test.

[tvanfosson]

Long test times are a code smell. The display of the time should make it easier, not harder to determine where your tests have issues. I agree with @bradwilson only the most significant digit (optional fraction) plus the units should display. I might even go further and say that the style (color, warning icon (!)) might change if a test is an order of magnitude different than related tests.

[thuru]

Guys, as devs what we do in situations like this? Just give an option in the Tools to specify the format of the time. Let the user decide.

[SvenAelterman]

@tvanfosson Long test times may be a code smell in some scenarios, like unit tests. However, in integration tests, they may be perfectly normal.

[ardalis]

Right, it's also worth remembering the same Test Explorer is used to display results from Load and UI tests, not just unit tests (that should run in ms, not seconds or minutes). However, the current VS2015 behavior that adapts to the appropriate unit is ideal and shouldn't be changed, at least not as the default, IMO.

[tvanfosson]

@SvenAelterman absolutely, that's why it would be relative to other tests and only highlighting. That is to say that generally your tests should be within an order of magnitude BY TYPE. It's not perfect but it beats getting a wall of numbers that make it difficult to spot outliers.

[SvenAelterman]

Adding my $0.00000002 million, there are too many zeroes. If there is a detail view that provides it or some stats you could analyze by downloading test results or an API, that should be in a fixed format. However, for a UI which is meant to be read by humans, using the VS 2015 approach is the way to go.

[SvenAelterman]

@tvanfosson So why not have that highlighting capability regardless of the elapsed time display format? It seems we're confounding two different issues here. I could even get behind being able to sort the list by test duration.

[tvanfosson]

@SvenAelterman sure, but still don't need the wall of numbers that makes it more difficult to read

[SvenAelterman]

@tvanfosson So what are you asking for? The original premise of the issue that seeing a bunch of unnecessary leading 0's creates a wall of numbers that takes up more space and hides more of the name of the test... Seems to me not seeing as much of your test's name makes it more difficult.

[tvanfosson]

@SvenAelterman Make it red and add a ! with tooltip (for accessibility) to the end when the run time differs by an order of magnitude. Not talking about a lot of real estate here.

[SvenAelterman]

@tvanfosson Right, but keep it displayed as "3 ms" vs "00:00:00.003" then?

[tvanfosson]

@SvenAelterman correct

[SvenAelterman]

@tvanfosson I like it when we get on the same page! I re-read your original comment and get your drift completely now!

[bsherwin]

If my tests are designed to running in sub-second values, then having 3-5 leading zeros for hours, minutes and seconds means there is little value in presenting the data. Having them presented as "3 ms" becomes much more meaningful. If I find I have a test that runs in "1530 ms"...I don't care that it is 1.5 seconds...I care to see that as an order of magnitude of 500 times the average.

Please make it configurable...and if that's too hard...leave it as milliseconds.

[dibley1973]

3ms would be preferable. Hard enough to detect test names as it is as often many start with same name.

[bradwilson]

@thuru wrote:

Guys, as devs what we do in situations like this? Just give an option in the Tools to specify the format of the time. Let the user decide.

Worst. Possible. Answer. Please do not abdicate the responsibility for good UX behind "a user option".

[thuru]

@bradwilson : i'm not sure where you get the ideas for the UX, Learning and adapting the design is one aspect. Based on the user option team can decide which is the most used option.

Also, just arguing whether to go for shorter form or longer format is pointless, as this can be tackled in many ways. As you suggested if one has to select a test to see the full format with exact precision this is going to kill the user experience where there are many tests.

This can also be managed by the width of the Test Explorer, the display format changes accordingly when the user adjusts the width, this would be another option.

[FranksBrain]

I agree with the original post - keep it the way it is in VS2015. Worst case, make a little clickable link that can show "shoretened" or "expanded" times in the test runner. Or just click the times themselves to toggle back and forth if it's that important to have the expanded version.

[sbaid]

Thanks for the feedback. We are looking at bringing back vs 2015 style on this. will revert.

[pvlakshm]

We are fixing this such that the time format will be the way it was in VS 2015.
I have also reactivated the uservoice item, and marked its status as "Started".
Thank you for the feedback.

[sbaid]

It will be part of the upcoming RC update.

[ardalis]

Awesome, thank you!