Elvish: Improve test coverage to at least 80% (measured by codecov) before releasing 0.14

[xiaq]

As of writing, Elvish has an overall test coverage of 73%, measured by codecov.

This issue tracks the effort of improving this to 80% by the release of the next version, 0.14.

[krader1961]

I am impressed that Elvish has 73% test coverage of its code. Related projects, such as the Korn Shell (ksh), have significantly lower coverage. That the leader of the project is serious about the quality of the code is one of the reasons I have switched to this as my day-to-day interactive shell. The other, more significant, reason being that this is not a POSIX shell with all the problems of that standard.

[xiaq]

I know Elvish's test coverage is better than many projects, but honestly, 73% is still embarrassingly low. The entire software industry still has a long way to go in terms of awareness and adherence to best automatic testing practices.

But anyway, @krader1961, I am happy to know that this has played a role in luring you into Elvish :)

[krader1961]

At my $dayjob any change with coverage lower than 80% is generally unacceptable and won't be merged. And 90% coverage or better is considered the minimum desired coverage. That doesn't change the fact that most open source projects have coverage far less than 80% -- and too many do not have any test coverage. So this project's 73% coverage is pretty good in comparison.

What I'm more concerned about are the failures observed when running with race detection enabled. See issues #73, #623, #754, and #888. So while I have no objection to making resolving this issue a pre-requisite for a 1.14 release I think a higher priority is resolving the race failures reported by issue #73. Regardless whether or not that addresses the question of Elvish scripts that concurrently mutate global vars. We really should be running unit tests with race detection enabled as the default condition. Much like running C/C++ programs with ASAN enabled is the gold standard.

[xiaq]

Test coverage now at 80.09% 🎉

Instead of closing this, I will

• Continue to improve the coverage to 80.5% to allow some small fluctuations
• Keep this issue open until 0.14 release, so that further improvements can still reference this issue