Rules_nodejs: Demonstrate how to gather code coverage for web tests

[alexeagle]

Should serve to answer https://stackoverflow.com/questions/56556944/how-to-measure-coverage-when-running-typescript-tests-using-karma-and-bazel and also probably will reproduce the issues described there.

[iirina]

Unfortunately there are no updates in bazel coverage land and there is no future work planned at this time. For this particular use case I can recommend using aspects to collect the transitive source files. An aspect can be attached to karma_web_test_suite. The aspect can collect sources and propagate through all dependencies attributes. There is an advance example which uses an aspect to count the files in the transitive path. Let me know if this is reasonable or if there are further questions.

[alexeagle]

Thanks for replying!
Sure, we already have an aspect here that gathers the source files, and we know how to instrument the app for coverage. The problem is we have no way to report the result, other than write something to TEST_UNDECLARED_OUTPUTS, and as you know that means you can't get coverage data spanning targets/languages.

But if bazel coverage is destaffed then I guess that answers the question and we'll have to write our own coverage tooling that presents the results...

[iirina]

Thanks @alexeagle. The test runner creates COVERAGE_OUTPUT_FILE as an output for coverage invocations and COVERAGE_DIR for coverage generated files, but the variables are not exposed to Skylark. We can look into what it takes to expose them.

[Toxicable]

We'd also like to apply the same coverage stratergy for nodejs test coverage, only it uses a slightly different instrumentation method, but transforms are easy.

@iirina I don't know if we need those variables exposed to skylark, although im sure we'll need them there eventually, but just having them avilable to the program being instrumented I think would get us there.
In my testing process.env.COVERAGE_DIR (in a JS program) was never defiend when running under bazel coverage - is there some flag I need to enable?

If you could take a quick look over: https://github.com/bazelbuild/rules_nodejs/pull/1135/files
It's roughly how we'd collect coverage in nodejs and also has a script for processing, merging and writing reports

[sgammon]

@alexeagle / @iirina with the release of bazel 2.1.0 and the associated coverage fixes, could i give this issue a bump?

just like the poster on SO says, i've got it to a point where i can run coverage and generate lcov reports. i just need to figure out where to put the report so i can trigger it being merged with the combined report

[Toxicable]

@sgammon I've integrated bazel coverage into rules_nodejs and @bazel/jasmine in #1135
Take a look there, specifically at coverage_runner.js on where and how to handle the coverage data.

[sgammon]

@Toxicable thank you!! i'll take a look

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs in two weeks. Collaborators can add a "cleanup" or "need: discussion" label to keep it open indefinitely. Thanks for your contributions to rules_nodejs!

[github-actions[bot]]

This issue was automatically closed because it went two weeks without a reply since it was labeled "Can Close?"