Sdk: code coverage seems not to work with content_shell

[DartBot]

_This issue was originally filed by @zoechi_

---

dart bin/collect_coverage.dart --port=9999 -o coverage.json
content_shell --dump-render-tree --remote-debugging-report=9999 test/test.html

results in this output:

    {"type":"CodeCoverage","coverage":[]}

Dart VM version: 1.6.0-edge.38759 (Wed Jul 30 16:01:16 2014) on "linux_x64"

see also https://groups.google.com/a/dartlang.org/forum/#!topic/misc/eScY-0qLSic

[johnmccutchan]

Hi,

We cannot reproduce locally. Can you share a reproduction with us?

Thanks,
John

[johnmccutchan]

_Set owner to @johnmccutchan._
_Added Area-Observatory, Triaged labels._

[iposva-google]

_Added Accepted label._

[DartBot]

_This comment was originally written by @zoechi_

---

It's accepted, does that mean you can reproduce it?

[DartBot]

_This comment was originally written by [email protected]_

---

I have a reproduction now. Basically it boils down to the order in which you start your scripts. While the docs say that you can start the collect script first this did not work for me.

Setup (latest -dev as of yesterday):
Dart SDK version 1.6.0-dev.6.0
content shell: drt-lucid64-full-dev-38621.0

Working:
./content_shell --remote-debugging-port=9999 index.html
dart bin/collect_coverage.dart --port=9999

Not working
dart bin/collect_coverage.dart --port=9999
./content_shell --remote-debugging-port=9999 index.html

...
{"type":"CodeCoverage","coverage":[]}
...

---

When started before the content shell, the collect script receives "{type: ServiceException, id: , message: Dart not running}" internally as soon as content shell is up.

---

Attachment:
index.html (130 Bytes)

[DartBot]

_This comment was originally written by @zoechi_

---

I tried it with content_shell starting before collect_coverage but then content_shell ended before I could start collect_coverage. I'll try again.

[johnmccutchan]

Hi,

This is starting to sound like a bug in collect_coverage and not the VM.

John

[DartBot]

_This comment was originally written by @zoechi_

---

The problem is --dump-render-tree.
It works without this argument but when --dump-render-tree is used the content_shell just ends and there is no chance to collect coverage information.

[nicolasgarnier]

1. Lots of people will use content_shell to run tests to run (unit-)tests and typically this means running content shell with --dump-render-tree which makes content_shell exit when the page finished loading and dump the content fo the page. This allows to get the result of web unit tests easily and headlessly.

I'm writing a test runner and I'm impacted by this issue. Basically I can't gather code coverage data for browser tests because content_shell exited before we could gather the code coverage from the observatory.

I think this is an important issue because one of code coverage's main purpose is to be ran while executing all tests and most (all) test runners will use content_shell in a headless mode which means using --dump-render-tree.

What we need is an option like --pause_isolates_on_exit in content_shell that exists today on dart standalone VM. That way we should be able to gather code coverage and resume all isolates which will cause the page to finish loading and which will cause content_shell --dump-render-tree to dump the page content and exit only afterwards.

+Kevin: We need this to build a nice test runner which would output test coverage :)

---

cc @kevmoo.

[nicolasgarnier]

_Removed Area-Observatory label._
_Added Area-Dartium label._

[elmcrest]

we'd like to get test coverage, too.
any updates on this?