Chrome-remote-interface: Code coverage

Created on 19 Jun 2017 · 6Comments · Source: cyrus-and/chrome-remote-interface

Hi devs, maybe you can bring some light to this.

Is it possible to get code coverage as featured in chrome 59 in an understandable way (reporting lines used, unused, percentage)?

I tried to use the Profiler methods but I get function offsets(lines?) and counts that does not match if I use the coverage feature manually in Chrome.

I have some tests running in localhost:3000. The execution takes 10 seconds (I have a few), so is there a way to extend the recording of the coverage to 10 secs? using setTimeout?.

I'm lost , I'm not grasping this :(

My code:

const CDP = require('chrome-remote-interface');

CDP((client) => {
  // extract domains
  const {
    Network,
    Page,
    Profiler
  } = client;
  // setup handlers
  Network.requestWillBeSent((params) => {
    console.log(params.request.url);
  });

  // enable events then start!
  Promise.all([
      Network.enable(),
      Page.enable(),
      Profiler.enable(),
      Profiler.startPreciseCoverage()
    ])
    .then(() => {
      return Page.navigate({
        url: 'http://localhost:3000'
      });
    })
    .then(() => Profiler.getBestEffortCoverage())
    .then((data) => data.result.forEach((el) => {
      console.log('-----', el.url)
      el.functions.forEach((el) => {
        console.log('--', el.functionName)
        console.log('ranges', el.ranges)
      })
    }))
    .catch((err) => {
      console.error(err);
      client.close();
    });
}).on('error', (err) => {
  // cannot connect to the remote endpoint
  console.error(err);
});

some output

-- myFunction
ranges [ { startOffset: 907, endOffset: 1490, count: 1 } ] // But it is a 3 line function and it is not executed!!

All the best.

protocol question

Source

sergueyarellano

All 6 comments

Have you taken a look at automated-chrome-profiling, it should address all your profiling-related questions with the CDP.

Anyway, I think that the proper use of the Profiler is:

Profiler.startPreciseCoverage();
run your code;
Profiler.takePreciseCoverage();
Profiler.stopPreciseCoverage().

Which basically translates to:

function timeout(msecs) {
    return new Promise((fulfill, reject) => {
        setTimeout(fulfill, msecs);
    });
}

await Profiler.startPreciseCoverage();
await Page.navigate({url: 'http://localhost:3000'});
await Page.loadEventFired();
await timeout(10000);
const {result} = await Profiler.stopPreciseCoverage();

You can use something smarter instead of the timeout, e.g., arrange your tests in a way that a promise is resolved when they're done then use something like this instead of the timeout:

await Runtime.evaluate({
    awaitPromise: true,
    expression: 'startTest()'
});

Now about getting code coverage in an understandable way there's a nice trick which always comes in handy when one wants to figure out how DevTools uses the CDP to implement some of its features. It's described here under "Sniffing the protocol". If you try that during a code coverage audit you'll see something like:

{"id":174,"method":"Profiler.startPreciseCoverage"}
{"id":174,"result":{}}
{"id":176,"method":"Profiler.takePreciseCoverage"}
{"id":177,"method":"Profiler.stopPreciseCoverage"}
{"id":176,"result":{"result":[{"scriptId":"325","url":"file:///path/to/issue-170.html","functions":[{"functionName":"foo","ranges":[{"startOffset":2,"endOffset":45,"count":1}]},{"functionName":"bar","ranges":[{"startOffset":47,"endOffset":156,"count":1}]}]}]}}
{"id":178,"method":"Debugger.getScriptSource","params":{"scriptId":"325"}}
{"id":177,"result":{}}
{"id":178,"result":{"scriptSource":"\n\nfunction foo(x) {\n    return Math.sin(x);\n}\n\nfunction bar() {\n    let x = 0.1;\n    for (i = 0; i \u003C 10000; i++) {\n        x = foo(x);\n    }\n    return x;\n}\n\nsetInterval(bar, 1000);\n\n"}}

And that's it, I'm afraid that you need to compute lines numbers and percentages by hand as there's no CDP method that will aid you in doing that. It should't be too hard though.

Hope this helps!

cyrus-and on 19 Jun 2017

Thanks for the response @cyrus-and I will take a look at this ;)

sergueyarellano on 22 Jun 2017

Hey @cyrus-and, thanks again, now I'm getting results, but I still do not understand the meaning of startOffset and endOffset.

Now I'm going to bother you with more questions!

In your foobar example you get this:

[{"functionName":"foo","ranges":[{"startOffset":2,"endOffset":45,"count":1}]},{"functionName":"bar","ranges":[{"startOffset":47,"endOffset":156,"count":1}]}]

Has foo 45-2 (43) lines? (I don't think so XD)
Has bar 156-47 (109) lines?
What is representing count here? (total calls? all the range is executed?)
What are the total lines of the script?

Cheers my friend.

sergueyarellano on 23 Jun 2017

The scriptSource is:

"\n\nfunction foo(x) {\n    return Math.sin(x);\n}\n\nfunction bar() {\n    let x = 0.1;\n    for (i = 0; i \u003C 10000; i++) {\n        x = foo(x);\n    }\n    return x;\n}\n\nsetInterval(bar, 1000);\n\n"

The offsets are relative to this string, so for example:

> foo = scriptSource.slice(2, 45)
'function foo(x) {\n    return Math.sin(x);\n}'
> console.log(foo)
function foo(x) {
    return Math.sin(x);
}

You can roughly get the number of lines of the script (or of any range or function) with:

> scriptSource.split('\n').length
17

For what concerns count I'm a bit puzzled, it seems to be always 1 even though the function foo is called 10000 times per second... The documentation says: