Istanbul: Instrument ES6 generators for coverage

[Raynos]

When using istanbul on ES6 code it prints a bunch of errors

Transformation error; return original code
{ [Error: Line 7: Unexpected token *] index: 174, lineNumber: 7, column: 9 }
Transformation error; return original code
{ [Error: Line 11: Unexpected token *] index: 266, lineNumber: 11, column: 9 }
Transformation error; return original code
{ [Error: Line 11: Unexpected token *] index: 323, lineNumber: 11, column: 9 }
Transformation error; return original code
{ [Error: Line 10: Unexpected token *] index: 241, lineNumber: 10, column: 9 }
Transformation error; return original code
{ [Error: Line 9: Unexpected token *] index: 206, lineNumber: 9, column: 9 }
Transformation error; return original code
{ [Error: Line 8: Unexpected token *] index: 223, lineNumber: 8, column: 9 }
Transformation error; return original code
{ [Error: Line 12: Unexpected token *] index: 280, lineNumber: 12, column: 9 }
Transformation error; return original code
{ [Error: Line 8: Unexpected token *] index: 174, lineNumber: 8, column: 9 }
Transformation error; return original code
{ [Error: Line 9: Unexpected token *] index: 194, lineNumber: 9, column: 9 }
Transformation error; return original code
{ [Error: Line 11: Unexpected token *] index: 260, lineNumber: 11, column: 13 }
Transformation error; return original code
{ [Error: Line 12: Unexpected token *] index: 302, lineNumber: 12, column: 13 }

The result is that every piece of code I have that uses generators is not instrumented.

[Raynos]

For reference I have a build of the tests that is running with istanbul on travis and the following project can be used to confirm the issue

I've configured that project to produce coverage reports using npm run cover

I've also uploaded the resulting coverage report to coveralls. This file does not show coverage for files in my repo & routes folder which are files that include ES6 generators syntax.

[gotwarlost]

Looks like those are esprima parse errors. The reason most probably is that istanbul depends on esprima 1.0.x and you probably need the 1.1 series of esprima to parse the JS correctly.

https://github.com/gotwarlost/istanbul/blob/master/package.json#L34

Could you try with esprima 1.1.x instead and see what happens? The instrumentation may still not be correct but at least the parsing phase should work.

[gotwarlost]

Hmmm,

$ npm info esprima versions

[ '0.7.0',
  '0.8.0',
  '0.8.1',
  '0.8.2',
  '0.9.0',
  '0.9.1',
  '0.9.2',
  '0.9.3',
  '0.9.4',
  '0.9.5',
  '0.9.6',
  '0.9.7',
  '0.9.8',
  '0.9.9',
  '1.0.0',
  '1.0.1',
  '1.0.2',
  '1.0.3' ]

I need the guru's help to understand the level of generator support in esprima.

/ping @ariya

Ariya, is there ES6 generator support in esprima?

[Raynos]

@gotwarlost I have used generators parsing with esprima by depending on ariya/esprima#harmony. i.e. there is a harmony git branch that will parse them. I can try changing to that and seeing what istanbul does.

[Raynos]

Running the same code with the ES6 version of esprima gives

Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
{ [Error: Line 13: Illegal yield expression]
  index: 347,
  lineNumber: 13,
  column: 50,
  description: 'Illegal yield expression' }
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]
Transformation error; return original code
[TypeError: Cannot read property 'children' of undefined]

I suspect that the ast interface has changed in esprima.

[ariya]

What makes you think this is related to AST interface change?

I assume Istanbul's instrumenter needs to be modified to understand those new syntax node. In addition, escodegen also need to recognize it.

You can't just feed a foreign node into such a chain of tools and expect everything will still work.

[Raynos]

@ariya you are correct.

The esprima interface didn't change. Istanbul has not been configured YieldExpression in it's ast tree walker implementation and thus does not know how to handle it.

[gotwarlost]

I suppose it's now time to create a harmony branch for istanbul :)

[meritt]

ping

[kornelski]

I've added yield. It was pretty simple actually, except I'm not sure what's the best way to make test for it ES6-only.

[meritt]

Nice work! Could you create PR for this?

[kornelski]

@meritt I'm not sure it's ready yet, but there you go: #129

[gotwarlost]

This change is now on the istanbul harmony branch which in turn has an npm git dependency on the esprima harmony branch

[Raynos]

baller!

[ColCh]

For now: add this dependency to devDependencies:

"istanbul": "git://github.com/gotwarlost/istanbul.git#harmony",

this will install istanbul from harmony branch. Good for me :+1:

[simonzack]

There's also a istanbul-harmony package. It's rather old, could there be a merge from master?

[kunalgolani]

harmony branch is now 152 commits behind master. alternatives?

[tnagapavan]

I tried configuring Istanbul in my ember-cli application. And End up in getting the error of the same kind. Below is the details of the error.
group-test file contains the test cases and applied istanbul on that file. I tried with different approches like downgrading the Istanbul
version,using the code from harmony branch both i end up in getting the same exception.
Any inputs in related to resolving this issue will be very much helpful and appreciated.
D:\codecoverage\istanbul_kafka_1\tests\unit>istanbul cover group-test.js
Transformation error; return original code
{ [Error: Line 1: Unexpected reserved word]
index: 15,
lineNumber: 1,
column: 16,
description: 'Unexpected reserved word' }
No coverage information was collected, exit without writing coverage information
D:\codecoverage\istanbul_kafka_1\tests\unit\group-test.js:1
(function (exports, require, module, __filename, __dirname) { import { test, m
^^^^^^
SyntaxError: Unexpected reserved word
at exports.runInThisContext (vm.js:73:16)
at Module._compile (module.js:443:25)
at Object.Module._extensions..js (module.js:478:10)
at Object.Module._extensions.(anonymous function) as .js
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)
at runFn (C:\Users\pavan\AppData\Roaming\npm\node_modules\istanbul\lib\command\common\run-with-cover.js:122:16)
at C:\Users\pavan\AppData\Roaming\npm\node_modules\istanbul\lib\command\common\run-with-cover.js:248:17
at C:\Users\pavan\AppData\Roaming\npm\node_modules\istanbul\lib\utilfile-matcher.js:68:16**

[albertorestifo]

I just want to :+1: this. Generators are amazing and should be supported, and not on an outdated harmony branch

[wavded]

:+1: - also experiencing issue

[kinshuk]

arrow functions work with istanbul version 0.3.14, but 'import' and 'export' usage gives error: Unexpected reserved word because esprima version is still 2.1.x. esprima 2.2 supports import and export. Any plans for new istanbul version that leverages esprima 2.2?

[Murphydbuffalo]

+1 @kinshuk

[gotwarlost]

@kinshuk - could you try the latest version of istanbul that uses the latest stable esprima? Please close this issue if it is not a problem any more. The harmony branch is now defunct and mainline has everything the harmony branch has and more.

[kinshuk]

Still seems unhappy...

$ ./node_modules/.bin/istanbul instrument test.js
/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:5474
throw e;
^
Error: Line 3: Unexpected token
at createError (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:2336:21)
at unexpectedTokenError (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:2411:13)
at tolerateUnexpectedToken (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:2420:21)
at parseStatementListItem (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:3814:21)
at parseFunctionSourceElements (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:4649:23)
at parseFunctionExpression (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:4859:16)
at parsePrimaryExpression (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:3167:24)
at inheritCoverGrammar (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:2592:18)
at parseLeftHandSideExpressionAllowCall (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:3292:20)
at inheritCoverGrammar (/tmp/node_modules/istanbul/node_modules/esprima/esprima.js:2592:18)

$ cat test.js
'use strict';

export default {
bar: 22
};

Digging deeper, it seems esprima wants sourceType = module
3809: function parseStatementListItem() {
3810: if (lookahead.type === Token.Keyword) {
3811: switch (lookahead.value) {
3812: case 'export':
3813: if (sourceType !== 'module') {
3814: tolerateUnexpectedToken(lookahead, Messages.IllegalExportDeclaration);

I added "sourceType: 'module'" in ESP.parse call in instrumenter.js, but esprima died with a different error...

[RobinQu]

I've just converted a es5 project to es6 syntax including class, super.

The coverage can be done, but it was logging something I don't understand.....

{ range: [ 646, 651 ],
  loc: 
   SourceLocation {
     start: Position { line: 34, column: 6 },
     end: { line: 34, column: 11 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]
{ range: [ 618, 623 ],
  loc: 
   SourceLocation {
     start: Position { line: 26, column: 10 },
     end: { line: 26, column: 15 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]
{ range: [ 172, 177 ],
  loc: 
   SourceLocation {
     start: Position { line: 7, column: 4 },
     end: { line: 7, column: 9 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]
{ range: [ 816, 821 ],
  loc: 
   SourceLocation {
     start: Position { line: 23, column: 4 },
     end: { line: 23, column: 9 } },
  type: 'Super' }
Unsupported node type:Super
{ range: [ 1376, 1381 ],
  loc: 
   SourceLocation {
     start: Position { line: 45, column: 6 },
     end: { line: 45, column: 11 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]
{ range: [ 363, 368 ],
  loc: 
   SourceLocation {
     start: Position { line: 13, column: 4 },
     end: { line: 13, column: 9 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]
{ range: [ 521, 526 ],
  loc: 
   SourceLocation {
     start: Position { line: 22, column: 4 },
     end: { line: 22, column: 9 } },
  type: 'Super' }
Unsupported node type:Super
{ range: [ 1635, 1640 ],
  loc: 
   SourceLocation {
     start: Position { line: 66, column: 11 },
     end: { line: 66, column: 16 } },
  type: 'Super' }
Unsupported node type:Super
{ range: [ 1771, 1776 ],
  loc: 
   SourceLocation {
     start: Position { line: 74, column: 11 },
     end: { line: 74, column: 16 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]
{ range: [ 559, 564 ],
  loc: 
   SourceLocation {
     start: Position { line: 15, column: 4 },
     end: { line: 15, column: 9 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]
{ range: [ 362, 367 ],
  loc: 
   SourceLocation {
     start: Position { line: 17, column: 4 },
     end: { line: 17, column: 9 } },
  type: 'Super' }
Unsupported node type:Super
{ range: [ 1870, 1875 ],
  loc: 
   SourceLocation {
     start: Position { line: 75, column: 13 },
     end: { line: 75, column: 18 } },
  type: 'Super' }
Unsupported node type:Super
{ range: [ 2035, 2040 ],
  loc: 
   SourceLocation {
     start: Position { line: 81, column: 11 },
     end: { line: 81, column: 16 } },
  type: 'Super' }
Unsupported node type:Super
{ range: [ 2088, 2093 ],
  loc: 
   SourceLocation {
     start: Position { line: 85, column: 4 },
     end: { line: 85, column: 9 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]
{ range: [ 183, 188 ],
  loc: 
   SourceLocation {
     start: Position { line: 8, column: 4 },
     end: { line: 8, column: 9 } },
  type: 'Super' }
Unsupported node type:Super
Transformation error; return original code
[TypeError: this[type] is not a function]

[davidrapin]

Fore those coming by and looking for a fix, [email protected] fixed this for me: https://github.com/gotwarlost/istanbul/releases/tag/v1.0.0-alpha.2

[hong-boy]

@davidrapin Thx for your suggestion!