Istanbul: Istanbul with Mocha

[mikechabot]

Having difficulty integrating Istanbul into my existing test suite, which uses Mocha. I've look at issue after issue (via StackOverflow and this repo), and while some seem very similar to my problem, none of the proposed solutions work. As I understand it, Istanbul should integrated very easily with Mocha, so I assume this is an issue on my end.

I have two test commands, test:unit and test:integration. Both work successfully and have no issues. While the commands are long, they are straightforward:

1. Set NODE_ENV to test
2. Transpile ES6 tests back to ES5
3. Require a test file needed for faking the DOM
4. Point to a specific directory

"test:unit": "echo 'Running unit tests' && cross-env NODE_ENV=test mocha --compilers js:babel-core/register --require ./test/utils/test-dom.js --recursive --colors \"test/unit/**/*.js\" ",
"test:integration": "echo 'Running integration tests' && cross-env NODE_ENV=test mocha --compilers js:babel-core/register --require ./test/utils/test-dom.js --recursive --colors \"test/integration/**/*.js\" ",

Again, these execute successfully, and I'd just like to wrap these tests with coverage. I started with just the unit test command, and wrapped it with:

"test:coverage": "echo 'Running coverage' && cross-env NODE_ENV=test && istanbul cover node_modules/mocha/bin/_mocha --compilers js:babel-core/register --require ./test/utils/test-dom.js --recursive --colors \"test/unit/**/*.js\""

It errors with the following, indicating it can't find the ES6 transpilers? What's going on here? This issue seems identical to #589, although this issue seems to indicate I can use ES6.

'Running coverage' 
No coverage information was collected, exit without writing coverage information
C:\_workspaces\patient-dashboard-web\node_modules\mocha\lib\utils.js:628
        throw new Error("cannot resolve path (or pattern) '" + path + "'");
        ^

Error: cannot resolve path (or pattern) 'js:babel-core/register'
    at Object.lookupFiles (C:\_workspaces\patient-dashboard-web\node_modules\mocha\lib\utils.js:628:15)
    at C:\_workspaces\patient-dashboard-web\node_modules\mocha\bin\_mocha:326:30
    at Array.forEach (native)
    at Object.<anonymous> (C:\_workspaces\patient-dashboard-web\node_modules\mocha\bin\_mocha:325:6)
    at Module._compile (module.js:399:26)
    at Object.Module._extensions..js (module.js:406:10)
    at Object.Module._extensions.(anonymous function) [as .js] (C:\_workspaces\patient-dashboard-web\node_modules\istanbul\lib\hook.js:109:37)
    at Module.load (module.js:345:32)
    at Function.Module._load (module.js:302:12)
    at Function.Module.runMain (module.js:431:10)
    at runFn (C:\_workspaces\patient-dashboard-web\node_modules\istanbul\lib\command\common\run-with-cover.js:122:16)
    at C:\_workspaces\patient-dashboard-web\node_modules\istanbul\lib\command\common\run-with-cover.js:251:17
    at C:\_workspaces\patient-dashboard-web\node_modules\istanbul\lib\util\file-matcher.js:68:16
    at C:\_workspaces\patient-dashboard-web\node_modules\istanbul\node_modules\async\lib\async.js:52:16
    at C:\_workspaces\patient-dashboard-web\node_modules\istanbul\node_modules\async\lib\async.js:361:13
    at C:\_workspaces\patient-dashboard-web\node_modules\istanbul\node_modules\async\lib\async.js:52:16
    at done (C:\_workspaces\patient-dashboard-web\node_modules\istanbul\node_modules\async\lib\async.js:246:17)
    at C:\_workspaces\patient-dashboard-web\node_modules\istanbul\node_modules\async\lib\async.js:44:16
    at C:\_workspaces\patient-dashboard-web\node_modules\istanbul\node_modules\async\lib\async.js:358:17
    at LOOP (fs.js:1614:14)
    at nextTickCallbackWith0Args (node.js:433:9)
    at process._tickCallback (node.js:362:13)

Istanbul installed with: npm install --save-dev istanbul --> "istanbul": "^0.4.3",

[mikechabot]

If I change the command to use babel-node, I get further, but it's still failing on ES6 syntax it seems:

"test:coverage": "echo 'Running coverage' && cross-env NODE_ENV=test && babel-node node_modules/istanbul/lib/cli.js cover node_modules/mocha/bin/_mocha --require ./test/utils/test-dom.js --recursive --colors \"test/unit/**/*.js\"",

Results in:

'Running coverage' 
Transformation error; return original code
{ [Error: Line 1: Unexpected token] lineNumber: 1, description: 'Unexpected token', index: 15 }
Transformation error; return original code
{ [Error: Line 6: Unexpected token] lineNumber: 6, description: 'Unexpected token', index: 103 }
Transformation error; return original code
{ [Error: Line 1: Unexpected token] lineNumber: 1, description: 'Unexpected token', index: 15 }
Transformation error; return original code
{ [Error: Line 1: Unexpected token] lineNumber: 1, description: 'Unexpected token', index: 15 }
Transformation error; return original code
{ [Error: Line 1: Unexpected token] lineNumber: 1, description: 'Unexpected token', index: 15 }
Transformation error; return original code
{ [Error: Line 1: Unexpected token] lineNumber: 1, description: 'Unexpected token', index: 15 }
Transformation error; return original code
{ [Error: Line 1: Unexpected token] lineNumber: 1, description: 'Unexpected token', index: 15 }
=============================================================================
Writing coverage object [C:\_workspaces\patient-dashboard-web\coverage\coverage.json]
Writing coverage reports at [C:\_workspaces\patient-dashboard-web\coverage]
=============================================================================

=============================== Coverage summary ===============================
Statements   : 50% ( 2/4 )
Branches     : 100% ( 0/0 )
Functions    : 33.33% ( 1/3 )
Lines        : 50% ( 2/4 )

[codekirei]

Could be a bunch of things, but the first thing that jumps out to me to try is inserting -- after _mocha and before the rest of the flags in your "test:coverage" script.

$ istanbul help cover

...
Usage: istanbul cover [<options>] <executable-js-file-or-command> [-- <arguments-to-jsfile>]

Because that crucial -- is missing, I'm guessing Istanbul is trying to process all those flags that are intended for mocha, and that's causing the whole thing to throw. Hopefully that's it, because that's a nice and easy fix :sweat_smile:

EDIT:

Just a quick note — to clean up those npm scripts, you might consider creating a test/mocha.opts file with the mocha flags common to all scripts running mocha:

--compilers js:babel-core/register
--require ./test/utils/test-dom.js
--recursive
--colors

Then you don't have to manually write the same flags multiple places, and it's easier to make updates down the road if you need to tweak something.

[mikechabot]

@codekirei Well don't I feel like a horse's patoot - thanks! The tests now compile and run, and thanks for the suggestion on mocha.opts, I've had a TODO for that for far too long. At any rate, I cleaned up the command to the point where I can run the following, which is much more in line with the docs:

"test:coverage": "istanbul cover node_modules/mocha/bin/_mocha",

However, when my tests finish, I don't see the coverage summary, nor any output in the coverage folder, which does get created during each test run. What's interesting is in my second post of this thread I noted that I did get a coverage summary although the tests failed, however now the execution is successful, but no summary?

  33 passing (133ms)


Process finished with exit code 0

Is there perhaps anything further I'm missing? Taking a look at this: https://www.npmjs.com/package/istanbul#usage-on-windows, it doesn't look as though I need to specify an output dir/file for Istanbul, but perhaps I'm wrong?

[codekirei]

No worries, it's an easy thing to miss :smile:

From your first post, it looks like you're running Istanbul 0.4.3 — correct? If so, see if using the 1.0.0-alpha.2 branch fixes things for you.

$ npm install --save-dev [email protected]

If that doesn't fix it, I'd be curious to see exactly what you put in mocha.opts.

[mikechabot]

You are 2/2 tonight, my friend; the latest lib did the trick; no modifications to mocha.opts required (and it's identical to your edit in fact, FWIW).

All looks well, per the below, and I very much appreciate your quick assistance here. If I find a bit of time, I'll submit a pull request with some doc changes, maybe a common errors or "gotchas" section or something).

  33 passing (185ms)

File [D:\_workspaces\patient-dashboard-web\src\common\data\status.js] ignored, nothing could be mapped

=============================== Coverage summary ===============================
Statements   : 57.28% ( 122/213 )
Branches     : 44.68% ( 42/94 )
Functions    : 49.15% ( 29/59 )
Lines        : 58.42% ( 118/202 )
================================================================================

Process finished with exit code 0

[codekirei]

Woot! Happy to help :+1:

For the cherry on top, that File ... ignored, nothing could be mapped warning is likely due to this issue, to save you a bit of hunting.

I'm neither author nor maintainer of Istanbul, just a sometimes-helpful passerby who uses Istanbul a lot. That said, it does seem like there have been many similar issues with similar fixes reported lately (especially around ES6 and the 1.0.0 branch), so I imagine @gotwarlost might appreciate a PR covering common current "gotchas" in the docs.

That said, he's mentioned that he's hoping to devote some time to hacking on Istanbul soonish, so I would definitely get confirmation from him before putting much time into a PR, as things may be subject to change.

[mikechabot]

Cool, thanks for the heads up! And will do.

[msmfsd]

I had so many issues with getting ES6 imports to work with istanbul..
I just installed babel-plugin-istanbul + nyc + cross-env and then ran:
"test": "cross-env NODE_ENV=test nyc --reporter=lcov --reporter=text mocha --compilers js:babel-register test/*.js"