Karma: Do not fail when the test set is empty

[matthughes]

Karma fails if there are no matching tests. Shouldn't it just skip tests if there are none? If not the default, than I think this should be configureable.

I realize you may ask "why are you running karma if you have no tests?" but it's common to run karma along with a bunch of other tasks in grunt. If there are no matching tests, the whole task fails.

› ./node_modules/karma/bin/karma start
INFO [karma]: Karma v0.10.9 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [launcher]: Starting browser Firefox
WARN [watcher]: Pattern "/Users/mhughes/foo/src/test/js/**/**.js" does not match any file.
INFO [Firefox 27.0.0 (Mac OS X 10.9)]: Connected on socket rreoh8Sf1OuByCl2ZRyZ
INFO [Chrome 32.0.1700 (Mac OS X 10.9.1)]: Connected on socket 0nvZF76Y3Aay8m1-ZRya
Firefox 27.0.0 (Mac OS X 10.9) LOG: 'Tests loaded, ready to run.'
Firefox 27.0.0 (Mac OS X 10.9): Executed 0 of 0 SUCCESS (0 secs / 0 secs)
Chrome 32.0.1700 (Mac OS X 10.9.1) LOG: 'Tests loaded, ready to run.'
Firefox 27.0.0 (Mac OS X 10.9): Executed 0 of 0 ERROR (0.057 secs / 0 secs)
                              Firefox 27.0.0 (Mac OS X 10.9): Executed 0 of 0 ERROR (0.057 secs / 0 secs)
Chrome 32.0.1700 (Mac OS X 10.9.1): Executed 0 of 0 ERROR (0.283 secs / 0 secs)

[vojtajina]

This is by design. I guess it could be configurable.

[matthughes]

Any chance you could point me to where this would go? I poked around a little bit this morning, but didn't see it. Ideally it would happen before we start the server.

[vojtajina]

It's here https://github.com/karma-runner/karma/blob/249b27d97004b081465be91982b6a83ab6184550/lib/browser.js#L140-L142

Probably not the best place ;-)

Long term, we should add some notion of "reason why it fails". Now there is just "error", which was originally error in the browser (eg. syntax error)... but now we use this "error" for other reasons such as "0 test" failure or when a test run got cancelled (eg. because a preprocessor failed).

[eddiemonge]

I would prefer it to warn if there are no tests and not fail. Maybe its a new project with no tests yet.

[deLaDude]

I agree this should be configurable. A warning instead of a failure would be nice as well.

At the company I work for we setup a shared grunt workflow for all projects to use however we maintain some legacy code-bases that have no tests. This is also a pain the butt for prototyping and rapid application development. Right now we are creating dumby test files just to appease karma which is less than ideal.

[budde377]

I wouldn't mind to take a shot at this. I guess a configuration option should be added to the runner and server (for singlerun). Any preferences on a name for this option? E.g. failOnEmptyTestSuite : boolean.

[dignifiedquire]

@budde377 that'd be great, yes the config name sounds good to me, I think it should default to true, so it does not break existing behaviour and if it's set to false it should not fail, but still print a warning that there have been no tests run.

[adamreisnz]

@dignifiedquire what version of Karma was this fix included in? I'm using 0.13.19 and the config setting seems to be ignored:

failOnEmptyTestSuite: false

10 02 2016 14:51:38.728:INFO [karma]: Karma v0.13.19 server started at http://localhost:9876/
10 02 2016 14:51:38.753:INFO [launcher]: Starting browser PhantomJS
10 02 2016 14:51:39.563:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: Connected on socket /#BAWUm-MAtCqIImEJAAAA with id 85239157

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 0 ERROR (0.004 secs / 0 secs)


npm ERR! Darwin 14.5.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "test"
npm ERR! node v5.5.0
npm ERR! npm  v3.3.12
npm ERR! code ELIFECYCLE
npm ERR! [email protected] test: `karma start karma.conf.js`
npm ERR! Exit status 1

[budde377]

@adambuczynski This was committed on Jan 11 and the latest release (0.13.19) is from Jan 6, so the the feature hasn't been released yet.

[adamreisnz]

@budde377 cheers thanks, guess it just missed it. I will await it in the next release :)