Angular-cli: Testing directly with karma

[krokofant]

Versions

    "@angular-devkit/build-angular": "~0.6.0",
    "@angular-devkit/core": "0.0.29",
    "@angular/cli": "6.0.0",
    "@angular/common": "6.0.0",
    "@angular/compiler": "6.0.0",
    "@angular/compiler-cli": "6.0.0",
    "@angular/core": "6.0.0",
    "@angular/forms": "6.0.0",
    "@angular/http": "6.0.0",
    "@angular/language-service": "6.0.0",
    "@angular/platform-browser": "6.0.0",
    "@angular/platform-browser-dynamic": "6.0.0",
    "@angular/router": "6.0.0",
    "@angular/upgrade": "6.0.0",

Windows 10

Repro steps

Run karma start karma.conf.js

Observed behavior

s\angular_devkit\build_angular\src\angular-cli-files\plugins\karma.ts:53
    throw new Error(`The '@angular-devkit/build-angular/plugins/karma' karma plugin is meant to` +
          ^
Error: The '@angular-devkit/build-angular/plugins/karma' karma plugin is meant to be used from within Angular CLI and will not work correctly outside of it.

Desired behavior

Be able to run test directly with karma as before version 6

This became a problem after upgrading to version 6.

[jbandi]

This also breaks integration with the test runner in WebStorm & IDEA ... it's not possible to run tests from WebStorm any more by pressing the "green triangles" next to the test code :-(

[vycoder]

Anybody know any alternative on running the tests? I just migrated to angular 6 as well. Now I can't run our tests. Wondering what's the suggested way to run test now? Do I need some cli configuration or something?

[krokofant]

@josephharveyangeles You can tunnel everything via the angular cli like

ng test --karma-config karma.somestuff.conf.js

[jbandi]

For the IntelliJ (WebStorm, IDEA) integration the issue is tracked here:
https://youtrack.jetbrains.com/issue/WEB-32653

[buster]

Obviously this was broken by https://github.com/angular/angular-cli/issues/10478 but the issue doesn't say why this is an error now..
Wouldn't the sensible thing to do, to make it a warning instead of an error before turning it into an error?
This currently prevents us from updating a project to angular 6 and i can imagine that the IntelliJ side won't be fixed in the short term..

Also, please vote on https://youtrack.jetbrains.com/issue/WEB-32653 if you are affected.

[buster]

FYI, according to the jetbrains ticket, the issue will be fixed in IDEA 2018.1.4

[ciarancourtney]

Still broken on:

IntelliJ IDEA 2018.2 EAP (Ultimate Edition)
Build #IU-182.2371.4, built on May 15, 2018

[timdoes]

And it is still broken on

Webstorm 2018.2 EAP
Build #WS-182.2371.25, built on May 17, 2018

[AlexGS74]

@timdoes It works if you create a new run configuration. It will detect Cli and use it as Karma engine. If you run an old configuration still pointing to a global/local karma instance, it fails.

[maprox]

@AlexGS74 I've used your suggestion but still see "Error: Cannot find module '[myproject]\node_modules\@angular\cli\lib\server.js'". I'm om 2018.1.4

[timdoes]

@AlexGS74 That worked. Thanks!

[AlexGS74]

@maprox Latest released version of Webstorm is 2018.1.3. It doesn't support this as it was release before Angular Cli 6 release.
What I describe above works with current Webstorm 2018.2 EAP, which will eventually turn into release.

[tschach]

According to the WebStorm Release Notes this should be fixed: https://confluence.jetbrains.com/display/WI/WebStorm+181.5087.27+Release+Notes
I am unable to test it since I am working with the IDEA.
For the IDEA I have opened a separate issue, as this seems to be only fixed in WebStorm: https://youtrack.jetbrains.com/issue/IDEA-192411

[tschach]

Hi, I have updated IntelliJ IDEA to 2018.1.4 (Build #IU-181.5087.20, built on May 16, 2018) and the Karma plugin to 181.5087.27 and now it works, provided you point your Karma package to your Angular-CLI package and not to your Karma package (as suggested by @AlexGS74).

[ilancohen]

With IntelliJ IDEA 2018.1.4, I'm using the config that @tschach posted, but still getting the error that @maprox got:
"Error: Cannot find module '[myproject]\node_modules\@angular\cli\lib\server.js'".
Does anyone have a solution to this?

[ilancohen]

@maprox - Fixed it, once I updated the Karma plugin.

[Zephilim]

This is not just an intellij issue. I am running into this problem on macOS running "karma start" from the command line and I get this error. I'm using Angular 6.0.4 and karma 2.0.2

"dependencies": {
"@angular/animations": "^6.0.4",
"@angular/common": "^6.0.4",
"@angular/compiler": "^6.0.4",
"@angular/core": "^6.0.4",
"@angular/forms": "^6.0.4",
"@angular/http": "^6.0.4",
"@angular/platform-browser": "^6.0.4",
"@angular/platform-browser-dynamic": "^6.0.4",
"@angular/router": "^6.0.4",
...

"devDependencies": {
"@angular-devkit/build-angular": "~0.6.5",
"@angular/cli": "^6.0.8",
"@angular/compiler-cli": "^6.0.4",
"@angular/language-service": "^6.0.4",
"@types/chartist": "^0.9.37",
"@types/jasmine": "^2.8.8",
"@types/jasminewd2": "~2.0.2",
"@types/jquery": "^3.2.5",
"@types/node": "~6.0.60",
"codelyzer": "^4.3.0",
"jasmine-core": "~2.6.2",
"jasmine-spec-reporter": "^4.2.1",
"karma": "^2.0.2",
"karma-chrome-launcher": "^2.2.0",
"karma-cli": "~1.0.1",
"karma-coverage-istanbul-reporter": "^1.2.1",
"karma-jasmine": "~1.1.0",
"karma-jasmine-html-reporter": "^0.2.2",
...

If there is a work around for this environment, please let me know.

[krokofant]

@Zephilim I mentioned this solution in an above comment:

Tunnel the karma config via ng test:

ng test --karma-config karma.somestuff.conf.js

[avelow]

@krokofant does your command solve the problem exposed by @jbandi : "running the test from the green triangles in webstorm next to the test code"
I get empty test suite all the time I start the test from the test runners triangles.

[Zephilim]

Thanks krokofant, I guess I didn't really get what you meant by tunnel. I actually discovered (by accident) another way of running the coverage and that is simply "ng test --code-coverage", just not very well documented.

[therepo90]

For me updating webstorm it helped

[mscudlik]

doesn't help if you want to use @angular-devkit/build-angular without @angular/cli

[nanoTitan]

The original issue as the author has stated is that upgrading to Angular 6 has broken the ability to do:
Run karma start karma.conf.js

This by itself has nothing to do with Webstorm, and many of us don't use that IDE. Is there any resolution to this problem as stated?

[nikdo]

@krokofant Tunneling it via ng test does not work for me. The command immediately terminates with no output. Do I need some configuration in angular.json to make it work? Is there an option to make ng test more verbose?

I'm really disappointed that there is no option to go without @angular/cli. We've ejected our Angular project two years ago and now after long and painful upgrade to Angular 6 (there is almost no documentation for upgrading ejected projects) we are left with @angular/[email protected].

[Danieltreiber]

I fixed this problem by removing and re-enabling the Angular CLI Framework. I did this by removing the AngularJS plugin, restarting, enabling it, restarting x2 (not sure why I needed to restart again).

This will tell karma to use node_modules/@angular/cli/bin/ng test vs node_modules/karma/bin/karma start (you can see the command that karma is running in the Karma Server tab)

[nikdo]

Well, we have found a workaround: We ditched Karma and replaced it with Jest.

I just wonder why ejected and non-cli solutions are left behind with many issues and no documentation. :(

[edarioq]

Having the same issue, really don't want to have to start over with another testing framework, any fixes? We don't use the Angular CLI and we have a hybrid AngularJS/Angular 6 app, reason which we are not using the CLI, the project is too big and too costly for an immediate migration.

[peter-snr]

This is just going to encourage developers to ignore tests and they are so difficult to set up and there is so much information out there that is just plain wrong.

I wanted some html reports for our build server, been at it hours now and still nothing?
I have the karma-jasmine-html-reporter installed in my karma.conf but nope, no html reports?

Really lost, this is so easy in JAVA with JUNIT, why so difficult using Angular 6 ???
Seems @nikdo has the solution ditch it and start again with something that actually works, Jest??

Edit: - This might be useful so I have added it here.

https://medium.com/welldone-software/an-overview-of-javascript-testing-in-2018-f68950900bc3

• In short, if you want to “just get started” or looking for a fast framework for large projects, go with Jest.

And this will help you get started with Jest.
https://codeburst.io/angular-6-ng-test-with-jest-in-3-minutes-b1fe5ed3417c

[james-schwartzkopf]

@peter-snr this issue doesn't seem to be related to your problems. This isn't stack exchange, but karma-jasmine-html-reporter doesn't do what you think it does. It provides a nice interface to show in the browser while your test are running, not a final HTML report. Given your JUnit comment, you might be interested in karma-junit-reporter (remember to also add it to the reporters array).

[IonelLupu]

Did anyone encountered ChromeHeadless have not captured in 60000 ms, killing when running from WebStorm/PHPStorm 2018.3? It works fine when I run ng test in my terminal but when creating a run configuration inside the IDE it doesn't work.

[cport1]

Did anyone encountered ChromeHeadless have not captured in 60000 ms, killing when running from WebStorm/PHPStorm 2018.3? It works fine when I run ng test in my terminal but when creating a run configuration inside the IDE it doesn't work.

When you go to Run/Debug Configurations, make sure the Config file is set to your src/karma.conf.js and your Karma Package is pointed at your node_modules/@angular/cli ...

[IonelLupu]

@cport1 . I think I've setup all of theese. This is my IDE configuration:

And this is the output:

/frontend/node_modules/@angular/cli/bin/ng test frontend --karma-config ~/.PhpStorm2018.3/config/plugins/js-karma/js_reporter/karma-intellij/lib/intellij.conf.js --source-map
 10% building modules 1/1 modules 0 active(node:13055) DeprecationWarning: Tapable.plugin is deprecated. Use new API on `.hooks` instead
 10% building modules 3/4 modules 1 active /frontend/src sync /\.spec\.ts$/11 12 2018 11:40:55.931:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/
11 12 2018 11:40:55.932:INFO [launcher]: Launching browser ChromeKarma with unlimited concurrency
11 12 2018 11:40:55.959:INFO [launcher]: Starting browser ChromeHeadless
Waiting for a captured browser... To capture a browser open http://localhost:9876/
11 12 2018 11:41:55.983:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
11 12 2018 11:41:55.990:INFO [launcher]: Trying to start ChromeHeadless again (1/2).
11 12 2018 11:42:56.043:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
11 12 2018 11:42:56.055:INFO [launcher]: Trying to start ChromeHeadless again (2/2).
11 12 2018 11:43:56.114:WARN [launcher]: ChromeHeadless have not captured in 60000 ms, killing.
11 12 2018 11:43:56.120:ERROR [launcher]: ChromeHeadless failed 2 times (timeout). Giving up.

As you can see, even if I set my karma.conf.js it still uses the config/plugins/js-karma/js_reporter/karma-intellij/lib/intellij.conf.js, which is strange

[avoerman]

@JCWolf Node might be running out of memory; this started happening to me and I added

--max_old_space_size=5048

to the Node options in the debug configuration. It could also mean a type error. WebStorm occasionally swallows these, so make sure your tests can compile 'ng test --watch=false' from a terminal.

[IonelLupu]

@avoerman The tests are running fine from terminal (when I run inside the docker container) but In WebsStorm/PHPStorm it fails with the same error. Maybe I have to set something similar to Remote CLI Interpreters from docker for karma like I set for PHP?

[mgechev]

In the CLI we want to provide a well encapsulated build; invoking the tests manually with karma breaks this encapsulation.

I'd recommend you to use the solution from this comment https://github.com/angular/angular-cli/issues/10703#issuecomment-387319138.

If I've missed something or misunderstood your use case, please open a new issue with details.

[marthinus-engelbrecht]

@mgechev Problem is when that encapsulated build has issues. See #11672. The tight coupling here isn't making angular robust. Really don't want to be without test coverage reporting even though I let my tests drive implementation. And yes, I should look more into the issue myself and help fix it but right at this moment, I don't have the time. I'm also sure there are a billion other use cases where you don't want to be forced to use ng test

[angular-automatic-lock-bot[bot]]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{_This action has been performed automatically by a bot._}