Karma: Option to restart browser after test run

[diwu1989]

There might be a memory leak with my test suite and PhantomJS for some reason, I suspect it has to do with angularjs services being created repeatedly. Other people are also hitting this https://github.com/karma-runner/karma/issues/1364

Is there any way to tell Karma to restart the browser client after each full test run?
When Karma is set to watch files and run continuously, it's very easy for me to eat up 3GB of memory after a few test runs.

[maksimr]

Is there any way to tell Karma to restart the browser client after each full test run?
No

But you can use external tools like watchdog and start karma when files changed

I think we should fix issue with memory leak instead add new options

Thanks!

[nicbou]

This wouldn't solve the issue. We need to run all tests at once, and the PhantomJS crashes cause a serious problem for us. It's not only about running the recently modified tests.

[maksimr]

@nicbou

watchmedo shell-command \
    --patterns="*.js" \
    --recursive \
    --command='karma start --single-run' \
    .

This command will be start karma on every change js file and run all tests. Yup, this will be slowly because karma will process all files on the new.

Problem should be solved but I don't think that extra option that will refresh browser best solution now.

Thanks

[nicbou]

@diwu1989 suggests restarting the browser after each _full_ test run, but I think we should look into an option to restart after _n_ runs.

Our PhantomJS instance crashes after about 2000 tests, when it reaches 8-10 GB of RAM, and fails to restart. There are several other reports of the same problem. Chrome and Firefox are using much less RAM and finish running fine.

According to my research, it could be an issue with PhantomJS not garbage collecting until page.close is called, so memory consumption keeps climbing until it crashes. Restarting the browser or creating a new page after a certain number of tests might fix it.

This might open the door for a solution to https://github.com/karma-runner/karma/issues/439

Sources:
http://www.quora.com/What-behavior-have-you-seen-from-PhantomJS-that-differs-from-that-of-a-mainstream-non-headless-browser
http://stackoverflow.com/questions/27239708/how-to-get-around-memory-error-with-karma-phantomjs

[erindru]

I am also experiencing the issue that @nicbou describes with PhantomJS. Since my application is packaged by feature, I am going to do multiple Karma runs, one for each feature. This will be driven by grunt in an attempt to minimise the amount of duplicate configuration required.

If there was a builtin way to make Karma execute tests in batches, that would be ideal. This is loosely described in #412 and #439 but those two issues seem to be mainly about running multiple tests in parallel, which would be handy but not essential.

Arguably there should be a hook to somehow make PhantomJS perform garbage collection periodically, but I realise that Karma is not meant to know anything about PhantomJS

I should probably add that the only reason I am using PhantomJS is to run the tests on the build server, locally I use Firefox and Chrome to develop them. Its a rigmarole to get Firefox and Chrome to work in a headless environment, and more so if you're constrained to Amazon Linux (like I am at the moment)

[Splaktar]

Using the forked karma-phantomjs2-launcher plugin seems to work for me in solving the crash from memory leak issues: http://stackoverflow.com/a/32676008/633107

[rodoabad]

@Splaktar PhantomJS2 will still crash after a certain threshold. For Window users this is around the 1.2 GB mark.

[tpflueger]

I've noticed this as well and even see it occur within Chrome.