Karma: Node v4.2.6 - FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory

[busticated]

hey there -

i'm hitting the following error after upgrading to node v4 LTS.

02 02 2016 11:13:08.092:INFO [karma]: Karma v0.13.19 server started at http://localhost:9876/
02 02 2016 11:13:08.122:INFO [launcher]: Starting browser PhantomJS

<--- Last few GCs --->

    22555 ms: Scavenge 1441.4 (1502.2) -> 1441.4 (1502.2) MB, 1.6 / 0 ms (+ 2.4 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
    22610 ms: Mark-sweep 1441.4 (1502.2) -> 1441.2 (1502.2) MB, 55.5 / 0 ms (+ 3.6 ms in 2 steps since start of marking, biggest step 2.4 ms) [last resort gc].
    22664 ms: Mark-sweep 1441.2 (1502.2) -> 1441.2 (1502.2) MB, 54.0 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3acf8b8b4629 <JS Object>
    1: exec [native regexp.js:~85] [pc=0x1ab48201d158] (this=0x3477de668ed1 <JS RegExp>,k=0x1dc821cffa39 <String[106]: Mozilla/5.0 (Macintosh; Intel Mac OS X) AppleWebKit/538.1 (KHTML, like Gecko) PhantomJS/2.1.1 Safari/538.1>)
    2: parse [/path/to/my/project/node_modules/karma/node_modules/useragent/index.js:427] [pc=0x1ab4821e1cbf] (this=0x276f61054869 <JS Funct...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6`

Likewise, i see a similar error when running tests in chrome:

>>karma start
02 02 2016 11:49:26.366:WARN [karma]: No captured browser, open http://localhost:9876/
02 02 2016 11:49:26.447:INFO [karma]: Karma v0.13.19 server started at http://localhost:9876/
02 02 2016 11:49:26.455:INFO [launcher]: Starting browser Chrome

<--- Last few GCs --->

21051 ms: Mark-sweep 1429.9 (1488.2) -> 1429.4 (1488.2) MB, 26.3 / 0 ms (+ 124.4 ms in 20 steps since start of marking, biggest step 82.2 ms) [allocation failure] [GC in old space requested].
21163 ms: Mark-sweep 1429.4 (1488.2) -> 1429.4 (1488.2) MB, 110.6 / 0 ms [last resort gc].
21298 ms: Mark-sweep 1429.4 (1488.2) -> 1429.3 (1488.2) MB, 135.5 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x24bd825b4629 <JS Object>
    2: decodeURIComponent(aka decodeURIComponent) [native uri.js:248] [pc=0x2575a442da41] (this=0x24bd825041b9 <undefined>,W=0x111d0ab40cd9 <String[2]: id>)
    3: /* anonymous */(aka /* anonymous */) [querystring.js:83] [pc=0x2575a4426aee] (this=0x24bd825041b9 <undefined>,s=0x111d0ab40cd9 <String[2]: id>,decodeSpaces=0x24bd825041b9 <undefined>)
    4: arguments adaptor frame: 1->2
    5: decod...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6

My environment:

• node: v4.2.6
• phantomjs: v1.9.8
• karma: v0.13.19
• karma-phantomjs-launcher: v0.2.3
• karma-chrome-launcher: v0.2.2
• karma-chai: v0.1.0
• karma-mocha: v0.2.1
• karma-sinon: v1.0.4
• karma-sourcemap-loader: v0.3.7
• karma-webpack: v1.7.0
• webpack: v1.12.12

I tried updating to the latest phantomjs but the error persisted - upgrades:

• phantomjs phantomjs-prebuilt: v2.1.1
• karma-phantomjs-launcher: v1.0.0

Running under the old configuration works fine:

• node: v0.10.41
• phantomjs: v1.9.8
• karma: v0.13.19
• karma-phantomjs-launcher: v0.2.3
• karma-chrome-launcher: v0.2.2
• karma-chai: v0.1.0
• karma-mocha: v0.2.1
• karma-sinon: v1.0.4
• karma-sourcemap-loader: v0.3.7
• karma-webpack: v1.7.0
• webpack: v1.12.12

I'm on os-x v10.10.5

[busticated]

more info: if i limit the number of test files - e.g.

instead of:

    files: [
        'public/js/**/*.test.js'
    ]

i use:

    files: [
        'public/js/util/*.test.js'
    ]

both chrome and phantomjs run the reduced test suite fine. not headline news i realize but still :-D

is there a way to set the V8 option --max_old_space_size to increase the max memory limit of the karma process? i'm not seeing anything obvious in the docs.

edit: fwiw, looks like i can run ~800 files _tests_ across 87 files before hitting node's 1.7gb memory limit and falling over

[busticated]

fwiw, my work-around is to run karma as a child process using the public api - this allows me to set the --max_old_space_size flag (whereas running using karma's CLI does not).

var karma = fork('./karma-worker', {
    execArgv: ['--max_old_space_size=4096'] // increase v8's available memory to 4gb
});

karma.send({ type: 'start' /*, config: { ... } */});

and then in the worker:

process.on('message', function(msg){
    if (msg.type === 'start'){
        return new KarmaServer(config, done).start();
    }
});

function done(code){
    if (process.connected){
        process.send({ type: 'done', code: code });
    }
    process.exit(code);
}

there's some additional fiddly bits around error / disconnect handling, etc but this is the basic gist. only thing i'm missing at the moment is colored output - for whatever reason the only messages that retain their color (green) are 'karma started', 'starting browser', and 'connected' messages (the first 3 you typically see during a run).

[dignifiedquire]

I'm not sure but have you tried karma start -- --max_old_space_size=4096 ? If not I'd be happy to accept a PR that allows for that possibility

[busticated]

re: memory consumption - comparing node v4 LTS to v0.10.41 i see similar memory use. v0.10.41 maxes out at ~1.9gb, node v4 at ~2.3gb. in both cases, that seems really high to me (as a reference point, my build task - LESS & webpack - uses ~100mb of memory). the _really_ weird thing is that v4 crashes but v0.10 does not yet my understanding is that both use the same default memory limit.

¯_(ツ)_/¯

[busticated]

@dignifiedquire thanks for taking a look.

have you tried karma start -- --max_old_space_size=4096 ?

yep, it doesn't work (same for setting it via config.client.args)

I'd be happy to accept a PR that allows for that possibility

i poked at that approach a bit but it seemed like a dead-end - in part b/c of this.

[dignifiedquire]

A lot of things have changed in the v8 versions between 0.10 and 4.0 so it might be that sth there is the result of the crashing in one point and working fine in the other.

From the file count these limits should never be reached (I've run karma on larger projects, e.g. take a look at the angular main source) so it seems to be that there is probably some memory leak/bug why this is happening :/

[dignifiedquire]

Could you try reducing the amount of files/tests you are loading and see if there is a limit/specific file that triggers this?

[busticated]

@dignifiedquire yep, can do. buried in my note above --> i can run ~800 files tests across 87 files before hitting the limit and crashing. that was just a quick try though. i'll be more deliberate and see what i see :)

[dignifiedquire]

What's the target file count? (test files and source files being loaded)

[busticated]

@dignifiedquire just looking at chrome's network tab while running in the debug view, i see 232 files / 458mb(!). this includes helpers like chai.js, mocha.js, etc but is mostly local source. (NOTE: this is while running to completion w/ --max_old_space_size=4096)

it's not uncommon to see individual test files > 4mb since these are built by webpack and include external dependencies (e.g. react, etc) as well as sourcemaps. that certainly seems like something i can tune a bit better - maybe by loading common deps in a single file or something.

edit: like this (>_<)

[havenchyk]

@dignifiedquire do you need any additional info? I faced the same issue

[ianks]

This issue is also coming up with me. I can confirm that karma is using ~1600Mb of virtual memory and ~600Mb of actual memory.

Some notes about my config:

• uses karma-webpack
• it is a react React application
• uses typescript

I attempted to take some heap snapshots of karma using node-inspector, and noticed that when I would attempt to snapshot the heap for when I was testing my React components, i would trigger the Allocation failed - process out of memory, which left me unable to get the final results of the snapshot :disappointed:

Does anyone have any pointers about further debugging this issue?

[havenchyk]

@ianks what I did is, maybe it can be helpful

• update babel to the latest version that supports passing params like max_old_space_size
• set max_old_space_size

[teleaziz]

@ianks just wondering if this is related to karma-sourcemap-loader plugin? same setup except it's angular 2 project with TS

[daviddias]

We are facing the same issue with https://github.com/ipfs/js-ipfs-api, both in Node.js v4 and Node.js v6.

@dignifiedquire do you recommend any solution? Running it on a child_process? Adding a option to karma to enable using more memory for the proc?

[dignifiedquire]

After some investigation, this seems to be related with webpack, some issues for reference that could influence this

• https://github.com/webpack/webpack/issues/2157
• proposed fix: https://github.com/webpack/webpack/pull/2497 (does not work for me)
• https://github.com/webpack/webpack/issues/1914

[dignifiedquire]

It might be that we just need to give it more memory on larger projects though, like suggested in https://github.com/webpack/webpack/issues/1875

[dignifiedquire]

When using gulp this works fine:

> node --max_old_space_size=4096 node_modules/.bin/gulp task

[havenchyk]

@dignifiedquire the thing is we're using babel-node and there wasn't any way to increase space size.

[havenchyk]

proposed fix: webpack/webpack#2497 (does not work for me)

doesn't work for me as well. still have crashes

[dignifiedquire]

Thanks @havenchyk. From your comment above it sounded like the latest version of babel-node does support the flag passing though?

[havenchyk]

yep, it works (node takes 4GB of my memory), but I don't like this solution, it's a workaround. I believe that the problem is in webpack itself, but didn't find any quick or even not a quick fix.

So for you as for karma maintainer, I'd say it's not a problem of karma itself and issue can be closed.

I have the same error (rarely) with webpack in dev mode

[Anmo]

I had the same problem and I used the require.context like is suggested in karma-webpack docs.

The problem with this approach is that you can't include test dynamically (just test one spec for instance).
I did a work around with webpack.definePlugin. With the karma config pointing to the test/test_index.js I changed that file to have something like this: var testsContext = require.context(".", true, __REQUIRE_CONTEXT_REGEX__) and in the webpackConfig I just add a new definePlugin({__REQUIRE_CONTEXT_REGEX__: new RegExp(...)}) (this regex will depends on process.argv).
With this I now can do something like: npm test -- --spec sagas

Note: the require.context will not fix the Allocation failed - process out of memory issue.

[pbio]

If others are having this issue still, I solved it buy using

node --max_old_space_size=4096 node_modules/karma/bin/karma start

instead of

karma start

Make sure karma is included in your project's node_modules.

[gomesNazareth]

I used --max_old_space_size and I again got the memory error. After doing some online research i realized it not underscore but dash like --max-old-space-size

[stevemao]

another workaround is:

1. Create a test loader file: loadtests.js
2. require all the tests in loadtests.js

const testsContext = require.context('./src', true, /\.spec\.js$/);
testsContext.keys().forEach(testsContext);

1. Include loadtests.js in karma.conf.js

files: [
      'loadtests.js',
],
preprocessors: {
      'loadtests.js': ['webpack', 'coverage'],
},

[aaronisme]

@stevemao thanks, facing the same issue, in webpack use this workaround to fix that

[Opty1712]

Got same error. After trying a lot of methods for me helped this one (npm script in package.json):

"test": "node --max_old_space_size=4096 ./node_modules/karma/bin/karma start --single-run || exit 0",

Also waiting for new TS-loader, see details here: https://github.com/TypeStrong/ts-loader/issues/502#issuecomment-307370502

[chennas]

Hi,

After increasing the size, we are facing syntax issues on windows machines, on MAC machines working fine with out any issues, any help?

[stevemao]

@chennas your problem doesn't seem to related to this issue. More like different versions of node.

[chennas]

@stevemao Thanks for your quick reply. If I remove the "max size setting" karma is running, but with heap memory issues? Any thoughts are welcomed.

Thanks

My node version is v8.3.0

[anothermarkus]

I found that with a growing number of tests, this was a problem even though I allocated more memory with the flag --max_old_space_size=4096

karma start karma.conf.js --max_old_space_size=4096

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Removing source mapping as defined here: https://webpack.github.io/docs/usage-with-karma.html#source-maps helped get over this

karma.conf.js

webpack: {
// devtool: 'inline-source-map',

Good luck out there!

[epernice]

@stevemao's solution is the only one that worked for me. Setting max_old_space_size to 4096 did not help (we have about 1500 tests).

[jfcwagner]

Agree with @epernice. @stevemao's solution worked for me, too. My team and I have been dealing with this problem on and off for months. Thanks @epernice for calling it out; thanks @stevemao for the workaround!

[GalitTugi]

Happened to me too, and found out that caused by webpack's
EvalSourceMapDevToolPlugin (aso happened with SourceMapDevToolPlugin).
I need this plugin for debugging, did anybody else handle this situation?

[nodkrot]

So is this going to be addressed?

[johnjbarton]

Based on all of the information posted here, the issue is not something karma-runner can fix. The runner itself is not causing memory overflow but some combination of webpack and sourcemaps. It looks like @stevemao has a workaround.

[nosus]

to avoid the above comments confuse people in the future
karma start karma.conf.js --max_old_space_size=4096 does NOT work
and
node --max_old_space_size=4096 node_modules/karma/bin/karma start works

At least for me.