Webpacker: How to run node with flag --max_old_space_size

[kopylovvlad]

After starting webpack-dev-server I have running process like 'node node_modules/.bin/webpack-dev-server --progress --color '
How to set node proccess flag --max_old_space_size=?
I want it 'cos sometimes I have an error "JavaScript heap out of memory"

[dahal]

@kopylovvlad did you come up with the solution? Me and @Praneeta are going through the same here.

[gauravtiwari]

You can set it like so:
package.json:

{
  "scripts": {
    "webpacker": "node --max_old_space_size=4096 node_modules/.bin/webpack-dev-server --progress --color --config config/webpack/development.js"
  }
}

yarn webpacker

[inkhrt]

I have not.
On 19 Jan 2018 20:58, "Puru Dahal" [e-mail] wrote:

@kopylovvlad https://github.com/kopylovvlad did you come up with the
solution? Me and @Praneeta https://github.com/praneeta are going
through the same here.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rails/webpacker/issues/1189#issuecomment-359073240,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Ah9VyM3n4bPKs0AU8fuN9RXBI2yGpmkSks5tMPPqgaJpZM4RjVV7
.

[kopylovvlad]

@dahal I have found this weird solution:
1) open file node_modules/.bin/webpack-dev-server
2) change first line '#!/usr/bin/env node' to '#!/usr/bin/env node --max_old_space_size=4096'

I know it is weird solution, but it works for me 😅

[gauravtiwari]

@kopylovvlad The problem with that solution is every time you will run yarn or yarn install - it will overwrite binaries.

[Praneeta]

@kopylovvlad we settled for a similar situation, sudo npm install -g increase-memory-limit
But as @gauravtiwari suggested it's temporary and I am worried will cause trouble when deploying to prod.

[gauravtiwari]

Perhaps webpacker can offer an option through binstubs to set memory size on demand.

./bin/webpack-dev-server --max_old_space_size 2048
./bin/webpack --watch --max_old_space_size 20148

---

Another working option is to set NODE_OPTIONS env variable like so:

export NODE_OPTIONS="--max-old-space-size=2048"

[kopylovvlad]

@gauravtiwari
./bin/webpack-dev-server --max_old_space_size 2048
It is does not work. I see an error 'Unknown switches '--max_old_space_size''

[gauravtiwari]

Oh yes, I was just saying that may be we can offer that option out of the box (later).

[Praneeta]

Adding this to package.json worked.

"scripts": {
    "webpacker-dev": "node --max_old_space_size=4096 node_modules/.bin/webpack-dev-server --progress --color --config config/webpack/development.js --port 3002"
  }

Then run NODE_ENV=development yarn webpacker-dev. It needed the node env.
But then I ran into other issues where javascript_pack_tag was not getting the correct port.

But instead I made an edit to my ./bin/webpacker-dev file and that seems to do better

before:

newenv = {
  "NODE_PATH" => NODE_MODULES_PATH.shellescape,
  "ASSET_HOST" => DEV_SERVER_HOST.shellescape
}.freeze

cmdline = ["yarn", "run", "webpack-dev-server", "--", "--progress", "--color", "--config", WEBPACK_CONFIG] + ARGV

Dir.chdir(APP_PATH) do
  exec newenv, *cmd
end

My change

newenv = {
  "NODE_PATH" => NODE_MODULES_PATH.shellescape,
  "ASSET_HOST" => DEV_SERVER_HOST.shellescape
}.freeze

Dir.chdir(APP_PATH) do
  exec newenv, "node --max_old_space_size=4096 node_modules/.bin/webpack-dev-server --progress --color --config #{WEBPACK_CONFIG}"
end

And then you can continue running your server as ./bin/webpack-dev-server and javascript_pack_tag has the right port.

[Praneeta]

@gauravtiwari do you suspect that the memory limit issue could also affect asset precompile?
Trying to run NODE_ENV=production RAILS_ENV=development bundle exec rake assets:precompile --trace locally
It basically hangs at [Webpacker] Compiling assets 🎉
I cannot really tell what it's doing then.
I checked my output path packs. The files are generated but are completely empty.
Any suggestions?

[kengreeff]

Hi Guys, how would you do this for assets:precompile as mentioned above? We run this after our tests before building our docker image and are seeing crashes due to running out of memory as well on a server with 8gb ram.

[kopylovvlad]

@kengreeff Does folder app/javascript/packs contain only webpack entry files? It should not contains all js-files.
My colleague put all js-files there and 8gb was not enough for compiling. After, he moved all files except webpack entry files.

[gauravtiwari]

Thanks @kopylovvlad

Yes, please make sure only webpack entry files are there i.e. the JS files you are referencing in your Rails views. Rest of the files can go directly under app/javascript

app/javascript: 
  packs: 
     - only packs # i.e. one directly linked in the views
  src: 
     - components
     - images
     - styles 
     - etc. 
  test: 

Also, make sure there is no circular dependency inside packs folder.

[Praneeta]

@kengreeff - just throwing this out there - In our case there were a lot of files which were served from packs but they had a lot of common code. Using the common chunks plugin really helped. Check this thread -> https://github.com/rails/webpacker/issues/1217

[kengreeff]

Hey guys, thanks for getting back to me so quickly!

We only have entry files in there, but our app is quite large. I can get it to compile with node --max_old_space_size=4096 node_modules/.bin/webpack-dev-server --progress --color --config config/webpack/development.js

I monitored the memory use while compiling and it looked like it was topping out when hitting around the 1500mb mark which is the default I believe.

On a positive note, I spent yesterday migrating from Webpacker 2/Webpack 3 to Webpacker 4/Webpack 4 and have seen that it compiles in almost half the time and can succeed without increasing memory.

We just need to a solution while we test everything and make sure the new branch is ready for production.

I will investigate a way and see if I can alter the node config on the test servers.

[vedeshkm]

export NODE_OPTIONS="--max-old-space-size=2048"

How to set "export NODE_OPTIONS="--max-old-space-size=2048"" and where to set it?

[kengreeff]

@vedeshkm you can set it as an environment variable or add the option to the command as above --max_old_space_size=4096

The name of the env var is NODE_OPTIONS and the value is --max-old-space-size=2048

[stoplion]

Even with this set, I'm running out of memory like 90% of time..

[BassamAziz]

i only get this in production, deploying on heroku. would that still be the fix? and where to set that max-old-space-size in that case?

[wildjcrt]

@BassamAziz I provide another way here but not sure it works on heroku or not. https://github.com/rails/webpacker/issues/2033#issuecomment-486053914

[darewreck54]

No matter what I do, I can never get that 2048MB memory limit to change. I've even tried to set the NODE_OPTIONS= --max_old_space_size=4096

Any advice on how to get that to change.

> node --max_old_space_size=4096 node_modules/.bin/react-scripts-ts build

Creating an optimized production build...
Starting type checking and linting service...
Using 1 worker with 2048MB memory limit

Thanks,
Derek

[iamsoorena]

@darewreck54 did you find a solution? I'm struggling with this too.

[vcavallo]

https://github.com/rails/webpacker/issues/1189?_pjax=%23js-repo-pjax-container#issuecomment-404082851

Yes, please make sure only webpack entry files are there i.e. the JS files you are referencing in your Rails views. Rest of the files can go directly under app/javascript

app/javascript: 
  packs: 
     - only packs # i.e. one directly linked in the views
  src: 
     - components
     - images
     - styles 
     - etc. 
  test: 

Also, make sure there is no circular dependency inside packs folder.

this is the first time i've seen this suggestion (maybe i missed it elsewhere) and it has proved to be very valuable!

[jakeNiemiec]

👆I can't promote this enough. If you are running out of memory, webpacker is doing more work than it needs to be doing. FA is a great example of this: https://fontawesome.com/how-to-use/with-the-api/other/tree-shaking#issues

You can take things even further by using dynamic imports (aka split chunks) from just one pack file https://webpack.js.org/guides/code-splitting/#dynamic-imports

Hopefully, we will have more/better docs on this soon: https://github.com/rails/webpacker/pull/2118

[shanesmith]

No matter what I do, I can never get that 2048MB memory limit to change. I've even tried to set the NODE_OPTIONS= --max_old_space_size=4096

@darewreck54 use dashes rather than underscores for the node option =)

NODE_OPTIONS="--max-old-space-size=4096"

[cristiancpl]

Any solution ?

[madhu131313]

@cristiancpl
run the below command in terminal (12GB RAM, change as per your RAM)

export NODE_OPTIONS="--max-old-space-size=12048"

then build

[sontd-0882]

I my case, upgrade node from 8.x to 12.x and the problem gone away.

[WagnerMoreira]

is it --max-old-space-size or --max_old_space_size ? It doesn't seem to do anything as the build is still failing at 96% for me 100% of the attempts, and also gets stuck on the terserPlugin