Keystone-classic: Running Out of Memory on Small Servers

[christroutner]

I recently upgraded a KeystoneJS install to commit 17e359b434bd155fc0b6bc227cf65f9040d0a42b and keystone is getting killed by the Linux OS for running out of memory. I've been running is successfully on a small Digital Ocean $5 Droplet with 512MB memory before this. Here is a pastebin of my output showing my current commit installation and the out of memory error:
http://pastebin.com/bdNU8re6

Is this a concern to the project? Why is KeystoneJS growing larger?

Are there any benchmarks for memory usage of previous versions?

Is there a memory usage target for v4.0?

[jstockwin]

@christroutner I think that's just the nature of node.

I run keystone on $5 DO droplets, you just need to add some SWAP.
https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-14-04

[christroutner]

Great tip, @jstockwin. Thanks!

[peterpme]

Hey @christroutner,

SWAP files are cool, but you might be able to get away with compiler flags. Compiler flags allow you to optimize your application for your environment.

Node uses a very lazy and greedy garbage collector.

For a 512mb digital ocean box, you can try running the following:

node --optimize_for_size --max_old_space_size=460 --gc_interval=100 keystone.js

[abale]

Another suggestion that can work in addition to either suggestion above is using a nodejs process manager (good practice anyway) such as https://github.com/Unitech/pm2 (in particular, http://pm2.keymetrics.io/docs/usage/process-management/#max-memory-restart) which can trigger a restart if memory usage goes over a specified size. This can help with memory leaks. It can also help you keep totals (and logs) of how many times this has happened.

for example:

pm2 start keystone.js --node-args="--optimize_for_size --max_old_space_size=460 --gc_interval=100" --max-memory-restart 475M

you can monitor realtime cpu/memory usage with pm2 monit and see restart counts by pm2 list

[mhoffmeyerDC]

Using a caching reverse proxy can ease the CPU and memory load too! I did it with keystone and Nginx. Example nginx.conf, also see Nginx Caching Guide.

[htor]

Enabling swap is always a good idea on low-memory boxes for building and running node apps. I often see npm eating a lot of memory and this sometimes crashes my deployment build on the server.

Also, make sure you run node with NODE_ENV=production node keystone.js and disable in-memory session storage with keystone.set('session store', 'mongo'); in keystone.js.

[jstockwin]

Since this isn't really a keystone issue, I'm going to close it to keep the tracker unpolluted.
Thanks for all the contributions though - hopefully people will find this helpful!