Rocket.chat: Address use of connect.session() MemoryStore

[saplla]

Description:

Upon starting Rocket Chat, a warning is printed to the console:

Warning: connect.session() MemoryStore is not
designed for a production environment, as it will leak
memory, and will not scale past a single process.

This should be addressed or at least provied a link to documentation for how to remove it for a production deployment.

Server Setup Information:

• Version of Rocket.Chat Server: rocketchat/rocket.chat:0.64.1
• Operating System: linux
• Deployment Method(snap/docker/tar/etc): docker
• Number of Running Instances: 1
• DB Replicaset Oplog:
• Node Version:
• mongoDB Version:

Steps to Reproduce:

1. node main.js
2. Read console

Expected behavior:

Not to see this error or be shown how to make it production-ready

Actual behavior:

Relevant logs:

[flayks]

👍

[Kanozis]

Someone have a solution?

[YuhuaDeng]

same issue +1

[kattukj002]

Same issue. I'm trying to move Rocket.Chat to production, but I'm concerned about this issue, is there any way to address it?

[apiontek]

I'm guessing some way to make rocket.chat use, for example, redis, would be the way to go, but not sure if it's possible?

[evaso]

Same issue. How can we fix this? What should we do to do not have any memory leaks? Is the using a different session store (MemoryStore), for example connect-mongo helpful?

[stickinyourhead]

Same issue. Memory leaks aren't acceptable. I had it running for a while and had to manually reboot my AWS instance.

[craymichael]

This looks like something relating to express-session, MemoryStore is the default session storage and is ONLY meant for debugging purposes. To mitigate this message, the store option for the session must be set to one meant for production environment, such as any of these compatible session stores. A Rocket dev should look into this (I believe the session created in this file is the culprit).

[faust64]

somewhat related: https://github.com/keystonejs/keystone/issues/3478#issuecomment-333564259

[reyalpsirc]

This is open since May and no one from Rocket Chat answered it?

[rezakamalifard]

+1

[TheNullPlayer]

They should use connect-mongo. https://www.npmjs.com/package/connect-mongo
Implementing it is easy.

[maurice-does-software]

Implementing it is easy.

Then @TheNullPlayer surely it's easy to describe as well? What file needs modifying? And how? The linked documentation is incredibly sparse. @craymichael can you briefly explain how to set a different store for those who are not intimately familiar with the workings of express?

[faust64]

@maurice-does-software : as a PoC, I used to create a lib allowing me to switch express session backends (https://github.com/faust64/somesession)

Assuming a store such as redis, initializing express would go like that:

const express = require('express');
const sessions = require('express-session');
const store = require('somesession');
const config = {
    driver: 'redis',
    database: 0,
    expires: 86400,
    host: '127.0.0.1',
    port: 6379,
    prefix: 'sessions:',
    secret: 'secr3t',
    secureCookie: (process.env.NODE_ENV === 'production')
};

const app = express();
const storage = store(config);

app.use(sessions({
        cookie: { secure: config.secureCookie },
        store: storage, secret: config.secret,
        resave: false, saveUninitialized: false,
        cookie: { maxAge: Math.round(config.expires) }
    }));

AFAIU, in the case of rocketchat, you'ld want to patch around there:

https://github.com/RocketChat/Rocket.Chat/blob/6e88d4d474bdd4493222d174077e5139319668d0/packages/rocketchat-grant/server/index.js#L16

https://github.com/expressjs/session/blob/master/README.md is pretty clear on that matter: as long as you do not pass a storage (which is our case here), then the default would be to use MemoryStore, which is probably not a good pick scaling out.

Now a question remains: what kind of backend would we want to use?
MongoDB seems to be the obvious answer, although I do not have much feedback on that one.

[maurice-does-software]

@faust64 thanks for identifying the location in the RocketChat app where this can be changed. The express docs and docs for various express stores assume the app.use line for initializing express, so for someone less familiar with NodeJS in general, I was having a difficult time finding that.

[faust64]

Problem is, one year after first reporting that issue, no one seems to both understand how rocket works and why in-memory sessions store are not meant to be used outside of a developer's workstation.
Which says long on how mature RocketChat is.

[maurice-does-software]

Oops. I made my commit not seeing that there already was one. Disregard...

[fabien4455]

Same problem here on 0.73.1 is it fixed ?

@maurice-does-software Is your code fix the problem ? If yes how can I put that in production to avoid memory leak ?

[moonwolf-github]

Does anyone have plans to fix this issue? It's way more important than fancy new emojis in my opinion.

[fabien4455]

Does anyone have plans to fix this issue? It's way more important than fancy new emojis in my opinion.

It's moooore important of course... but they don't care

look at this issue... 10 months

[filipef101]

@ggazzo @rodrigok Can this be fixed? There is already some code made here I think

https://github.com/maurice-does-software/Rocket.Chat/commit/ed171743f4fc6703164a66101c67d98bb7e79bc4

[maurice-does-software]

My code change seemed to fix it for the small organization that's using it. But, every few days there are memory overflow issues in mongodb that cause our instance of the app to crash, requiring monit to bring it back up. I haven't had time to look into whether those are due to my code changes or would have been happening anyway. So, a more informed, well-researched, well-tested fix from someone on the rocket.chat team would still be much better. If a little instability is ok, feel free to use my changes at your own risk.

[faust64]

@maurice-does-software : can you tell us more about your setup?

Can you confirm the symptoms you're mentioning are local to your MongoDB deployment? Is memory usage on your RocketChat container consistent with what you used to see?

Can you give us your:

• version of nodejs
• version of mongodb
• version of RocketChat you used to have prior applying your patch/what are we comparing to?
• memory allocations for mongodb/rocket
• average amount of active users seen on your installation

Note there's currently an open case regarding memory leaks:

• https://github.com/RocketChat/Rocket.Chat/issues/12880

Any graph that would enlighten your issue?
If not, I could recommend you with netdata (https://github.com/netdata/netdata). Would not write metrics to disk, doesn't involve any central server collecting metrics, ... just start it when you want to troubleshoot your system, ...

[sgohl]

If this would ever be "fixed", I'd prefer to have the option at least between mongo and redis. I think many users want to use mongo just because it's already there, but I think redis is actually the better choice, since session-data will not blew up mongo and can easily be flushed.

Still no one at least a dirty hack?

[mhow2]

I just started a rocket.chat docker image to try out and noticed this message which sounds a bit risky... I'm interested to have a clean installation too.

[engelgabriel]

The MemoryStore warning regarding memory leak only really applies if you enable our GraphQL experimental feature. Normal operations of the server are not affected, as we don't use ExpressJS for any other APIs. Having said that, we are planning to remove the GraphQL implementation from the main codebase and leave it to be added later as an auxiliary Apolo base project.

https://github.com/RocketChat/Rocket.Chat/issues/14959

[Aschay]

They should use connect-mongo. https://www.npmjs.com/package/connect-mongo
Implementing it is easy.

the problem persists

[sampaiodiego]

it is gone since version 2.0.0