Botpress: [CRASH/FEATURE] Language server should check RAM before overuse

[JustusNBB]

Is your feature request related to a problem? Please describe.
The language server uses a lot of RAM, even more with multiple languages installed (1GB per language by default).
It crashes without any error messages if there is not enough available.

Describe the solution you'd like
Please don't have it just crash when out of RAM, put a meaningful error message there.

In my case the solution was to increase the RAM docker is allowed to use on my workstation.

Describe alternatives you've considered
Add a warning into the boot log of the language server if a RAM check is too much effort.

Additional context
I had an interesting adventure trying to get multiple languages to work with the trial pro license, I received today.
The language server crashed Killed while rebooting trying to load the second language (using 100dim models).

Confusingly I also had getaddrinfo ENOTFOUND ny3.digitaloceanspaces.com errors in the logs.

[JustusNBB]

Also, related question: How do you debug the language server?

[franklevasseur]

Hi @JustusNBB!

Thank you for taking the time to open this issue. It is indeed a pain-point for me too.

The reason why you don't have more detail when your server crashes is that the lack of memory occurs in a node-binding (either node-fasttext or node-sentencepiece). A node-binding is a C/C++ compiled library and is not executed by the node VM so the usual node debugging information about memory consumption is not available when the server crash.

For your second question; to debug the lang server, add debug configuration in file launch.json (You can copy and paste the Debug App configuration and add the cli argument lang).

I hope this answers your questions!

François

[Ierezell]

@franklevasseur what do you think about adding something with

process.memoryUsage() // Get node current usage
os.freemem()                  // Get free ram still available
os.totalmem()                 // Get total machine ram capacity

before launching the server.

Check if there is languages * 1.5Gb free else print a big red warning ?

I can do it if needed

[JustusNBB]

I've read for 300dim languages even 4-6GB of RAM are needed in production (4-6GB per language, 1GB per bot, 4GB botpress itself).
Such a warning would be very helpful especially if the RAM might get exhausted not on boot but potentially later. Maybe it should also include these recommendations for production!

[franklevasseur]

@Ierezell nice solution here.

This should work, although this is not really a priority. You can try out your idea and submit a PR, but ultimatly we, won't be updating our public lang server instance before holidays.

Still the message would be available for users with local language servers.

[Ierezell]

Hi @JustusNBB, I just created a PR for this problem.
However, Ram usage is approximative and without getting technical it will be just a warning.

The lefover memory will always be estimated lower than the actual so even with a warning it could pass.

[JustusNBB]

Great work, you already implemented exactly what I had in mind! Left a comment in the PR :D

[Ierezell]

Thanks, glad that it solve what you wanted. Let's talk about the implementation on the PR then. It still need a bit of love.

[franklevasseur]

The PR was merged. @JustusNBB This should at least warn you on why you are experiencing issues.

Thanks for your contribution.

Feel free to reopen this issue if you feel I've closed it to fast

François