Kibana: [DOCS] Document how to increase heap memory limit for Kibana

[kostasb]

Kibana version: v5.x/v6.x

Description:

How to increase Kibana's heap size limit is a common question from users who have been experiencing issues with memory pressure. Due to the frequency of the question and no publicly recommended approach until now (except for sparse references in various discuss/GH issues), this should be documented.

To the best of my knowledge the max-old-space-size option sets the heap limit in node.

This option can be set as an env variable in /etc/default/kibana

NODE_OPTIONS="--max-old-space-size=4096"

which is then passed to the node executable in /usr/share/kibana/bin/kibana:

exec "${NODE}" $NODE_OPTIONS --no-warnings "${DIR}/src/cli" ${@}

which is the script started by the systemd unit file.

Alternatively, if Kibana is started from command line as a node executable:

/usr/share/kibana/bin/../node/bin/node --max-old-space-size=4096 --no-warnings /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml

Related issues referencing this topic:

https://github.com/elastic/kibana/issues/1924

https://github.com/elastic/kibana/issues/9006

cc @jaijhala @nephel @nerophon for visibility.

[epixa]

We don’t recommend increasing the node heap size, at least anymore. Can you provide details on specific scenarios where that is necessary (including exact Kibana version numbers)?

[kostasb]

@epixa I do not have a report of a specific issue where this was really needed, but has been asked by users several times (usually for false alarms, e.g. ES issues propagating as slow or failing responses to Kibana) and there isn't a source of documentation to point at.

If our answer is that we don't recommend modifying it, we should document that it is not recommended & provide a brief justification for it.

I am not sure if someone else has real examples to share where raising the heap limit was necessary.

[jbudz]

Brain dump:

/etc/default/kibana NODE_OPTIONS won't work for syv until https://github.com/elastic/kibana/pull/15900 is merged.

In the older days node's garbage collection was less aggressive so in some cases we actually recommended limiting max-old-space-size to 512m. There was also some strange memory that we were never able to pinpoint but bounding it did seem to help. These are both gone in new versions

More recently we ran into memory issues in __development__ with the webpack bundling process, and recommended bumping it (for development). This has been fixed.

optimize takes a decent amount of memory (i.e. 512mb droplets will likely segfault) but the default max-old-space-size is sufficient.

What Court said^ all supported versions of Kibana (5.6+) shouldn't need to worry about it. I'd love data points if there's issues happening but we should treat them as bugs first and memory limits second. Here's our loaded demo server running at less than 200mb, http://demo.elastic.co/status. My server's monthly uptimes without x-pack and is sitting at 68mb.

kostasb do you know where the recommendation is coming from? Is this older versions that need to set it, or historical reasons?

[jbudz]

I'm not against documenting the NODE_OPTIONS env variable, there's probably some other useful options. Sorry about the wall of text, tldr is I want to fix bugs if there are any.

[kostasb]

@jbudz I have asked around but didn't get any actual valid cases in recent versions (v5 and v6) in which increasing max-old-space-size has fixed an issue. There may be, but I am not aware of it and didn't get inputs from others.

However the question "how can I increase heap for Kibana" does come in once in a while, typically as a "let's give it more memory" reaction or proactively during setup, and there have been no official guidelines/recommendations about, or advice against, modifying defaults.

[jbudz]

Cool, thanks for the info.

[jbudz]

Discussed at standup. We'll document memory (explicit) expectations.

[iahmad-khan]

Hi , In my case , the default is 1.4G.

How can I increase the memory size for kibana? or make kibana to handle more connections.

[tylersmalley]

Closing in favor of https://github.com/elastic/kibana/issues/9006