Elasticsearch: ES fails to start when assign the network.host in CentOs

[prashanttct07]

Hi Team,
I have installed the ES 5.0 as service on CentOs. When I run the default ES with default configuration , I can see ES running.

But when I run the same by modifying the network.host : 0.0.0.0 , I am unable to see ES running and while checking the status it gives an error as elastic service failed :
Nov 09 19:39:09 monitoring systemd[1]: elasticsearch.service: main process exited, code=exited, status=78/n/a
Nov 09 19:39:09 monitoring systemd[1]: Unit elasticsearch.service entered failed state.
Nov 09 19:39:09 monitoring systemd[1]: elasticsearch.service failed.

Kindly let me know if this is a bug or is there any work around for same.

[jasontedor]

I suspect that you're hitting the bootstrap checks. Have you looked in the Elasticsearch logs (/var/log/elasticsearch/elasticsearch.log)?

[prashanttct07]

attached are the logs :

[2016-11-09T19:29:46,322][INFO ][o.e.n.Node               ] [] initializing ...
[2016-11-09T19:29:46,387][INFO ][o.e.e.NodeEnvironment    ] [IJ29-46] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [17gb], net total_space [19.4gb], spins? [unknown], types [rootfs]
[2016-11-09T19:29:46,387][INFO ][o.e.e.NodeEnvironment    ] [IJ29-46] heap size [1.9gb], compressed ordinary object pointers [true]
[2016-11-09T19:29:46,388][INFO ][o.e.n.Node               ] [IJ29-46] node name [IJ29-46] derived from node ID; set [node.name] to override
[2016-11-09T19:29:46,390][INFO ][o.e.n.Node               ] [IJ29-46] version[5.0.0], pid[18426], build[253032b/2016-10-26T04:37:51.531Z], OS[Linux/3.10.0-327.22.2.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_77/25.77-b03]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [aggs-matrix-stats]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [ingest-common]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [lang-expression]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [lang-groovy]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [lang-mustache]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [lang-painless]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [percolator]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [reindex]
[2016-11-09T19:29:47,001][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [transport-netty3]
[2016-11-09T19:29:47,002][INFO ][o.e.p.PluginsService     ] [IJ29-46] loaded module [transport-netty4]
[2016-11-09T19:29:47,002][INFO ][o.e.p.PluginsService     ] [IJ29-46] no plugins loaded
[2016-11-09T19:29:48,561][INFO ][o.e.n.Node               ] [IJ29-46] initialized
[2016-11-09T19:29:48,561][INFO ][o.e.n.Node               ] [IJ29-46] starting ...
[2016-11-09T19:30:08,702][INFO ][o.e.t.TransportService   ] [IJ29-46] publish_address {172.24.93.105:9300}, bound_addresses {[::]:9300}
[2016-11-09T19:30:08,706][INFO ][o.e.b.BootstrapCheck     ] [IJ29-46] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2016-11-09T19:30:08,707][ERROR][o.e.b.Bootstrap          ] [IJ29-46] node validation exception
bootstrap checks failed
max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
[2016-11-09T19:30:08,741][INFO ][o.e.n.Node               ] [IJ29-46] stopping ...
[2016-11-09T19:30:08,792][INFO ][o.e.n.Node               ] [IJ29-46] stopped
[2016-11-09T19:30:08,792][INFO ][o.e.n.Node               ] [IJ29-46] closing ...
[2016-11-09T19:30:08,802][INFO ][o.e.n.Node               ] [IJ29-46] closed

I see the error as :
[2016-11-09T19:30:08,707][ERROR][o.e.b.Bootstrap ] [IJ29-46] node validation exception
bootstrap checks failed

And I am adding these value in yml
``` cluster.name: elk
node.name: ES_Master
network.host: 0.0.0.0

[jasontedor]

As suspected, you're failing the bootstrap checks:

max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]

You must address if this if you want to form a cluster with this node. You can find additional information the bootstrap check docs that I linked to previously.

Elastic reserves GitHub for verified bug reports and feature requests. You can obtain general help on the Elastic Discourse forum.

[prashanttct07]

Thanks , but actually running with same config as above in Ubuntu seems to be working, so was just not sure why its not working in centos

[jasontedor]

Because the maximum number of memory maps is too low, whether that be from defaults or whatever, that's the story.

[LeeDr]

@prashanttct07 I hit this too. The Elasticsearch 5.0 rpm package sets the vm.max_map_count but it isn't applied until you reboot. Please give that a try.
Or do sysctl -w vm.max_map_count=262144

[jasontedor]

Since #21507, Elasticsearch will apply this setting on package install now.

[dalenoe]

I also ran into this. Ubuntu 16.04 using latest java9.

@LeeDr had the fix that resolved this for me.