Elasticsearch: Elasticsearch fails to open if `modules` directory contains any files other than module folders.

Created on 29 Jul 2017 · 4Comments · Source: elastic/elasticsearch

Elasticsearch version: 5.5.1

Plugins installed: []

JVM version (java -version): jdk1.8.0_111

OS version (uname -a if on a Unix-like system): macOS Sierra 10.12.06

Description of the problem including expected versus actual behavior:
Elasticsearch fails to open if modules directory contains any files other than module folders.

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including
(e.g.) index creation, mappings, settings, query etc. The easier you make for
us to reproduce it, the more likely that somebody will take the time to look at it.

Download and unzip elasticsearch version 5.5.1 from official site.
Browse folders with Finder.
Run command bin/elasticsearch

Provide logs (if relevant):
[2017-07-29T00:45:33,788][INFO ][o.e.n.Node ] [] initializing ...
[2017-07-29T00:45:33,896][INFO ][o.e.e.NodeEnvironment ] [qUVdkxC] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [14.7gb], net total_space [232.6gb], spins? [unknown], types [hfs]
[2017-07-29T00:45:33,897][INFO ][o.e.e.NodeEnvironment ] [qUVdkxC] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-07-29T00:45:33,899][INFO ][o.e.n.Node ] node name [qUVdkxC] derived from node ID [qUVdkxCMSFqtAFmIXBts5g]; set [node.name] to override
[2017-07-29T00:45:33,900][INFO ][o.e.n.Node ] version[5.5.1], pid[41245], build[19c13d0/2017-07-18T20:44:24.823Z], OS[Mac OS X/10.12.6/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_111/25.111-b14]
[2017-07-29T00:45:33,900][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/Users/alpert/Downloads/elasticsearch-5.5.1]
[2017-07-29T00:45:33,924][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Unable to initialize modules
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.5.1.jar:5.5.1]
Caused by: java.lang.IllegalStateException: Unable to initialize modules
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:123) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.node.Node.(Node.java:312) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.node.Node.(Node.java:244) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:232) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.5.1.jar:5.5.1]
... 6 more
Caused by: java.nio.file.FileSystemException: /Users/alpert/Downloads/elasticsearch-5.5.1/modules/.DS_Store/plugin-descriptor.properties: Not a directory
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[?:?]
at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_111]
at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0_111]
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384) ~[?:1.8.0_111]
at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0_111]
at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:114) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.plugins.PluginsService.getModuleBundles(PluginsService.java:273) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:117) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.node.Node.(Node.java:312) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.node.Node.(Node.java:244) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:232) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:351) ~[elasticsearch-5.5.1.jar:5.5.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.5.1.jar:5.5.1]
... 6 more

Is that an expected behaviour? If not I would be happy to fix that by only looping over folders in modules directory instead of all entries.

Source

alpert

Most helpful comment

sudo find /Path/to/your/elasticsearch-folder -name ".DS_Store" -depth -exec rm {} \;

Sumaccamus on 7 Oct 2017

👍3

All 4 comments

This is intentional. We allow no leniency here, do not browse the modules and plugins folders in Finder and the .DS_Store entries will not appear.

jasontedor on 29 Jul 2017

👎1 👍1

@jasontedor I browsed the folders one time. Then deleted the files. But MacOS now keeps recreating them (though not immediately...). I'm all for discipline in general, but can we not help our customers out by ignoring hidden files or something? I mean, who would deliberately hide a plugin file, then complain about ES not finding it?

nerophon on 7 Sep 2017

sudo find /Path/to/your/elasticsearch-folder -name ".DS_Store" -depth -exec rm {} \;

Sumaccamus on 7 Oct 2017

👍3

Hi,
I've just had this issue on RHEL/CentOS 7. After upgrading from Elasticsearch 5.x (elasticsearch-5.0.0_alpha5-1.noarch) it's seem that the 6.2 rpm doesn't clean some old directories in /usr/share/elasticsearch/modules:

/usr/share/elasticsearch/modules/lang-groovy
/usr/share/elasticsearch/modules/transport-netty3

So after upgrade ES was unable to start:

[2018-02-22T19:25:16,394][INFO ][o.e.n.Node               ] [collect01] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.4QqwfVQs, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch]
[2018-02-22T19:25:16,402][ERROR][o.e.b.Bootstrap          ] Exception
java.lang.IllegalStateException: Unable to initialize modules
    at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:125) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:303) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) [elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.2.2.jar:6.2.2]
Caused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/modules/lang-groovy/plugin-descriptor.properties
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
    at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[?:?]
    at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_161]
    at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0_161]
    at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384) ~[?:1.8.0_161]
    at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0_161]
    at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:202) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:184) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.plugins.PluginsService.getModuleBundles(PluginsService.java:290) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:119) ~[elasticsearch-6.2.2.jar:6.2.2]
    ... 12 more
[2018-02-22T19:25:16,410][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [collect01.virt.linuxed.net] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Unable to initialize modules
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.2.2.jar:6.2.2]
Caused by: java.lang.IllegalStateException: Unable to initialize modules
    at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:125) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:303) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.2.jar:6.2.2]
    ... 6 more
Caused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/modules/lang-groovy/plugin-descriptor.properties
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
    at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[?:?]
    at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_161]
    at java.nio.file.Files.newByteChannel(Files.java:407) ~[?:1.8.0_161]
    at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384) ~[?:1.8.0_161]
    at java.nio.file.Files.newInputStream(Files.java:152) ~[?:1.8.0_161]
    at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:202) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:184) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.plugins.PluginsService.getModuleBundles(PluginsService.java:290) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:119) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:303) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.2.jar:6.2.2]
    ... 6 more

Best regards,
Edouard