Elasticsearch: When JNA is not found, leave the warning but don't show the stracktrace

[costin]

Using a default install on windows, one gets:

16:02:51,783  WARN main elasticsearch.bootstrap - JNA not found. native methods will be disabled.
java.lang.ClassNotFoundException: com.sun.jna.Native
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:195)
    at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:43)
    at org.elasticsearch.bootstrap.BootstrapInfo.isMemoryLocked(BootstrapInfo.java:44)
    at org.elasticsearch.monitor.process.ProcessProbe.processInfo(ProcessProbe.java:139)
    at org.elasticsearch.monitor.process.ProcessService.<init>(ProcessService.java:44)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:858)
    at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
    at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
    at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:59)
    at org.elasticsearch.node.Node.<init>(Node.java:190)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:157)
    at org.elasticsearch.hadoop.EsEmbeddedServer.<init>(EsEmbeddedServer.java:47)
    at org.elasticsearch.hadoop.LocalEs.before(LocalEs.java:67)
    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
16:02:52,563  INFO main elasticsearch.node - [Leader] initialized
16:02:52,563  INFO main elasticsearch.node - [Leader] starting ..

The stracktrace is confusing - it looks like there's something fatal/wrong with Elasticsearch when that is not the case.
Considering the warning is properly displayed, there's no value in showing the stracktrace as well even more so in the logs which are user-friendly (as opposed to focused on devs).

[rmuir]

What default install? JNA works with the .zip file just fine on windows, if its not working, then there is a misconfiguration and we should understand why its not working?

[costin]

I'm not using the zip (just checked and it starts fine) but the jar to start an embedded server (to be used in integration tests as seen in the stacktrace above).
Looking at the pom, JNA is marked as an optional dependency which might be the reason why it is not being downloaded and added to the classpath. Which is fine.
And brings us to the initial report - the warning is in place and the stacktrace is just noise.

[rmuir]

But the stacktrace is necessary to determine what is wrong when it does not work: and logging should be 100% geared at elasticsearch.jar being used as a server, nothing else, since thats its purpose.

[costin]

Maybe I'm misunderstanding then the use of JNA. Since it's marked as optional and there's a warning it can't be found, my assumption was that it is just that, optional - nice to have but not mandatory. in other words, ES can run just fine without.
If that is not the case, then JNA should marked as non-optional in the POM (resulting in its download).

As for the stacktrace, in this case it indicates no memory information can be displayed since JNA was not found. But ES already knew this yet it looks like it wasn't prepared for it.

[rmuir]

As for the stacktrace, in this case it indicates no memory information can be displayed since JNA was not found. But ES already knew this yet it looks like it wasn't prepared for it.

No, the stacktrace is super valuable, it tells you what is happening here:

    at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:43)
    at org.elasticsearch.bootstrap.BootstrapInfo.isMemoryLocked(BootstrapInfo.java:44)
    at org.elasticsearch.monitor.process.ProcessProbe.processInfo(ProcessProbe.java:139)
    at org.elasticsearch.monitor.process.ProcessService.<init>(ProcessService.java:44)

So this could be avoided when "using elasticsearch.jar as something other than a server" by changing BootStrapInfo to have booleans set to false, and having bootstrap logic change them to true on success. This way any ProcessService initialization won't cause Natives.class to be loaded.

See, stacktrace is good :)

[costin]

See, stacktrace is good :)

Agreed. I was paying attention only to the ClassNotFound which was redundant considering the previous warning:

16:02:51,783  WARN main elasticsearch.bootstrap - JNA not found. native methods will be disabled.
java.lang.ClassNotFoundException: com.sun.jna.Native

It turned out it was something else related to JNA.

[jpountz]

@costin so should we close now?

[costin]

Has there been any update to address/prevent the exception from occurring? I'm not clear whether @rmuir's solution is in place or not.

[rmuir]

i havent done anything yet. it would have the disadvantage of being a little harder to follow: these booleans wouldn't be static final and would be modified/set elsewhere and hard to track where they are coming from...

[ltregan]

Just adding a testimony as a user: I admit I got confused too for a couple of hours.

I'd propose as short term option to (i) modify the warning message to "WARNING: JNA not found, native methods will be disabled. This is ok for embedded / non-server mode.", and (ii) display the full stack trace only in DEBUG mode ?

[rmuir]

I don't think we should modify this native code, its too important when running as a server.

running as a server is _all that matters_

[sverhagen]

It would be great if it were possible to suppress the stack trace, though. It's a red flag to see stack traces, even during integration tests, or wherever we would be running Elasticsearch where we don't have these native libraries available. The way it is set up now, there is no way to reconfigure _just_ that log statement, or to extend Natives in a way that let's us override this behavior (I haven't seen a way yet). I appreciate that you consider the stack trace important ("a server is _all that matters_"), so perhaps don't remove the stack trace, but change Natives in a way that allows people to change the implementation, if it is really important to these people. Thank you.

[gary-harpaz]

sverhagen Agreed. My Application now starts up with class not found exception in the logs. How many times will someone else see this log and waste time googling about it?

[eximius313]

so I'm another victim of this stack trace ;)

[huntc]

I'm now quite confused. When running as a server is JNA required? If so then you have another problem. JNA is LGPL thus violating your Apache 2 license declaration. Please clarify on the optionality of JNA and the impact on your license. Thanks.

[jasontedor]

If so then you have another problem. JNA is LGPL thus violating your Apache 2 license declaration.

@huntc JNA is dual-licensed with Apache License 2.0 since version 4.0.0 of JNA.

[huntc]

Thanks @jasontedor ! I had not noticed the dual license. I shall update our licensing tool!

[Malu44]

I'm also confused - should we use JNA for Elastic as a server? Stack trace leads to unnecessary effort.
e.g. search in google, is a relevant dependency.

[jasontedor]

I'm also confused - should we use JNA for Elastic as a server?

Yes, otherwise Elasticsearch can not mlockall and can not use seccomp (among some other features that will be disabled).

[sverhagen]

We're getting distracted here :wink: I wasn't arguing the (in-)validity of the exception. But it would be nice to suppress it during my integration tests. Any chance? :smile:

[Malu44]

Same problem here: in our jenkins-job, we're starting an elastic server for integration testing.
I've 'fixed' this by suppress the message via log configuration
<logger name="org.elasticsearch.bootstrap" level="ERROR" />

[Malu44]

Also you can put the following dependency into your POM
<dependency> <groupId>net.java.dev.jna</groupId> <artifactId>jna</artifactId> <version>4.1.0</version> <scope>test</scope> </dependency>

[jillesvangurp]

The warning is confusing as it is because it doesn't explain the problem properly, nor how to fix it and it.

Two ways to improve: 1) make JNA not optional (why is it optional to begin with?). or 2) explain how to fix it based on @Malu44's suggestion.

[s1monw]

@jillesvangurp I think it used to be LGPL (hence the optional) but it's dual licensed under the ASL 2.0 now since version 4.0.0 upwards. I think we can make it non optional and be done with it, @rmuir @costin WDYT?

[costin]

+1

[martin-g]

+1

[jillesvangurp]

For the record, I don't think optional false/true in the pom file makes any difference legally. What matters with respect to the license is what you ship in the binary (included, I assume so it's not an issue there?) or what the developers that use es download and bundle themselves (through using maven). They can always choose to exclude the dependency for legal or other reasons in their own pom file. Lgpl and apache licenses are not incompatible as far as I know, though admittedly v3 and afferror are tedious to deal with from a legal point of view.

BTW. I fully appreciate that you guys probably had lengthy internal debates on this already with your legal people and definitely don't want to reboot any of that (I had some exposure to that stuff in Nokia back in the day). Just curious what the legal reasoning is for setting optional to true.

[clintongormley]

Reopening to discuss making JNA non-optional

[nik9000]

Reopening to discuss making JNA non-optional

We just made it non-optional. It seems like we had consensus there.