Elasticsearch: Java Client API 5.0.0-RC1 dependes on log4j

[guenhter]

Describe the feature:
I'm using the java client api for ES 5.0.0-RC1
This depends strongly on log4j and not uses any facade like slf4j
This is a bit annoying in the application, because in our case we use logback and don't want a second logging framework included. As workaround, we use the lo4j-sl4j-bridge but IMO the ES-java-API should work with a facade like slf4j

[dadoonet]

The REST client is using:

    <dependency>
      <groupId>org.apache.httpcomponents</groupId>
      <artifactId>httpclient</artifactId>
      <version>4.5.2</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.apache.httpcomponents</groupId>
      <artifactId>httpcore</artifactId>
      <version>4.4.5</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.httpcomponents</groupId>
      <artifactId>httpasyncclient</artifactId>
      <version>4.1.2</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.apache.httpcomponents</groupId>
      <artifactId>httpcore-nio</artifactId>
      <version>4.4.5</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <groupId>*</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>commons-codec</groupId>
      <artifactId>commons-codec</artifactId>
      <version>1.10</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>commons-logging</groupId>
      <artifactId>commons-logging</artifactId>
      <version>1.1.3</version>
      <scope>compile</scope>
    </dependency>

See https://repo1.maven.org/maven2/org/elasticsearch/client/rest/5.0.0-rc1/rest-5.0.0-rc1.pom

If you are talking about the TransportClient, this one depends on elasticsearch server which uses indeed LOG4J2 in the server.
My guess is that when it will become more mature, we will remove at some point the Transport Client.

In the mean time, you can try to shade elasticsearch and relocate the LOG4J jars if you wish.
Read https://www.elastic.co/blog/to-shade-or-not-to-shade

[guenhter]

Thanks for the quick response.
As long as log4j is present, I will use log4j-over-slf4j.

[reda-alaoui]

Hello,

I encoutered the same issue.
I have the following dependencies:

<dependency>
  <groupId>org.elasticsearch.client</groupId>
  <artifactId>transport</artifactId>
  <version>${elastic.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-to-slf4j</artifactId>
  <version>2.7</version>
  <scope>runtime</scope>
</dependency>

Even with the log4j to slf4j adapter, I have this error:

Exception in thread "elasticsearch[N8GnTfl][clusterService#updateTask][T#1]" java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/config/Configurator
    at org.elasticsearch.common.logging.Loggers.setLevel(Loggers.java:149)
    at org.elasticsearch.common.logging.Loggers.setLevel(Loggers.java:144)
    at org.elasticsearch.index.SearchSlowLog.setLevel(SearchSlowLog.java:114)
    at org.elasticsearch.index.SearchSlowLog.<init>(SearchSlowLog.java:109)
    at org.elasticsearch.index.IndexModule.<init>(IndexModule.java:114)
    at org.elasticsearch.indices.IndicesService.createIndexService(IndicesService.java:402)
    at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:374)
    at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:347)
    at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45)
    at org.elasticsearch.cluster.service.ClusterService.runTasksForExecutor(ClusterService.java:555)
    at org.elasticsearch.cluster.service.ClusterService$UpdateTask.run(ClusterService.java:894)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:444)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:237)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:200)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.core.config.Configurator
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 17 more

It seems that the only way to avoid this is to add full log4j implementation to the classpath.

@dadoonet, are you saying that at some point, Transport Client will be dropped in favor of REST Client ?

[dadoonet]

Yes. You sadly need to have log4j impl in the classpath. The TransportClient is tight with elasticsearch core which needs to have this log4j impl.
One of the thing you can may be do is to shade all that to avoid conflicts with your project. This can help: https://www.elastic.co/blog/to-shade-or-not-to-shade

are you saying that at some point Transport Client will be dropped in favor of REST Client ?

This is my guess indeed. I mean that having a REST client with less deps will be a great win.
But it won't happen anytime soon as we first have to add some query builders and more feature on top of the low level REST client.

[mbechler]

Is it really necessary that you are fiddling around with the loggers like that? The only real core dependency on log4j-core seems to be setting logger levels from IndexSettings in IndexingSlowLog/SearchSlowLog. If that wouldn't be there (or fail gracefully) client users would still be able to use their respective logging backend with an adapter and without pulling in and configuring the log4j2.