Elasticsearch: No logging when using layout.type = JSONLayout

[eht16]

I tried to configure log4j2 to log JSON to a file.
My config:

appender.json_file.type = file
appender.json_file.name = json_file
appender.json_file.fileName = ${sys:es.logs}-json.log
appender.json_file.layout.type = JSONLayout
appender.json_file.layout.compact = true
appender.json_file.layout.eventEol = true

rootLogger.appenderRef.json_file.ref = json_file

First of, I got a backtrace like this:

Dec 11 08:27:05 jankok elasticsearch[17255]: 2016-12-11 08:27:05,823 main ERROR Unable to invoke factory method in class class org.apache.logging.log4j.core.layout.JsonLayout for element JSONLayout. java.lang.reflect.InvocationTargetException
Dec 11 08:27:05 jankok elasticsearch[17255]:         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at java.lang.reflect.Method.invoke(Method.java:498)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:132)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:942)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:882)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:874)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:874)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:498)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:227)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:239)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:530)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:258)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:116)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:83)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:272)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:96)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.cli.Command.main(Command.java:62)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82)
Dec 11 08:27:05 jankok elasticsearch[17255]: Caused by: java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ser/FilterProvider
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.layout.JsonLayout.<init>(JsonLayout.java:814)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at org.apache.logging.log4j.core.layout.JsonLayout.createLayout(JsonLayout.java:922)
Dec 11 08:27:05 jankok elasticsearch[17255]:         ... 24 more
Dec 11 08:27:05 jankok elasticsearch[17255]: Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.ser.FilterProvider
Dec 11 08:27:05 jankok elasticsearch[17255]:         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
Dec 11 08:27:05 jankok elasticsearch[17255]:         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

After downloading http://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.8.1/jackson-annotations-2.8.1.jar and http://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.8.1/jackson-databind-2.8.1.jar to /usr/share/elasticsearch/lib, the backtraces were gone but I also get no logging in JSON.
The logfile for the JSON appender keeps empty while normal logging continues to work.

I'm not sure if I'm doing something wrong in some way or if JSON logging is simply not supported yet (maybe related to #8786).

[jasontedor]

It works no problem for if you grant permissions:

grant {
  permission java.lang.RuntimePermission "accessDeclaredMembers";
  permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
};

Sadly, Log4j 2 does not handle these permissions properly, so you have to grant the permissions to everything. We are absolutely not going to grant these permissions to everything, so us being able to support JSON out of the box will have to wait until we can contribute a patch to Log4j 2. You are free to grant these permissions at your own risk.

[eht16]

Thanks a lot for the information.

[jasontedor]

You're very welcome.

[marcbachmann]

Is there still no clean way to set up json logging for elasticsearch?