Ecs: New event fields to be able to store info from log4j, log4net, nlog, ... frequently used log frameworks

[vbohata]

I propose to add several fields frequently used in Pattern Layout in log4j, log4net, nlog, ... log frameworks.

• event.logger (the most frequently used - the name of the logger that published the logging event, event.module etc. ... are not good to store this value)
• event.context (for NDC - nested diagnostic context, see https://logging.apache.org/log4j/log4j-2.1/manual/layouts.html, https://www.baeldung.com/java-logging-ndc-log4j, https://github.com/NLog/NLog/wiki/NDC-Layout-Renderer, ...)
• event.class, event.method (for name of the class and method where log event was issued, we frequently use at least one of this value). Could be also named class.name, class.method.name

[ruflin]

• logger: I would expect this to go into agent.name
• context: I think there will be separate discussion around context soon. Not sure if this fits into their.
• event.class = java class? event.method ? Could you share some examples?

[vbohata]

No, logger is not the same as agent.name, agent.name could be filebeat (or something which collects logs of some app), logger is name choosen by programmer of some java or .net app to specify which part of app logged that info. This is not module, or ... this is just logger name, you can find it in pattern layouts of specified log frameworks (https://docs.oracle.com/javase/7/docs/api/java/util/logging/Logger.html ...).
Yes class.name and class.method.name for name of the Java or .NET class and method. In our company we have a lot of custom apps and almost half of these apps are logging class name and method name at least in test env.

[ruflin]

In general I like the idea of having a definition for this. We started some discussions to add a log4j module to Filebeat and this would play into this. The part I'm not sure yet is if this would be in the scope of ECS.

Do you see the above mainly for Java apps / service or a broader scope?

[vbohata]

Almost all of our apps we need logs from are written in Java or .NET C#. They are usually using log4j, log4j2, log4net or nlog which shares most fields. We are currently gathering logs from tens of them, in future hundereds. So as these frameworks are very common (similar frameworks exists also for other programming languages) I think it should be part of ECS.

[willemdh]

We have similar requirements for a 'logger' ecs field and also 'thread_name'

[wolframhaussig]

I would like to have fields for logger and thread_name too. In any application I have seen the logging contained those fields independently from the programming language: .Net, Java, C++

[willemdh]

Is there any reason why ECS wouldn't integrate this? I have so many logs created by microservices using log4j2 that contain a thread_name and logger field.. It feels like there is a piece of the puzzle missing now.

@ruflin "logger: I would expect this to go into agent.name" => This doesn't apply as the microservice logs I'm talking about are indexed from Openshift with Filebeat. So the agent.name would be FIlebeat?

[vbohata]

As a result of https://github.com/elastic/ecs/issues/321 there is a new field event.provider which is great but there still should be a new field event.logger. To be honest all of the following fields would help us very much:

• event.provider (done)
• event.logger
• event.component (could be renamed from event.module as event.module is not much general)

[felixbarny]

With the recent additions to ECS I think we can close this issue:

• Added process.thread.name field. #517
• Added log.logger field. #521
• Added error.stack_trace field. #562
• Added log.origin.file.name, log.origin.function and log.origin.file.line fields. #563

NDC should be mapped to tags and MDC should be mapped to `labels. See also https://github.com/elastic/java-ecs-logging/blob/master/README.md#mapping