Ecs: Remove either `log.original` or `event.original`

[webmat]

Both fields are almost equivalent in nature, and are potentially pretty big.

Distinction between them makes log.original sound like a temporary debugging field, whereas event.original is the field is meant to capture the original untouched event, used for determining log integrity.

I think we should keep only one, with the purpose of capturing the original untouched event, to determine log integrity. I think the other one should go away and be captured in a custom field by data sources that need it.

I've personally been going back & forth which one I think makes the most sense to keep around ultimately. So which one we deprecate for future removal is up for debate. Feedback welcome.

Remove "event.original" and keep "log.original"

Pros

• I consider log.* to be a place for low level details, and I think log.original fits well in that philosophy

Cons

• This would make it a double breaking change, in a sense. This not only removes event.original, but moves its current definition around to log.original
• Not all events are logs. E.g. capturing a third party metric and having its original event captured in log.original may sound a bit weird to users.

Remove "log.original" and keep "event.original"

Pros

• The event.original field retains its current definition
• It's the guidance I've provided most often when people asked which one to use to capture the untouched log. As such, I think Beats uses mostly event.original (to be confirmed), so removing log.original would therefore be less painful down the line.
*

Cons

• I consider event.* a place to capture higher level details (often very short fields), and having a big payload in there feels a bit out of place.

[andrewkroh]

Not all events are logs.

That bullet point clinches it for me. I think event.original is more source agnostic so ECS should keep it.

[webmat]

cc @ruflin

[ebeahan]

+1 for keeping event.original - agree it feels more appropriate when dealing with event data outside of logs.

[ruflin]

If we only keep one, +1 on keeping event.original. There is the old discussion around what to use log.original for: https://github.com/elastic/ecs/issues/127 I think it still applies but we never started to use it this way. Even if Beats or someone starts to use it in the way discussed there in the future, it does not have to be in ECS. And seeing the confusion it created having both, removing is probably the best option :-)

[enotspe]

+1 event.original

[willemdh]

+1 event.original

[SHolzhauer]

For me definitely event.original should stay. It doesn't limit usage to "log sources".

[ebeahan]

Thanks, everyone, for the input here!

An RFC proposing the removal of log.original is in progress. Any feedback or additional discussion is welcomed.