Ecs: Application Fields (e.g. app name for Endpoint Detection logs)?

[dainperkins]

I couldn't find anything regarding a field for "Application" (as in executed program on an OS, not L7 application ID of a network flow)

Application.name
Application.verison
etc...

Am I missing it?

thx
/d

[webmat]

Yes, I think it makes sense to add application and some general fields around what the application is. The people can add their own application-specific fields in that object too.

Could be used for endpoint detection, as your title suggests, but also by developers writing an ECS compatible application log directly.

• application.name - agreed
• application.version - agreed
• application.build
• application.commit

Any other ideas? Other uses cases or angles we should consider?

[ruflin]

I think what you call above application we call in ECS service. An application is a combination of multiple services. I would rather extend services then get into applications for now: https://github.com/elastic/ecs#-service-fields

[dainperkins]

Would service be the appropriate place for e.g. an alert regarding
something excel, outlook, gimp, or chrome attempted to do?

Say tracking a host.process.id, application.name when Cylance or some such
catches an application doing something malicious?

seemed like service was more geared to non user interactive, daemons, etc.

I actually just looked to see if process.id was there and found process.name,
that might fit the bill

/d

On Thu, Sep 27, 2018 at 11:17 AM Nicolas Ruflin [e-mail]
wrote:

I think what you call above application we call in ECS service. An
application is a combination of multiple services. I would rather extend
services then get into applications for now:
https://github.com/elastic/ecs#-service-fields

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/elastic/ecs/issues/126#issuecomment-425132441, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AlmSyV6r5YiHQI6pJrkEOtuW1-J1-9o5ks5ufOv4gaJpZM4W7dkC
.

[jcallaha]

I would think events related to an application should readily fit under process.*?

Would be great if ECS _process._ were fleshed out to cover everything in _system.process._ in Metricbeat.

[dainperkins]

would it be appropriate to have e.g. process namespace for os process
information, and a separate application name space for name/version/etc, or
better to collapse it all under process?

My $.02, fwiw, is that with things like "source", "process", "device", and
"event" the top level name spaces are for discrete types of entities, a
process being any executed item on a host (service, part of an application,
plugin, etc) , an application being a logical entity that could encompass
multiple processes, etc. Thats how I read the intent, and it seems
easier, but I can certainly understand if there is a concern over too large
a top level set of namespaces

Application specifics (as opposed to process specifics in the case of a
process not necessarily being indicative of a specific application/ app
version - think about all the stuff that O365 runs) could also be useful
for dynamically looking up CVE / CVSS information (either to score an event
from aCVSS perspective, populating additional context, or for linking
events to relative CVE issues)

/d

Heres an example (making up some chrome memory access issue EDR alert)

[reporting device info - cylance mgmt console]
device.hostname = cylance_mgr.mydomain.com
device.ip = 10.192.1.10
device.vendor = Cylance
device.type = EDR

[info about the event itself]
event.type = Threat
event.action = Alert
event.module = Memory Protection
event.duration = 2s
event.type = Application attempted to access protected memory
event.risk_score = 7

[details of the entity flagged in event]
host.ip = 192.168.5.17
host.name = myWorkstation.mydomain.com
host.os.platform = Mac OS x
host.os.name = High Sierra
host.os.version = 10.13.6
host.architecture = x86_64

[user involved in event]
user.name = dain.perkins
user.id = some UUID

[process involved in event]
process.name = Google Chrome Helper (or maybe its a Flash Plugin... )
process.pid = 91408
process.ppid = 90023

[application that spawned the process - presumably ppid if present]
Application.name = Chrome
Application.version = 69.0.3497.100

[Threat information]
threat.type = Memory Access
threat.tactic = Privilege Escalation
threat.descripiton = CAPEC-69
&c

On Fri, Sep 28, 2018 at 12:53 AM John Callahan [e-mail]
wrote:

I would think events related to an application should readily fit under
process.*?

Would be great if ECS process.* were fleshed out to cover everything in
system.process.* in Metricbeat.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/elastic/ecs/issues/126#issuecomment-425320645, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AlmSycKGSOpDd9LqHInRFtm3RERWDDJWks5ufatXgaJpZM4W7dkC
.

[ruflin]

@dainperkins Nice example, thank you for putting this out here. This would make a really nice use-case: https://github.com/elastic/ecs/tree/master/use-cases

As mentioned earlier, the focus of service so far is more on things like mysql, redis and this definitively brings a new perspective on what application could be.

[willemdh]

@dainperkins My colleague @2Dman is also thinking on matching Cylance logs to ECS. Are you also using Logstash to parse these logs?

Any chance you could share your filter with us?

This is what we are using now (non-ECS conform):

if [type] == 'syslog-cylance' {
    grok {
      match => [ "message", "\A%{NUMBER:cy_messageid}%{SPACE}<%{POSINT:cy_syslog_pri}>%{NUMBER:cy_version}%{SPACE}%{TIMESTAMP_ISO8601:cy_timestamp}%{SPACE}%{SYSLOGHOST:cy_sysloghost}%{SPACE}CylancePROTECT - - - %{GREEDYDATA:cy_message}\Z" ]
      add_tag => "grokked_syslog_cylance"
    }
    date {
      match => [ "cy_timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSS'Z'" ]
      timezone => "Europe/London"
    }
    if([cy_eventname] == "SyslogSettingsSave"){
      mutate {
          gsub => [ "cy_message", '"', "" ]
        }
    }
    kv {
      source => "cy_message"
      value_split => ":"
      field_split => ","
      remove_field => [ "cy_message" ]
      prefix => "cy_"
      transform_key => "lowercase"
      remove_char_key => '^a-zA-Z0-9'
    }
    mutate {
      lowercase => [ "cy_autorun", "cy_ismalware","cy_isuniquetocylance","cy_isrunning" ]
    }
  }

Imho it would be nice to grow towards a common filter for Cylance logs.

[dainperkins]

Sorry - haven'y actually had a need to write the logstash yet, was just
looking at it from the view point of getting an alert and deciding if
process.name would be sufficient or if application.[fields] would make more
sense.

FWIW I went roughly the same route with Cisco Firepower alerts, grok the
common fields, then kv the rest

Thanks
/d

On Tue, Oct 2, 2018 at 7:53 AM willemdh [e-mail] wrote:

@dainperkins https://github.com/dainperkins My colleague @2Dman
https://github.com/2Dman is also thinking on matching Cylance logs to
ECS. Are you also using Logstash to parse these logs?

Any chance you could share your filter with us?

This is what we are using now (non-ECS conform):

if [type] == 'syslog-cylance' {
grok {
match => [ "message", "\A%{NUMBER:cy_messageid}%{SPACE}<%{POSINT:cy_syslog_pri}>%{NUMBER:cy_version}%{SPACE}%{TIMESTAMP_ISO8601:cy_timestamp}%{SPACE}%{SYSLOGHOST:cy_sysloghost}%{SPACE}CylancePROTECT - - - %{GREEDYDATA:cy_message}\Z" ]
add_tag => "grokked_syslog_cylance"
}
date {
match => [ "cy_timestamp", "yyyy-MM-dd'T'HH:mm:ss.SSSSSSS'Z'" ]
timezone => "Europe/London"
}
if([cy_eventname] == "SyslogSettingsSave"){
mutate {
gsub => [ "cy_message", '"', "" ]
}
}
kv {
source => "cy_message"
value_split => ":"
field_split => ","
remove_field => [ "cy_message" ]
prefix => "cy_"
transform_key => "lowercase"
remove_char_key => '^a-zA-Z0-9'
}
mutate {
lowercase => [ "cy_autorun", "cy_ismalware","cy_isuniquetocylance","cy_isrunning" ]
}
}

Imho it would be nice to grow towards a common filter for Cylance logs.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/elastic/ecs/issues/126#issuecomment-426245134, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AlmSyZCm2ZelS_7VcOSpEsqTO2bMThRTks5ug1OqgaJpZM4W7dkC
.

[willemdh]

Imho 1 applications could consist out of multiple processes? So adding an application root Object seems like a logical choice.

[2Dman]

Are you also sure, what 'process' is (source or target)? For example, Chrome.exe, downloads an virus, that attacks an smb exploit. That would mean that chrome.exe is the source, but explorer.exe is the target. (or visa versa). I know that Mcafee gives these details.

[dainperkins]

Thats a good point as well, maybe characterize in this type of instance as
a threat or detection (I'll use threat below because its shorter)

threat.source.application: chrome
threat.source.process: chrome-helper
threat.target.application: MS Explorer
threat.target.process: explorer.exe

Or similar (on one hand it seems to be ever growing, on the other the more
the core concepts can be connected like legos the better the standard could
be)

would "process.parent.id/name" solve the multiple process tree (any process
have 1 parent) unsure if "process.child.name(s)/id(s)" would ever be
necessary?

/d

On Tue, Oct 2, 2018 at 3:20 PM 2Dman [e-mail] wrote:

Are you also sure, what 'process' is (source or target)? For example,
Chrome.exe, downloads an virus, that attacks an smb exploit. That would
mean that chrome.exe is the source, but explorer.exe is the target. (or
visa versa). I know that Mcafee gives these details.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/elastic/ecs/issues/126#issuecomment-426398186, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AlmSybtiRuT4wgNLWyD_YC63P-Y3vP9Kks5ug7yRgaJpZM4W7dkC
.

[webmat]

@willemdh About multiple processes: adding a lot of information in one ES document, while certainly possible, will make it difficult to use later on. For example a document with an array of objects describing related processes (each: pid, name, etc) would be almost unuseable in Kibana, and for ES aggregations, for example.

If you need to describe a set of processes, indexing each in its own document and making sure you set each processe's ppid will let you reconstruct the process tree in the future.

[simitt]

Side note: APM uses service instead of application; specifically renamed some time ago (apm-server#373).