Ecs: host object considerations

[willemdh]

Hello,

Some fields I'd like to see added for the host object:

host.description
host.cluster
host.contact.domain (group of persons / users)
host.contact.person (or user)
host.customer (customer for this host)
host.domain (domain this servers has joined, eg ad domain)
host.engineer (engineer responsible for this host)
host.environment (QA/DV/PR/DR/ST)
host.installation_date
host.os.build_nr (eg 10.0.14393) (different then host.os.version which would be Server 2016 in this case)
host.os.service_pack
host.site.primary

Those are just some fields I will be using. I'll leave it up to Elastic to decide which ones they think could be useful to add to ECS.

Grtz

Willem

[ruflin]

Thanks for the suggestions. I wonder if we could find a place to have all these suggestions in which would make it easier also to see what other people use. If we see then some fields requested / used multiple times we can add them to ECS. Need to think more about how we could do that. Any suggestions welcome.

[MikePaquette]

Here's a re-statement of the list above with a few more suggestions for additions to the host object. Perhaps for now, just post a copy of this list in a new comment and tick the boxes you feel should be added to ECS?

• [ ] - host.sub_type - sub-type of host that generated the log { "Intel" , "vmware" , "docker" , "d2.xlarge" , ...}
• [ ] - host.provider - For use in cloud environments {"AWS" , "GCP" , "Azure" , ...}
• [ ] - host.availability_zone - For use in cloud environments { "europe" , "us-sw" , "us-mexico" , "ap-1" , ... }
• [ ] - host.region, host.site.primary - For use in cloud or other environments { "europe" , "us-sw" , "us-mexico" , "ap-1" , ... }
• [ ] - host.network_interface - identifier of the network interface associated with the event. {"en0" , "en1" , "en6", "eth0" , ...}
• [ ] - host.description - text description of the host
• [ ] - host.cluster - presumably the elasticsearch cluster if the host is a cluster node??
• [ ] - host.contact.domain - (group of persons / users)
• [ ] - host.contact.person - (or user)
• [ ] - host.customer- (customer for this host)
• [ ] - host.domain - (domain this servers has joined, eg ad domain) Does host.name cover this?
• [ ] - host.engineer - (engineer responsible for this host)
• [ ] - host.environment - (QA/DV/PR/DR/ST)
• [ ] - host.installation_date- date of deployment or istantiation
• [ ] - host.os.build_number - (eg 10.0.14393) (different then host.os.version which would be Server 2016 in this case)
• [ ] - host.city - host location city information pulled from GeoIP or other CMDB, etc.
• [ ] - host.country - host location country information pulled from GeoIP or other CMDB, etc.
• [ ] - host.address - host location physical address information pulled from CMDB, etc.

[webmat]

I'm not a fan of host.engineer, which implies a single point of failure, in being about one person. Perhaps we should find a more generic name (where people can put one person's name if they like, or a team name)?

What about host.responsibility?

[ruflin]

Note for availablitity_zone etc. I think we have it covered here: https://github.com/elastic/ecs#cloud

[willemdh]

• [x] host.sub_type - sub-type of host that generated the log { "Intel" , "vmware" , "docker" , "d2.xlarge" , ...}
• [x] host.provider - For use in cloud environments {"AWS" , "GCP" , "Azure" , ...}
• [x] host.availability_zone - For use in cloud environments { "europe" , "us-sw" , "us-mexico" , "ap-1" , ... } - [x] host.region, host.site.primary - For use in cloud or other environments { "europe" , "us-sw" , "us-mexico" , "ap-1" , ... }
• [ ] host.network_interface - identifier of the network interface associated with the event. {"en0" , "en1" , "en6", "eth0" , ...} => As a host can have multiple network_interfaces and for me personally a network_interface name doesn't have to be associated with a host
• [x] host.description - text description of the host
• [x] host.cluster - presumably the elasticsearch cluster if the host is a cluster node?? For me this mainly applias to MS Windows Failover clusters
• [x] host.contact.domain - (group of persons / users)
• [x] host.contact.person - (or user) => User seems more in line with other ecs fields
• [x] host.customer- (customer for this host)
• [x] host.domain - (domain this servers has joined, eg ad domain) Does host.name cover this? host.name does not cover this imho, as host.name should imho be the shortname. Maybe there should be another field host.fqdn, which should be equal to "%{host.name}.%{host.domain}"?
• [x] host.engineer - (engineer responsible for this host) =>host.responsibility works for me as well
• [x] host.environment - (QA/DV/PR/DR/ST)
• [x] host.installation_date- date of deployment or istantiation
• [x] host.os.build_number - (eg 10.0.14393) (different then host.os.version which would be Server 2016 in this case)
• [x] host.city - host location city information pulled from GeoIP or other CMDB, etc.
• [x] host.country - host location country information pulled from GeoIP or other CMDB, etc.
• [x] host.address - host location physical address information pulled from CMDB, etc.

[kinghuang]

• [x] - host.sub_type - sub-type of host that generated the log { "Intel" , "vmware" , "docker" , "d2.xlarge" , ...}
• [ ] - host.provider - For use in cloud environments {"AWS" , "GCP" , "Azure" , ...}
• [ ] - host.availability_zone - For use in cloud environments { "europe" , "us-sw" , "us-mexico" , "ap-1" , ... }
• [ ] - host.region, host.site.primary - For use in cloud or other environments { "europe" , "us-sw" , "us-mexico" , "ap-1" , ... }
• [ ] - host.network_interface - identifier of the network interface associated with the event. {"en0" , "en1" , "en6", "eth0" , ...}
• [ ] - host.description - text description of the host
• [x] - host.cluster - presumably the elasticsearch cluster if the host is a cluster node??
• [ ] - host.contact.domain - (group of persons / users)
• [ ] - host.contact.person - (or user)
• [ ] - host.customer- (customer for this host)
• [ ] - host.domain - (domain this servers has joined, eg ad domain) Does host.name cover this?
• [ ] - host.engineer - (engineer responsible for this host)
• [x] - host.environment - (QA/DV/PR/DR/ST)
• [ ] - host.installation_date- date of deployment or istantiation
• [ ] - host.os.build_number - (eg 10.0.14393) (different then host.os.version which would be Server 2016 in this case)
• [ ] - host.city - host location city information pulled from GeoIP or other CMDB, etc.
• [ ] - host.country - host location country information pulled from GeoIP or other CMDB, etc.
• [ ] - host.address - host location physical address information pulled from CMDB, etc.

I'm mostly interested in the host.cluster field. Most of my new hosts are part of a cluster. Related to that, a field for the node group within the cluster would be useful too (host.node_group).

[webmat]

@kinghuang Thanks for the feedback!

I agree support for concepts related to clusters would be useful.

What kind of information related to clustering would be helpful for you to have in your events?

[kinghuang]

I'm mainly dealing with Docker Swarm and K8s clusters. The EC2 instances have tags with the cluster name and node group that each instance belongs to.

Perhaps this doesn't belong under host fields, and a dedicated set of cluster fields would be better. Beyond the cluster name and node group, I'd be interested in also knowing the cluster type (e.g., swarm, kops, eks, ecs).

[ljufer]

Hi,

i was looking for the above mentioned information/discussion related to ECS; Looks like it's still tbd. The fields mentioned earlier seems to be a good start. _Always possible to "not populate" the information and don't use the fields._

My input for this discussion:

Especially useful for Compliance/Security driven reporting (which i often face) is information related to the "state" a system such as:

• [X] host.environment (P/Q/T/D or PROD/QUAL/TEST/DEV or ...)

For the "type, sub_type" ... i usually utilize multiple levels of information thus allowing easier/more granular reporting later.

• [ ] host.cmdb.class (e.g. server, client, firewall, router, switch, loadbalancer, hypervisor, storage, printer, iot, ... quiet generic)
• [ ] host.cmdb.type (windows, linux, aix, vmware, fortigate, cisco, netapp, thinclient, ... )
• [ ] host.cmdb.subtype (windows server, windows client, centos, rhel, suse, fortios, ciscoios, f5bigip, kvm, esxi, ... )

_REMARK: The .cmdb. would define where the information came from. As it is often "wrong", would be good to see i it differs to e.g. "host.os". But no idea if this should be part of a standard schema or if this just "Bloats" the data._

Additional information usually very handy for reporting and analytics.

• [ ] host.zone (e.g. red-zone, orange-zone, yellow-zone, green-zone or dmz-i, dmz-e, lan or ....)
• [ ] host.compliance (e.g. PCI [CDE|Connected|OutOfScope], SOX[Y|N], CIS (L1|L2) ...) _Most likely multiple values like "tags" would be required as many systems have to comply to different standards._
• [ ] host.exposure (e.g. internet, 3rdparty, trust, internal, ...)
• [ ] host.risklevel (e.g. high, medium, low or numbers ?)

_REMARK: I'm also not a fan of "host.engineer" and prefer something more "open" like the mentioned "host.responsibility" (we call it often "owner" which can be the "technical" owner such as an engineer or the responsible person/team for the Systems like: Security, Ops, NetworkOps, ....)_

_REMARK : Also other products have these types of "normalization" often already in place (e.g. splunk CIM or QRadar SIM data models) maybe it is worth checking there what they use._

I'm new to this ... so no idea if this "braindump" is actually helpful :)

[webmat]

@ljufer Braindumps are always useful, thank you for sending this along! But we can't answer all of them in detail (or in a timely fashion, as this later answer demonstrates) ;-)

Rest assured that we will consider the ideas discussed here, when working on inventory classification and risk profiling.

[willemdh]

It would be nice if at least some of the fields requested in this issue would be added in ECS asap. In particular I would love a destinationto put the environment abbreviations, such as ST, QA, PR, DV

What do you all think of using host.environment for that? And what is your opinion on using abbreviations for the value? Should we have a separate field for the abbreviations and one for the full name? Such as:

host.environment.name
host.environment.code

Adding this will enable me to make long lasting ml jobs for production systems only. Now a lot of anomalies are coming from development and QA systems, which makes ml very noisy.

Tx!

[webmat]

Yes, we haven't been working much on inventories and classification, but it's something we have in mind to look at.

In the meantime please capture this under labels or use another custom field to classify your hosts. You should never wait after ECS ;-)

[vbohata]

I noticed new orchestrator.cluster.* fields. I think the cluster.* (or similar named) fields should be reusable object nested under orchestrator, host and maybe other fields because there can be clusters in multiple layers - k8s clusters, app clusters running inside ... or in appliance world - clusters of device firewalls, ...

[kgeller]

This discussion has been stale for a while, and we're going to close it.

@vbohata feel free to open another issue if you want to discuss further.

[willemdh]

Euh, why close this? Just because the discussion went stale....? There clearly are a lot of unresolved host object things that should get cleared up...

[kgeller]

@willemdh There are definitely a lot of fields here that could be beneficial in ECS, however a good deal of them are now included since this issue arose.

We have the geo, os, and user fieldsets, as well as additional host fields that cover many of the ideas listed above. I think it's beneficial to re-capture those in fresh issues, just like this issue to add environment information.

And just because this issue is closed, doesn't mean we can't reference anything here in the future. If there is anything else here that you think should be revisited, don't hesitate to create another issue or even an RFC.