Ecs: URL Category / Policy (proxy, webfilter data)

[zlammers]

Proxy/webfilter/etc device types that handle URL data will generally categorize that data. E.g., "Search Engines", "Entertainment", "Education", etc

We're missing a field to place this information, url.category or url.type. I would have suggested event.category, but that's listed as a field where a future change is intended to enumerate field values, so that wouldn't fit with device-provided parsed data.

In addition, I would suggested a url.policy or url.rule which would contain the rule number or keyword value that either allowed or blocked a request through a device. This could also go under eventitself, as a police or rule field there.

If the httpobject is a better choice for this information, that is fine as well

[webmat]

Thanks for bringing this to our attention. We'll consider this for future versions of ECS.

I like url.category best, among these choices. I'd say it's safe for you to add url.category as a custom field right now.

[vbohata]

+1 We have also logs which contains not only category, but also URL supercategory and URL class.

[vbohata]

Probably following fields should be most universal:

• url.classification.class
• url.classification.category
• url.classification.super_category (or parent.category)
• url.classification.reason (some devices publish categorization_reason, ...)

[zlammers]

@vbohata - out of curiousity, what would be an example of a URL Class?

I like the 'reason' subfield - though I wonder if this should be more generic - as multiple types of devices may kick out a 'reason' for an action, and that data could go there. Just a thought.

Also, just be my argumentative self and provide more options, 'category/sub_category' works as well as 'super_category/category' :)

[vbohata]

URL class can be General Surfing, Privacy Risk, ... Supercategory Travel, Security, ... Category Adult Themes, Games. This is the real naming example of one vendor.

For "reason" I am also thinking about generic field like "event.reason" or "event.action_reson" or "event.explanation". The question is if one event could include more reasons (like policy.reason, url.classification.reason, ...)...

Definitely category/subcategory is also option. But as category is in fact very common for multiple vendors and exporting of super_category field can be optional, one could end with just "subcategory" without "category" field. Maybe both parent.category and subcategory fields could be useful.

[vbohata]

We now even require file classification which can be implemented the same way as URL one. So what about to create reusable *.classification.[class/category/subcategory] subfields?

[ebeahan]

A fieldset has been added after this issue was opened: rule.*. Some of the fields there, such as rule.category, rule.id, and rule.ruleset, fill some of the gaps originally discussed. Also, the event.reason field has been added.

[ebeahan]

We now even require file classification which can be implemented the same way as URL one. So what about to create reusable *.classification.[class/category/subcategory] subfields?

@vbohata Do the rule.* fields also align with some of your new needs? Happy to discuss more if needed, but I suggest we start up a new issue.