Ecs: Flesh out support for multiple users in an event

[webmat]

We need to flesh out support for multiple users in an event to ECS, when there is a semantic relationship between them. Here are some use cases (please comment if you see additional use cases):

1. User A manages User B (create, modify, delete)
2. Directory User A logs into a machine as User B
3. User A escalates privileges / assumes identity of User B

This has been discussed a bit in the past in #234 , but no concrete plans were made at the time.

ECS currently supports source.user and destination.user, which may be enough to satisfy the requirements for no 2 above. However since source and destination are both designed to represent each side of a network connection, the semantics may be problematic when trying to use source.user and destination.user for no 1 and 3, as these can be purely local events.

Questions

Has anyone used source.user and destination.user in ECS to model these use cases? If so, did this work well, and are there edge cases where it doesn't work as well?

Should we try to model all cases with one naming recommendation for all use cases? As an example, should we always use user & affected.user. Or should we adjust the naming depending on semantics? E.g. 1) user & affected.user, 2) source.user & destination.user, and a different way for 3).

Points to consider

• Some schemas use the word "target" to describe the affected user. However the word also has the connotation of sounding like a victim, or the goal of an investigation. It may be a good idea to avoid using this word if we can.
• It may be a good idea to define the non-nested user as the user initiating the action in an event, rather than nesting both sides. This way searching for user.name:bob would return any events where "bob" was doing something. Searching for events affecting "bob" would require searching for the more qualified field name, like e.g. affected.user.name:bob. Pivoting to find anything related to "bob" can somewhat be done in Kibana by searching for *.name:bob (not supported via the ES API), or could be done by populating related.user.name systematically in all sources, then searching for related.user.name:bob.

Naming ideas

Please comment with additional naming ideas.

• user & affected.user
• user & target.user
• subject.user & target.user (like Windows)
• user & assumed.user (privilege escalation)
• ... (your suggestions)

[webmat]

cc @neu5ron, @eternalyperplxed, @vbohata, @willemdh

Your thoughts on this would be appreciated, as always :-)

[webmat]

cc @rw-access if you can comment, or cc the right Endgamer to chime in for this discussion :-)

[willemdh]

Personally I think for clarity, it would be good to nest all user related data under user.*

user.name should always contain the most relevant user data.

Found an example logon event where 2 different user names are mentioned:

     "provider_name": "Microsoft-Windows-Security-Auditing",
      "record_id": 1530007,
      "provider_guid": "{54849625-5478-49554-a5ba-3e3b0328c30d}",
      "activity_id": "{a16e1baf-7f5a-0011-0b1c-6ea15a7fd501}",
      "event_data": {
        "TargetLogonGuid": "{55910e0f-a1ab-2985-9c09-cb2595029867}",
        "TargetInfo": "cifs/myserver.mydomain",
        "SubjectUserSid": "S-1-5-21-171585296-39277855-1598175747-29326",
        "LogonGuid": "{00000000-0000-0000-0000-000000000000}",
        "SubjectUserName": "mynormaluser",
        "SubjectLogonId": "0x18ca3a",
        "TargetServerName": "myserver.mydomain",
        "SubjectDomainName": "MYDOMAIN"
      },
      "event_id": 4648,
      "api": "wineventlog",
      "channel": "Security",
      "keywords": [
        "Controle geslaagd"
      ],
      "opcode": "Info",
      "task": "Logon",
      "computer_name": "myworkstation.mydomain"
    },
    "event": {
      "created": "2019-10-18T11:59:48.689Z",
      "kind": "event",
      "code": 4648,
      "action": "Logon",
      "type": "authentication_success",
      "category": "authentication"
    },
    "source": {
      "port": 445,
      "ip": "10.10.10.2"
    },
    "ecs": {
      "version": "1.0.1"
    },
    "log": {
      "level": "informatie"
    },
    "user": {
      "name": "myadminuser",
      "domain": "MYDOMAIN"
    },
    "process": {
      "name": "null",
      "pid": 4
    }
  }
}

Original event message:

Poging tot aanmelden met expliciete referenties.

Onderwerp:
    Beveiligings-id:        S-1-5-21-171585296-394481855-1598175747-29326
    Accountnaam:        mynormaluser
    Accountdomein:      MYDOMAIN
    Aanmeldings-id:     0x18CA3A
    Aanmeldings-GUID:       {00000000-0000-0000-0000-000000000000}

Account waarvan de referenties zijn gebruikt:
    Accountnaam:        myadminuser
    Accountdomein:      MYDOMAIN
    Aanmeldings-GUID:       {55910e0f-a1ab-2985-9c09-cb2445029867}

Doelserver:
    Naam van doelserver:    myserver.MYDOMAIN
    Aanvullende gegevens:   cifs/myserver.MYDOMAIN

Procesgegevens:
    Proces-id:      0x4
    Procesnaam:     

Netwerkgegevens:
    Netwerkadres:   10.10.10.2
    Poort:          445

Deze gebeurtenis wordt gegenereerd wanneer een proces probeert zich op een account aan te melden door expliciet de referenties van die account op te geven. Meestal gebeurt dit in batchconfiguraties zoals geplande taken, of bij gebruik van de opdracht Uitvoeren als.

In the above example, user.name is populated with myadminuser, which comes from winlog.event_data.TargetUserName

But as you can see this was initiated from my workstation which runs under users mynormaluser which is now in winlog.event_data.SubjectUserName.

Imho it won't be easy to find a solution which match all possible cases where multiple users are in 1 event. We could put winlog.event_data.SubjectUserName in user.subject.name for example..

But imho we should put winlog.event_data.TargetUserName not only in user.name, but also in
user.target.name for clarity..

Another point is to take some other fields into account at the same time, because together with the user comes also the domain... So we would also need user.target.domain and user.subject.domain?

To be continued... Looking forward to other people's opinions.. :)

[andrew-goldstein]

@webmat in the current version of ECS (1.2 at the time of this writing), the ECS Process fields have a field to store parent process id:

process.ppid

, however I'm not seeing a field to store the parent process name, which would be equivalent to:

endgame.parent_process_name

[webmat]

@andrew-goldstein Valid point, but let's keep this issue about multiple users :-) Please open a separate issue for this.

[webmat]

@willemdh I didn't initially envision nesting user within itself. But it may work better than the examples I gave in the issue text. While affected.user reads well out loud, we'd effectively be creating an empty top level namespace, only meant to nest a reuse of user. That may be confusing :-)

So I like the idea of nesting user within itself, user.subject.* & user.target.* would map the Windows use case really well. I think this approach makes most sense when it's the same person (remote logon, privilege escalation).

Would it make sense for user management scenarios? For the "User A creates User B" scenario, would user.subject.name:admin and user.target.name:newuser make sense? I'm not sure.

Perhaps as you say, we should not try to find one naming scheme to match all scenarios.

Naming ideas, take 2:

• user.subject.* & user.target.* (+ user.* populated like user.target.*)
• user.subject.* & user.* (the target user mapped to top level user.* only)
• user.* & user.affected.* for user management scenarios

[willemdh]

Another example (I stumbled upon today):

Peter D'hot (domain\domainuser) copied the password for 'Eaton 9PX' (UPS) to the clipboard. (Title = Eaton 9PX -- UPS Groendreef, UserName = [email protected]). Client IP Address = 1.5.24.146

In this case it's for the user domain\domainuser who copied a password from our password manager. The password he copied came from the user [email protected]

Atm I'm doing it like this:

user.domain: domain
user.name: domainuser
passwordappliance.user.name: [email protected]

But it seems like passwordappliance.user.name: [email protected] could be replaced with:

user.target.name: [email protected]

[janniten]

Hi @webmat, @willemdh
First of all I want to mention two more use cases:

1. An user A access data from user B (case similar to privilege escalation). A lot of this cases are found when you analyze the dba_audit_trail of Oracle Databases
2. In some events we can have 3 users in a relationship ( a combination of cases 1 and 2). For example: a user A logs into a machine and by the means of privilege escalation to user B creates an user C.
   Example of a sudo log in a linux system
   Oct 30 15:56:09 srvtest sudo: __user_a__: TTY=pts/0 ; PWD=/home/user_a ;USER=__user_b__ ; COMMAND=/usr/sbin/useradd __user_c__

For in cases 1-4, I think the idea of having nested users field fits very well for the cases I have seen so far and I really like the idea of both having the user.name and the user.target.name as @willemdh mentioned.
I've been working with windows user managment events (https://github.com/elastic/beats/pull/13530) and in the cases like "User A creates User B" the mapping of user.subject.name:admin and user.target.name:newuser makes sense.
Here is a summary of the roles of subject and target user names in the windows user management and other user related events. I think the proposed mapping fits well in all cases

Also if we look in how auditbeat maps those kind of events a similiar aproacch is used. For example: when a user A creates a user B then the following fields are mapped like this
       user.name -> user A
       auditd.summary.actor.primary -> user B
       auditd.summary.actor.secundary -> user A

So, in summary I believe is a very good idea to use the nesting approach but I would choose a different name from user.target.name and user.subject.name (those make me think in windows events). Maybe user.name.primary and secondary? and if we have cases like 5 where there are more "actors" will be possible add more user.name.n-ary fields

What do you think?

[willemdh]

Thanks for the detailed information and examples @janniten . Although I totally agree user.subject.name and user.target.name indeed makes you think of windows, imho that doesn't need to be a bad thing? Most organizations are using Active Directory and are used to subject an target user fields. I'm quite sure my Windows security colleagues would really appreciate some known terminology. Also target and subject terminology is used in a lot of documentation etc. Reusing subject and target would imho make things a lot less complex. For your example 5, maybe another solution could be acceptable? Let's think about some workarounds for cases such as 5, before reinventing the wheel. Some wild ideas:
user.new.name
user.object.name

[janniten]

Hi @willemdh
I don't think that using user.target.name/user.subject.name is a bad thing by itself, but IMHO is maybe too specific in order to provide semantic content for all cases
Anyway, despite which name will be chosen the idea of implement nested field in order to represent user's relationships seems very good to me.

[willemdh]

@janniten

is maybe too specific in order to provide semantic content for all cases

Definitely, all of the above is also just imho. 😄

If the majority of ecs desingers think using n-ary fields are our best option, I will definitely follow.

So what do you think of user.object.name? Considering the scenario of three user names like 'user a logging in as user b creating user c', then isn't the new user name the object?

[janniten]

@willemdh

So what do you think of user.object.name? Considering the scenario of three user names like 'user a logging in as user b creating user c', then isn't the new user name the object?

Sorry I don't follow the idea. Do you mean user.object.name -> user c ?

[willemdh]

@janniten correct

[janniten]

I think that always the created user should me mapped to the same field. So if we have 'user a logging in as user b creating user c' and user.object.name -> user c, then when we have 'user a creating user b' user.object.name should have user b as value in order to preserve the semantic... and then in that case we will be not using the user.target.name
I'll keep thinking how to solve it.... :bulb:

[janniten]

What about having multiple users in a user.subject.name? I mean a list of usernames...
I 'm not sure how this can affect correlation

[andrewstucki]

So, not to derail the conversation too much, but i think it would be pretty useful to have a more generic definition of source and destination than currently exists today. Some various use cases I can think of (including this one under point 1):

1. process escalation (i.e. setuid/setgid issues)
2. file modification events (i.e. renames, permissions), where you need to model 2 files
3. IPC calls where you need to model 2 processes
4. others?

Therefore, whatever is done, I'd advocate for trying to address the more general use case rather than just adding in support for user's specifically and then having to go back and add support, with slightly different, but similar semantics, for other use cases later on.

If we did something like created a general namespace for source-like things, and one for destination-like things where we could nest user, file, process, network or whatever else wouldn't that address a lot of these concerns?

[webmat]

Yes, I'm open to this idea. I've been thinking very rigidly about source/destination about the two sides of a network exchange. This shows also in the relationship between src/dst and client/server, which are meant to add semantic context about each side of that exchange.

But there's been so many requests for nesting process, user & so on under source and destination that it's something we should consider as well.

Thanks for raising the point, @andrewstucki!

[leehinman]

I wonder if we could solve this problem by having an ECS entry inside another ECS entry. For example if we had a case where "user alice adds user bob to group friends" a very simplified nested ECS might look like this:

{
    "event": {
    "action": "update-group-membership",
    "user" : {
        "name" : "alice"
    },
    "child_event" : {
        "action": "add-user-to-group",
        "user" : {
        "name" : "bob"
        },
        "group" : {
        "name" : "friends"
        }
    }
    }
}

If on ingest the child_event was also used for indexing it should be possible to retrieve this document with user.name:bob or user.name:alice.

One possible downside. If you want to search for all instances where alice does the adding and not bob you have to explicitly ask for the nesting:

"user.name:alice AND child_event.action:add-user-to-group"

But I don't think that is worse than target.user or affected.user. The difference being a generic "child_event" instead of the specific "target" or "affected". This should be a benefit in case we need to do nesting we haven't thought of. For example this should work for the case 5 @janniten mentioned above.

For example 5 above
user.name = A
child_event.user.name = B
child_event.child_event.user.name = C

FYI. I don't like the name "child_event" but I haven't thought of anything better.

[willemdh]

@leehinman Imho the child event option is not a good option. My preference still goes to the target and subject option. At least for ms related events... Try explaining an AD admin that targetusername would be called child_event.user.name..
For use case 5:
Imho user.name should always be populated with the original user.
Which is the same as user.subject.name (a).. user.target.name (b) would then be the user which has been impersonated by a.
If user.new.name and user.object.name don't fit, maybe for the case of the newly created user, something like user.affected.name (c) fits better? Some event field like event.action needs to be populated with 'created', 'modified', 'deleted' though in order for this to work.

[janniten]

I agree with @andrewstucki about having more generic definitions.
Still does not convince me to have the target/source naming...It is ok for the semantic of ms events but lack of semantics in other cases and anyway when processing ms events you still have the winlog.event_data.SubjectUserName and winlog.event_data.TargetUserName.

Regarding to the content of user.name IMHO I think that we should keep the newly created user as auditbeat do it.

I put an example of how auditbeat puts the fields in case 5

auditd.summary.actor.primary: user_a
auditd.summary.actor.secondary: user_b
user.name: user_c
user.audit.name: user_a

@willemdh is not very different about what you are proposing (but without user the subject/target naming) with the only difference that user.name is not the original user but the new user in this case

user.actor.primary: user_a
user.actor.secondary: user_b
user.name: user_c
user.audit.name: user_a (I think this field could be useful when you want to track all activities of an audited user)

Does it makes sense?

[vbohata]

In more complicated systems/logs we are currently facing following:

1. User "admin" is changing user name of "user1" to "user1test"
2. User "admin" is changing the email address of "user1" from "old@email" to "new@email" - we need tracking of changes
3. User "appuser" is logging to application "appdest" from application "appsrc", which is the web service running under service user "srvuser".
4. Users in our systems have different user names, we need to to correlate it.

So to support logging of the events above we will need fields like:
1.
"subject.user.name" - admin user name
"target.user.name" - user's name
"target.new.user.name" - new user's name

2.
"subject.user.name" - admin user name
"target.old.user.email" - new user email
"target.new.user.email" - new user email
"target.user"

3.
"target.user.name" - user's name
"source.application.name" - appsrc
"destination.application.name" - appdest
... user name the dest app is running as ...???

4.
"user.name" - user's name
... additional user names - we need to store them somewhere ... definitely not possible to use "user.name" array if this could be a pivot ...

If using subject/target, it should be a higher level fields (subject.user, not user.subject), as user is not only one possible subject).

So "multiple users" issue looks very complicated. Maybe impossible to implement without the nested objects => Kibana has to support it.

[janniten]

@willemdh I kept thinking about the user.object.name option and I believe is a good options.
IMHO I still feel that other name than target and subject should be use but ,in spite of the name, it seems that the following schema could work for most of the cases

|case_description|user.actor.primary|user.actor.secondary| user.name | user.object.name | user.object.domain | user.session.id/user.login.id|
-|--|--|--|--|--|--
User A Manages User B | A|B|A|B|e.g. Windows AD Domain|A logon.id
Directory User A logs into a Machine X as User B|A|B|B|Machine X|e.g. Windows AD Domain| B logon.id
User A escalates /Assumes identity of User B|A|B|B|A|e.g. Windows AD Domain|B logon.id
User A access User'B data|A|B|A|B|e.g. Windows AD Domain|A logon.id
User A Creates as user B the User (object) C|A|B|A|C|e.g. Windows AD Domain|A logon.id
User A changes the Username from B to C|A|B|A|C|e.g. Windows AD Domain|A logon.id
Other case: |||||
User A manages an object O (Any AD Object) |A|A|A|O|e.g. Windows AD Domain|A logon.id

Also I agree that user name who is doing the action should be in the user.name field.
This is something I would like to know if everyone agrees. If so and it is the best option I'll modify some mapping I did in (https://github.com/elastic/beats/pull/13530)

Another user-related field that I observed while working with windows events events is the one that glue the actions performed by an user.
In the case of windows is the winlog.event_data.SubjectLogonId/winlog.event_data.TargetLogonId, in case of auditbeat it seems to be the the auditd.session
You can correlate the initial login with the actions the user performs during its session with this field

Could have sense to have a field called user.session.id ?

Thank you!

[willemdh]

Nice summary @janniten.. Best model of everything I've seen here.
Rather busy atm, but imho we need some consensus on this with even more examples and referral to real world logs / event ids (logon and user mgmt events).

[neu5ron]

all these examples are here:
https://github.com/hunters-forge/OSSEM/tree/master/data_dictionaries/windows/security

its just going to boil down to semantics, source/target or destination/source or as something like OSSEM uses: reporter, subject, and target because each log is done

[janniten]

Hi All,
I have just found that in ECS version 1.3 we can there is a new category: Related fields.
This field set is meant to facilitate pivoting around a piece of data.
I think it will be useful (independently of the schema we choose in order to represent users relationship) to have the following field

related.users | All of the users seen on your event.
-- | --

@webmat what do you think? Does it make sense to open a PR in order to propose this addition?

[webmat]

@janniten Yes, agree we should flesh out related.* some more, and related.users does make a ton of sense :-)

Everyone who participated here may be interested in the follow-up discussion at #678. We need to converge on this in the short term.

[webmat]

Closing in favor of #1066 (see also RFC 0007 stage 3 PR #1017)