Opa: Add built-in function to execute LDAP queries

[tsandall]

In some cases, the typical approaches for getting LDAP data into OPA do not work (e.g., replication may not scale and authenticating proxies may not have enough knowledge to gather the right context.) For example. if the data required from LDAP is only known at request-time then it has to be fetched on the fly (either by the app and provided as input) or by the policy. Putting this logic into the app increases coupling between the app the policy which is undesirable.

In situations like this it would be useful to have the ability to execute LDAP queries from inside the policy that can fetch the relevant context on the fly. This capability would be similar to the existing http.send built-in function, except that it would work with LDAP instead of HTTP.

We can introduce a new built-in function, e.g., ldap.query(args, output) which takes as input a set of arguments and returns as output the JSON representation of the LDAP query result.

There are a few details that need to be fleshed out before we commit to adding this.

1. How should the query arguments be represented in JSON?

2. How should the query results be represented in JSON?

3. Which Golang LDAP library should we use to execute the queries? When it comes to selecting the client library, we should be careful to find one that is well supported, robust, and introduces minimal additional dependencies. gopkg.in/ldap.v2 is a good candidate.

[srenatus]

ℹ️ There's so many search variants in LDAP -- but luckily, cURL supports querying LDAP server, and maybe its mapping rules could be taken as an example: see for example here. As pointed out there, it's based on RFC 4516

[srenatus]

Just a side note, testing this could be difficult, but we might snatched a line or two from the ldap tests in dex: https://github.com/dexidp/dex/blob/master/connector/ldap/ldap_test.go

[tsandall]

Here's a rough sketch for the query function:

ldap.query(connection_params, query_params, output)

Where:

• connection_params is an object containing the required connection params, e.g., url, username, password, or whatever is most idiomatic from the perspective of LDAP clients.
• query_params is an object containing the required query params, e.g., base DN, scope, filters, etc.
• output is a JSON object containing the attributes resulting from the query. We may want to add a level of nesting to the output, e.g., {"attributes": ....} so that if we need to include additional information in the future, we're not boxed in.

As far as error handling goes, I'd expect that non-connection errors will result in an evaluation/runtime error that gets returned to the caller. Connection retries and timeouts could be controlled via the connection_params.

When we want to add additional built-ins, e.g., membership tests, we could follow a similar pattern:

ldap.is_member(connection_params, user, group)

[tsandall]

In the last community meeting @repenno indicated he was interested in looking into this and working on it. I'm going to assign to him for now.

[mudlark]

Hi all,

Is some of the OPA documentation aspirational? I node the reference to data.ldap on https://github.com/open-policy-agent/opa/blob/master/README.md and on the best practice page it really makes it sound like this is at lease experimentally available already: https://www.openpolicyagent.org/docs/best-practice-identity.html (#4)

Is there any current solution to this, or am I left having to implement something that bridges AD to http for this right now?

Thanks!

Jody

[timothyhinrichs]

Hi Jody,

Apologies if the docs came off as "aspirational". We could make it clearer
that there's NOT an LDAP integration available out of the box today. The
JWT and Push sections I think are pretty good, but the Bundle section
requires you to know the semantics of OPA's bundle API. Opened a PR that
should help: https://github.com/open-policy-agent/opa/pull/1230

No there's no bridge to AD/LDAP right now. Contributions are welcome!

Tim

On Fri, Feb 22, 2019 at 10:39 AM Jody Pearson [e-mail]
wrote:

Hi all,

Is some of the OPA documentation aspirational? I node the reference to
data.ldap on
https://github.com/open-policy-agent/opa/blob/master/README.md and on the
best practice page it really makes it sound like this is at lease
experimentally available already:
https://www.openpolicyagent.org/docs/best-practice-identity.html (#4
https://github.com/open-policy-agent/opa/pull/4)

Is there any current solution to this, or am I left having to implement
something that bridges AD to http for this right now?

Thanks!

Jody

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/open-policy-agent/opa/issues/938#issuecomment-466501359,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJlYK1v7RrliatL1ilWX35LLyWrJKq7Dks5vQDlWgaJpZM4WfFGC
.

[mudlark]

To Tim,

That makes sense. I do.appreciate the fact that this is solved when using jwt /oidc etc. I've been wanting to learn go for a while, so this looks like my itch to scratch ;) I wanted to make sure this was not already in hand before assuming.

I'm a huge fan of OPA - you have a very nice tool here - thanks for providing it!

Jody

[tsandall]

We should figure out how we're going to deal with other protocols before we implement this. We need to avoid to be careful about pulling in too many 3rdparty dependencies.

[repenno]

Agreed. We can go back to our discussion about allowing OPA to flourish vs. dependencies vs. stifling development. There will always be another protocol so we should have some generic built-in that calls an executable with some parameters. The executable would not be part off OPA distribution of course, just the generic built-in. This seems better: https://golang.org/pkg/plugin/

[ahmedtalhakhan]

Is there any PR raised for this yet?

[srenatus]

@ahmedtalhakhan I don't think so. 😃

[ahmedtalhakhan]

@srenatus @tsandall @repenno
I understand from the above commentary that this effort is not complete yet. But I would like to take this opportunity to ask the following.

What is the simplest way to write a new custom built-in function of the form ldap.query(params) or mongo.query(params). Is there a brief write-up/tutorial on what code change need to be done and what files/data structures need to be updated in order to achieve this.

Assume that the nitty gritty about the internals(what driver to use inside, how to represent data etc)of the new function have been resolved.

[patrick-east]

https://www.openpolicyagent.org/docs/latest/extensions/#custom-built-in-functions-in-go Shows how to add a custom built-in function through the Rego public golang API's. This is the easiest way if you want to customize OPA.

The process is pretty easy to add one to OPA as a default part of the language too, here is a simple example of a PR that adds one: https://github.com/open-policy-agent/opa/commit/7537f7ad06c3fa1a8038b861d9234bedf2a06c0c#diff-a6bb3036dbf3fa3bfaed6650acfa9879

At a high level you define the signature, implement the function, and connect the two by registering it at init time (or whenever, as long as it is before you start calling other OPA API's).

In both cases the implementation of the builtin function that is registered is pretty much the same.

[ahmedtalhakhan]

@patrick-east thanks for the swift reply. Will the approach explained at https://www.openpolicyagent.org/docs/latest/extensions/#custom-built-in-functions-in-go
work if I want to use that built-in inside policy files?

[patrick-east]

Depends on how you intend to run OPA, if you are doing everything through the Golang API then yes, you can register one like that and call the API's to load files and evaluate them.

If you want to run OPA CLI's and have the built-in function then use the second example on that page to build a customized opa https://www.openpolicyagent.org/docs/latest/extensions/#custom-plugins-for-opa-daemon

Instead of implementing the custom decision logger in that example you would instead define the function and register it similar to how that PR I linked to does it (just with the code in your files rather than in OPA). The only difference is instead of adding the signature into ast/builtins.go you call the ast.RegisterBuiltin function in addition to the topdown.RegisterXXXXX call.

[ahmedtalhakhan]

Thanks @patrick-east