Runtime: System.DirectoryServices.Protocols - Linux support

[brunohbrito]

I'm trying to make a LDAP Connection under aspnet:3.1-buster-slim container. It returns the error:

System.DirectoryServices.Protocols is not supported on this platform

I found a PR #35380 which says that support Linux now. My nuget package is 5.0.0-preview.4.20251.6 from may 18.

Am I missing something?

Configuration

• Which version of .NET is the code running on?
  .NET Core 3.1

• What OS and version, and what distro if applicable?
  Debian 10

[vcsjones]

I don't think preview4 contains the PR. The preview4 branch was last committed to on May 1st: https://github.com/dotnet/runtime/commits/release/5.0-preview4

And the Linux support was merged on May 7th: https://github.com/dotnet/runtime/commit/55d3260ad5dc48988228ac0cb1278541e87e513d

So the cut for preview4 was taken before this change was merged.

It does appear in the preview5 branch.

[brunohbrito]

Great! So I suppose the next preview version will be available.

[danmosemsft]

@brunohbrito when you do get the bits to try could you please let us know your feedback - even if you do not encounter any problems? This is the kind of feature that can have configuration specific issues and the more validation we get thru release the better. Cc @joperezr

[joperezr]

@brunohbrito just wondering, would you be willing to try our nightly builds so you get a chance to try this now? If you’d rather wait for next preview that’s fine too. Preview 5 will have Linux support but won’t have default credentials working yet (meaning you don’t need to pass in credentials to ldapconnection if your Linux machine is domain joined) but preview 6 will have that plus OSX support. Nightly builds have all 3.

[brunohbrito]

@joperezr I'm getting into night build right now, I'll try it and them I give you some feedbacks

[brunohbrito]

Hi @joperezr / @danmosemsft ,

To do theses tests I'm using an Active Directory under Windows Server 2019 Datacenter.

Scenario 1 - Authenticating user

I'm running the following code snippet both for Windows and Linux Environment.

``` c#
var credential = new NetworkCredential("", "");

var connection = new LdapConnection(new LdapDirectoryIdentifier(" {
AuthType = AuthType.Ntlm
};

connection.Bind();

## Windows Environment - Running in .NET Core 3.1 and .NET 5.0

The code runs fine.

## Linux - Running in a container. runtime:5.0-buster-slim

At first run I got an error:

`Unable to load shared library 'libldap-2.4.so.2' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibldap-2.4.so.2: cannot open shared object file: No such file or directory`

To fix that, I've changed the Dockerfile adding apt-get:

``` Dockerfile
FROM mcr.microsoft.com/dotnet/core/runtime:5.0-buster-slim AS base
RUN apt-get update && apt-get install -y libldap-2.4-2
WORKDIR /app

Then I got a 2nd error: The supplied credential is invalid.. (Running on Windows (wldap.dll), everything was ok)

So, to fix that I'd to concat the domain name with username. <domain-name>\<username>:

``` c#
// FROM
// var credential = new NetworkCredential("", "");
// TO:
var credential = new NetworkCredential(@"\", "");

And the connection was successfull. Even when added `domain` at NetworkCredential Ctor didn't work

## Linux - Using runtime:3.1-buster-slim

Everything was fine, except by the needs to concat the `<domain>`. 

## libldap suggestions

I was looking at [libldap docs](https://libldap.readthedocs.io/en/latest/LDAPObject.html?highlight=bind_s#LDAP.simple_bind_s). `bind_s` and `simple_bind_s` actually are the same and just support LDAP_AUTH_SIMPLE.

So, why we don't concat domain name in every case (when it's available) for Linux?

My suggestions is, at BindHelper() on [LdapConnection.cs](https://github.com/dotnet/runtime/blob/master/src/libraries/System.DirectoryServices.Protocols/src/System/DirectoryServices/Protocols/ldap/LdapConnection.cs#L1103) line 1099, remove this `if` and put it into Linux / Windows specific logic.

I've made a PR, in case you agree to theses changes.

# Scenario 2 - Searching

I was trying to search a user specific metadata by following code snippet:

```c#
var searchFilter = $"(|([email protected])(sAMAccountName={username}))";
var searchRequest = new SearchRequest("DC=brunobrito,DC=net", searchFilter, SearchScope.Subtree, "displayName", "cn", "mail" );

var response = (SearchResponse)connection.SendRequest(searchRequest);

tl;dr: I've tried in many ways to perform a Search in Linux scenarios. I just can't.

Windows Environment - Running in .NET Core 3.1 and .NET 5.0

The code runs fine. It search users in many ways. Returns all metadata.

Linux environment

The Search doesn't work. Always some kind of error.

I tried to change the search filter, parameters even the user credentials. But always get the same error: An unspecified operation error occurred.

Bug - search_ext_s wrong signature

When I changed the Timeout parameter:
c# searchRequest.TimeLimit = TimeSpan.FromMinutes(2);

Some crash occurs, bipassing even the trycatch. I got that the parameters timeLimit and sizeLimit of method search_ext_s at libldap was in wrong position. In fact there is a difference between Microsoft docs and libldap about timelimit. Maybe that was the reason.

Summary - Final discussion

I understand that under the hood we are at top of different drivers. While in Windows is wldap32.dll, Linux is libldap. But I think theses differences will make a huge difference for developers. They aren't used to deal with this kind of low level diffs.

Why not go in the same directions of Novell.Directory.Ldap.NETStandard? It's isn't a SO dependent. Btw I'had succes in all tests using it.

I'll create separated Issues for better discussion.

[danmosemsft]

@brunohbrito this is very useful thank you for opening the issues. We will take a look and fix as needed.

[joperezr]

Thank you so much for trying out our bits and we really appreciate the feedback 😄
Few comments regarding that:

At first run I got an error: apt-get...

Yes in order to run in Linux or OSX we do require libldap to be installed, which is the case for most Linux distros out there as well as OSX. That said, I do believe that minimal OS versions like the ones used in docker might stripped these dependencies out, so it makes sense you were hitting that. I think it may be a good idea to either add a note on the package to say that we have that as a dependency or perhaps in the code we could catch the dllnotfoundexception and wrap it in a more friendly message saying you are lacking that dependency, would that sound like a good plan?

So, why we don't concat domain name in every case (when it's available) for Linux?

I'm not sure if this is possible, in Windows if you are domain-joined you can probably get away with it because Windows carries a bit extra info on the domain realm you are joined in, but for Linux this is a bit different given that Linux has a separation between local accounts and LDAP accounts. All that said I would like to understand more about your scenario, is your Linux machine domain joined? If so, have you ran a kinit in order to get a valid Kerberos ticket? That would be the equivalent of being signed in on Windows.

I've made a PR, in case you agree to theses changes.

I'll take a look at it, thanks so much for your contribution.

The Search doesn't work. Always some kind of error.

For your search issue, have you ever tried passing in null as attributeList? What may be happening here is that one of the attributes you are passing in is not present on all entities found which could lead to an error. My suggestion would be to try something like:
```c#
var searchFilter = $"(|([email protected])(sAMAccountName={username}))";
var searchRequest = new SearchRequest("DC=brunobrito,DC=net", searchFilter, SearchScope.Subtree, null ); // Passing null as attributelist basically means return all attributes

var response = (SearchResponse)connection.SendRequest(searchRequest);
```

Bug - search_ext_s wrong signature

Good catch, yeah that sounds like a problem, we should probably log a separate bug for that and fix it.

I understand that under the hood we are at top of different drivers. While in Windows is wldap32.dll, Linux is libldap. But I think theses differences will make a huge difference for developers. They aren't used to deal with this kind of low level diffs.

You are correct that our managed implementations are pretty much the same between Windows and Linux, and we are basically just using different native libraries (or drivers) underneath with just a few places where we handle calls differently between Windows and Linux. That said, we only made this decision because both underlying drivers (wldap32 and libldap) are simply just the implementation of a protocol (LDAP Protocol) to the point where they even have about 95% same signatures between them and they also have the same behavior. Because of this, the only main differences are not in the calls themselves, but just in the way that OSes handle Active directory authentication and domain joined management (which are the issues you found in your first scenario). Windows does have some shortcuts available so that for example, you don't always need to pass in domain/REALM, or it is easier to find the right LDAP server without developers needing to do much configuration, but those differences are mostly just ways how OSes differ on authentication handling. If you have your code not rely on these Windows shortcuts (meaning you pass in the user@REALM as credential) then I would expect this to work both in Windows and Linux.

Again thank you so much for your feedback, it is very valuable in making sure that we have a great solution for managing LDAP on Linux/OSX 😄

[brunohbrito]

Yes in order to run in Linux or OSX we do require libldap to be installed, which is the case for most Linux distros out there as well as OSX. That said, I do believe that minimal OS versions like the ones used in docker might stripped these dependencies out, so it makes sense you were hitting that. I think it may be a good idea to either add a note on the package to say that we have that as a dependency or perhaps in the code we could catch the dllnotfoundexception and wrap it in a more friendly message saying you are lacking that dependency, would that sound like a good plan?

Yes, It sounds good to me. Maybe a message redirecting user to some docs explaining how to solve it. There is a chance to Visual Studio identify which package is referenced during Add Docker Support from Solution Explorer and auto add apt-get some required packages?

, is your Linux machine domain joined? If so, have you ran a kinit in order to get a valid Kerberos ticket? That would be the equivalent of being signed in on Windows.

No, either my Linux and Windows wasn't joined at domain. I've created a Virtual Machine at Azure and made a simple Active directory config just for theses tests.

For your search issue, have you ever tried passing in null as attributeList?
Yes, I tried to send null, also to change Scope Level, distinguishedName, even AuthType from Ntlm to Basic.

Yes. I've tried in many ways to check if something was wrong with my payload. But when I changed for Windows or performs the same search with Novell.Ldap, I got results.

That said, we only made this decision because both underlying drivers (wldap32 and libldap) are simply just the implementation of a protocol (LDAP Protocol) to the point where they even have about 95% same signatures between them and they also have the same behavior

Great! I'm very excited to use it! One of my projects rely on that!

Again thank you so much for your feedback, it is very valuable in making sure that we have a great solution for managing LDAP on Linux/OSX

A special thanks to @ralmsdeveloper who helped me in some scenarios, also to build and debug dotnet/runtime (was painful)

[GrowSharp]

Hello there,
I just wanted to let you know, that
connection.SessionOptions.SecureSocketLayer = false; throws operation is not supported on this platform exception. Tried on Ubuntu 20.04, .NET Core 3.1, *.Protocols version 5.0.0-preview.5.20278.1.

Edit: var connection = new LdapConnection("MyDomainName");

[joperezr]

Hello @GrowSharp yes, that is by design: https://github.com/dotnet/runtime/blob/adab8f20bb14677e5395e2c6a13c4ecda228adc8/src/libraries/Common/src/Interop/Interop.Ldap.cs#L67

the native library that we use underneath for our Linux implementation is openldap, which doesn't support this option like wldap32 does. Here is more info on openldapand the options they do support in case you are interested: https://www.openldap.org/software/man.cgi?query=ldap_get_option&sektion=3&apropos=0&manpath=OpenLDAP+2.4-Release

[GrowSharp]

Oh, I see, thanks. Today I'm going to implement role searching. Will let you know how it went.

[GrowSharp]

Okay, took me little longer than expected, but I have some news.

On windows everything works perfectly. I even got some performance boost from it which is nice (and understandable, because I went from .AccountManagement to .Protocols).

But on Linux: (same setup as last time, Ubuntu 20.04, .NET Core 3.1, .Protocols version 5.0.0-preview.5.20278.1)

1.) Anonymous bind doesn't work ( ldapConnection.Bind(); )
It doesn't throw exception, but then when I send SearchRequest it throws
000004DC: LdapErr: DSID-0C09075A, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v1db1

2.) When searching for a role by CN that contains \ character I get The LDAP server returned an unknown error. with error_message NULL. Stack trace: System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request, TimeSpan requestTimeout) at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request)

3.) This is really weird thing that happens sometimes and I have nothing to show for it. So what happens:
Sometimes when I was debugging on linux, the first few times everything works fine. But after I restart debugger few times, the code gets stuck at binding ( ldapConnection.Bind(); ) and it just doesn't move forward. Nothing really helps, I have to restart PC to get it to work again..which is really frustrating gotta admit 😄
I thought that it might be connected to bad disposing of connection (even tho I use using statement), but I made really, really sure that every time I do .Dispose();
I still believe that it has something to do with disposal, but it's probably on lower layer.

[joperezr]

For 1), we do support anonymous bind but we do require for you to be on a Linux machine which is domain joined to an Ldap server. The best way I found to test to make sure that I was successfully authenticated to the server, was by trying to run the ldapwhoami command on the terminal and passing in the realm I wanted to connect to, for example: ldapwhoami -R MYDOMAIN.COM. If that succeeds but you still have problems calling ldapConnection.Bind() then it means there is a bug in the library which we should fix so please file a bug for that. If the ldapwhoami routine also fails, you can find more diagnostic info on why that happened by passing in a debug level with the -d parameter.

For 2) it would be good to get more info includding your full code and what you get in windows vs Linux. In theory both underlying native libraries use the same filter language which follows the ldap protocol, so I would be interested in learning why is the behavior different. Please do log a separate issue for that so with more info so that we can take a look.

Number 3) I'm not really sure what might be going on to be honest, and unfortunately there is little info there we can use to come up with a repro to understand what is going on. If you manage to get a consistent repro we would be very interested in checking it out and understanding what is wrong.

[GrowSharp]

I did some more digging. This time on different machine. Centos 7 with .NET Core 3.1 and .Protocols version 5.0.0-preview.5.20278.1

1) Anonymous bind indeed works on linux systems that are domain joined. But then searching doesn't work.
var searchRequest = new SearchRequest(LdapConstants.SETTING_USER_BASE_DIRECTORY, searchFilter, SearchScope.Subtree, LdapConstants.SettingGroupSearchAttributeFilter.ToArray());
throws
An operation error occurred.
at
System.DirectoryServices.Protocols.DirectoryOperationException: An operation error occurred. at System.DirectoryServices.Protocols.LdapConnection.ConstructResponse(Int32 messageId, LdapOperation operation, ResultAll resultType, TimeSpan requestTimeOut, Boolean exceptionOnTimeOut) at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request, TimeSpan requestTimeout) at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request)

2) Almost same code as in the first case; Still same problem with cn containing the \ character.
var searchRequest = new SearchRequest(LdapConstants.SETTING_GROUP_BASE_DIRECTORY, searchFilter, SearchScope.Subtree, LdapConstants.SettingGroupSearchAttributeFilter.ToArray());
throws
The LDAP server returned an unknown error. (ldap error code: -7)
at
System.DirectoryServices.Protocols.LdapException: The LDAP server returned an unknown error. at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request, TimeSpan requestTimeout) at System.DirectoryServices.Protocols.LdapConnection.SendRequest(DirectoryRequest request)

3) Okay, so this problem still persist, but I got some more details for it. So problem is, that sometimes (really randomly) the bind takes 127 seconds to complete. It doesn't matter if it's anonymous bind or not. Just sometimes the .Bind(); takes roughly 127 seconds. Did some logging around it for you to see:
_Elapsed time 6 ms
Elapsed time 7 ms
Elapsed time 6 ms
Elapsed time 8 ms
Elapsed time 5 ms
Elapsed time 127317 ms
Elapsed time 127183 ms
Elapsed time 5 ms
Elapsed time 127180 ms
Elapsed time 9 ms
Elapsed time 4 ms
Elapsed time 127299 ms_

4) This time I also found out that it has more problems with special characters, because when I try to authenticate user with some special characters in password (either: ! or $, not sure..)
it throws
ldap error code: 49 (invalid credentials)
at
System.DirectoryServices.Protocols.LdapException: The supplied credential is invalid. at System.DirectoryServices.Protocols.LdapConnection.BindHelper(NetworkCredential newCredential, Boolean needSetCredential) at System.DirectoryServices.Protocols.LdapConnection.Bind(NetworkCredential newCredential)

On windows or with accounts without special characters it works correctly tho.

Now please tell me, if you need anything more and for which bugs you wan't me to create separate issues.
Thanks!

[joperezr]

can you log one issue for each of the problems you are seeing? That way we can focus each one on individual investigations. I haven't been able to reproduce1-3, and I haven't tried 4 but as part of the investigation of that issue I'll create a user with a password with special characters and try to reproduce that.

[GrowSharp]

Created separated issues as asked:

1. Anonymous bind searching #38608
2. Searching for roles with \ in it #38609
3. Binding time #38610
4. Users with special character credentials #38611

[ericstj]

I believe this can be closed now that the issues are broken out. Let me know if I'm mistaken.