Runtime: Expose UnixDomainSocket

[swernli]

dotnet/corefx#6833 made the UnixDomainSocketEndPoint internal, which of course prevents clients from using it to have direct access to Unix sockets in their own code. Docker.DotNet had to introduce a copy of UnixDomainSocketEndPoint in order to access sockets when running on Ubuntu (see https://github.com/Microsoft/Docker.DotNet/pull/113). Ideally, the language would provide a way for this access to happen, without needing each client to make a copy of the class for their use.

It's possible I missed a better way to do this, so advice would be appreciated...

Proposal

```C#
// in System.Net.Sockets assembly
namespace System.Net.Sockets
{
public sealed class UnixDomainSocketEndPoint : EndPoint
{
public UnixDomainSocketEndPoint(string path);
}
}

It'll work on Unix and result in PlatformNotSupportedExceptions on Windows.  Example usage:
```C#
var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.Unspecified);
socket.Bind(new UnixDomainSocketEndPoint(path));
socket.Listen(5);

[stephentoub]

dotnet/corefx#6833 made the UnixDomainSocketEndPoint internal

To clarify, prior to that change the type was only used in a test; when we started to actually build it in to the product, the visibility in the code was changed. So it wasn't that it was changed from public to internal in the product, but rather it was introduced to the product as internal.

Ideally, the language would provide a way for this access to happen, without needing each client to make a copy of the class for their use

I agree it would be a nice thing to expose at some point. In the meantime, copy-and-paste is the best option.

[swernli]

Thanks for the response! One clarifying question: in order to copy the code, I had to use the following code:
[DllImport("System.Native", EntryPoint = "SystemNative_GetDomainSocketSizes")]
internal static extern void GetDomainSocketSizes(out int pathOffset, out int pathSize, out int addressSize);

I know that usually the string provided for DllImport when using Windows is the name of the .dll file, but here I use a hard coded string that in the actual .NET source is a globally defined property. How likely is the location of the export/entrypoint to change, which could break copied hard coded values?

Additionally, where do these entrypoints come from? I tried to figure that out, but couldn't find anything in the code that shows where that is coming from. No real issue here, just trying to better understand how externs from native code work when running on Linux.

[stephentoub]

I had to use the following code

Ah. For simplicity, you probably want to use this one instead:
https://github.com/dotnet/corefx/blob/d0dc5fc099946adc1035b34a8b1f6042eddb0c75/src/System.Net.Sockets/tests/FunctionalTests/UnixDomainSocketTest.cs#L248
That System.Native.so/dylib is meant to be an internal detail of the implementation.

where do these entrypoints come from?

We build a set of "shims" for Unix to help smooth over differences between distros/versions/etc., e.g. where the sizes of various types change, names of exports are different, etc. That particular function is defined here:
https://github.com/dotnet/corefx/blob/cf1ebdc63412d6414e544be5784d4b5e8bb37614/src/Native/Unix/System.Native/pal_networking.cpp#L2649

[swernli]

Very interesting! So to make sure I understand from that sample, it's ok to hard code the values for the offsets, length and size instead of needing to use the extern to initialize those values in a static constructor? That definitely simplifies our code...

[stephentoub]

it's ok to hard code the values for the offsets, length and size instead of needing to use the extern to initialize those values in a static constructor?

Ideally you'd get the right values from native code, and we do so in the product library for robustness, especially as we want to take it to target more and more platforms. But these values should be sufficient for both OSX and Linux. The s_nativePathLength is actually smaller than it needs to be, in order to be as forgiving as possible, and while not guaranteed, the s_nativePathOffset value is correct for both Linux and OSX, at least for distros and versions we've seen. It's of course possible there will be versions where it's not, in which case you could do something like what we do, with a tiny amount of C code to fetch the right values for the managed code to use. You can of course use the DllImport that you showed earlier, but we don't guarantee that'll work from release to release... kind of like using reflection to access internal/private members.

[swernli]

Excellent, thanks for the clarification! Hopefully this will also be helpful to anyone who comes across this issue and wants to implement unix sockets in the interim until there is a class for it exposed via .NET itself.

[terrajobst]

In order to review this, we need a proposal for an API.

[khyperia]

This is a blocking issue for Roslyn. We are using unix domain sockets for compiler server IPC, and we cannot copypaste that class, as it calls private APIs (e.g. that Interop.Sys.GetDomainSocketSizes). Currently, we are using reflection to create an instance of the class, which is something that we'd like to not do ASAP.

[stephentoub]

as it calls private APIs (e.g. that Interop.Sys.GetDomainSocketSizes)

The test version at https://github.com/dotnet/corefx/blob/e48025a81b9a53e8e073c718c8714c4c8e3f5625/src/System.Net.Sockets/tests/FunctionalTests/UnixDomainSocketTest.cs#L363 does not call private APIs in the corefx shims; it just assumes the minimum values. You could do the same if it's really blocking.

[khyperia]

You said yourself:

we do so in the product library for robustness, especially as we want to take it to target more and more platforms

Which implies that the test version is not adequate for other platforms. Is that not the case?

[stephentoub]

You said yourself:

Yes, followed by "But these values should be sufficient for both OSX and Linux." What Unix platforms are you running on with .NET Core that aren't macOS or Linux? From my perspective, if you're really blocked otherwise, it's a better approach than being blocked.

[khyperia]

[...] at least for distros and versions we've seen. It's of course possible there will be versions where it's not, [...]

Perhaps running on a distro we haven't seen?

[stephentoub]

Perhaps running on a distro we haven't seen?

Since I wrote that, we've switched to having a single native build for all Linux distros/versions from the last few years and forward. And the value this gets from the native shim is based on compile-time details. So if this breaks on future distros/versions, our built-in implementation would effectively also be broken.

[stephentoub]

Let's just expose this thing. We keep talking about it, people want it, let's do it.
```C#
// in System.Net.Sockets assembly
namespace System.Net.Sockets
{
public sealed class UnixDomainSocketEndPoint : EndPoint
{
public UnixDomainSocketEndPoint(string path);
}
}

It'll work on Unix and result in PlatformNotSupportedExceptions on Windows.  Example usage:
```C#
var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.Unspecified);
socket.Bind(new UnixDomainSocketEndPoint(path));
socket.Listen(5);

[stephentoub]

cc: @eerhardt

[eerhardt]

I'll work on this next week.

[karelz]

FYI: The API review discussion was recorded - see https://youtu.be/o5JFZtgaJLs?t=1249 (6 min duration)