Runtime: Cannot set udpClient.ExclusiveAddressUse = false in macOS and Linux

[christianhuening]

I originally posted this as a comment to this closed issue: https://github.com/dotnet/corefx/issues/8552#issuecomment-243579967

Since the issue is somewhat different, I decided to open a new one.

I am implementing a Multicast application, which requires me to allow to bind multiple sockets to the same port on a machine. While working fine on windows 10 setting udpClient.ExclusiveAddressUse = false fails for me on Linux and macOS with a SocketErrorCode 45 (Operation Not Supported) and an HResult of 0x80004005, which translates to "E_FAIL" or "Unspecified Error" as per this listing: https://msdn.microsoft.com/en-us/library/cc704587.aspx

I am testing on Windows 10 and macOS using the latest version of everything.

On Windows 10 I have to use:
udpClient.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
for it to work.
Using only
udpClient.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ExclusiveAddressUse, false);
doesn't do a thing on windows and I receive an error.

On macOS I receive an error as follows, when using:
udpClient.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ExclusiveAddressUse, false);
Error:

System.Net.Sockets.SocketException : Operation not supported
at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue, Boolean silent)
at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue)
at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Boolean optionValue)

While using:
udpClient.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);
on macOS does not throw and exception, but does not work for me either.

The property udpClient.ExclusiveAddressUse behaves accordingly, in that it is working on Windows 10 and fails on macOS.

Is there anything I am missing or doing wrong here?

[davidsh]

cc: @stephentoub @ericeil

[ericeil]

OSX doesn't have the SO_EXCLUSIVEADDRUSE socket option, so currently we throw if you try to set that option.

It looks like Mono took a different approach: on systems with no SO_EXCLUSIVEADDRUSE option, they set SO_REUSEADDR the opposite way. So:

udpClient.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ExclusiveAddressUse, false);

...is translated to...

udpClient.Client.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true);

We should consider doing the same, if only for compat with code that works on Mono. @davidsh, do you have any opinion on whether this sort of translation makes sense?

[christianhuening]

@ericeil : thanks for the clarification! I used mono intensively over the past 3,5 years. The reuseaddress approach worked fine so far. Also the ReuseAddress option is not exposed as property on system.net.socket in dotnet core so it wouldn't be ambivalent.

Setting the reuse option via set socket options on dotnet core currently doesn't work however, is that intended?

Thanks for caring! Dotnet Core already is really great from a developer experience!

[davidsh]

We should consider doing the same, if only for compat with code that works on Mono. @davidsh, do you have any opinion on whether this sort of translation makes sense?

@CIPop Can you comment on this?

[ericeil]

I'm reading more about these options, which are new to me. 😄 There's a good writeup on this on stackoverflow, and more in the Windows docs.

It looks to me like SO_EXCLUSIVEADDRUSE is really asking Windows to behave more like other OSs. So it makes sense that this wouldn't exist on OSX. Setting this to "true" means "don't allow other sockets to reuse this address, even if they specify SO_REUSEADDR." Setting it to "false" means "allow other sockets to reuse this address." I'm guessing Mono's logic for translating this via SO_REUSEADDR is trying to simulate this: if my socket does _not_ have SO_REUSEADDR set, then other sockets can't reuse the address, so this is sort of like having SO_EXCLUSIVEADDRUSE set to true on Windows.

However, setting SO_REUSEADDR to "true" on OSX _also_ implies that _this_ socket can reuse another _pre-existing_ socket's address, which isn't implied by SO_EXCLUSIVEADDRUSE. So this isn't an exact translation, and I can imagine this translation causing other issues for code that worked well on the full CLR on Windows.

If we set Mono compatibility aside, it seems like perhaps the best thing would be to just make SO_EXCLUSIVEADDRUSE a no-op on platforms that don't support it. This way, setting ExclusiveAddressUse=true is really a way of telling us to make address reuse work the same way on all platforms. It would be nice if we could do this by default, but that would surely be a breaking change on Windows....

[ericeil]

Setting the reuse option via set socket options on dotnet core currently doesn't work however, is that intended

We just pass this through to the OS, as the native SO_REUSEADDR option. However, see the stackoverflow answer for the differences in the semantics of this option on Windows vs. OSX, etc. It would be nice if we could make these options equivalent on all platforms, but I'm not sure we can. I'll need to give that more thought.

[christianhuening]

@ericeil : Thanks for the stackoverflow and windows docs link! Hadn't found those before.
On a spontaneous note: Having ported my project initially from Java to C#, then from .NET Framework to Mono and now over to dotnet core (the overall process involving the three major OS platforms + virtualization) somehow puts me in the mood of "I'd like to configure it while being informed by a decent documentation" rather than having an equivalent approach on all platforms, which historically seems almost impossible to me as stated by the stackoverflow article.

That said, I noticed that the Microsoft documentation for UdpClient.ExclusiveAddressUse on DotNet Core states:

true if the UdpClient allows only one client to use a specific port; otherwise, false. The default is true for Windows Server 2003 and Windows XP Service Pack 2 and later, and false for all other versions.

(https://docs.microsoft.com/de-de/dotnet/core/api/system.net.sockets.udpclient#System_Net_Sockets_UdpClient_ExclusiveAddressUse)

Does this "(...) and false for all other versions" apply to versions of Windows or to all other OS versions? I think it's the former, because ReuseAddress is not working on OSX.
But if ExclusiveAddressUse=false is meant to be the default on all the most recent operating systems including Windows, making SO_EXCLUSIVEADDRUSE a no-op on non-supporting platforms, wouldn't be too much of a breaking change and allow us to reliy on SO_REUSEADDR, wouldn't it?

[ericeil]

Thanks for pointing out the comment about the default value; I missed that, and it's very helpful here.

Does this "(...) and false for all other versions" apply to versions of Windows or to all other OS versions?

I assume this was meant only to apply to Windows versions, as this was almost certainly written before this was ported to other operating systems.

[ericeil]

So, here's what I propose:

ExclusiveAddressUse: enabling this just turns on a rule that's always enforced on OSX and Linux anyway. So:

1. Getting the value of ExclusiveAddressUse on OSX/Linux should always return "true."
2. Setting ExclusiveAddressUse = true on OSX/Linux should do nothing.
3. Setting ExclusiveAddressUse = false on OSX/Linux should throw (PlatformNotSupportedException?).

ReuseAddress: according to everything I've read so far, the Windows version of this option is equivalent to SO_REUSEADDR _and_ SO_REUSEPORT _combined_, on OSX/Linux. Existing uses of this option on Windows will certainly assume this, and this is probably what's making ReuseAddress appear not to work for @christianhuening. Also, SO_REUSEPORT is not exposed via any managed API. So:

1. Getting the value of ReuseAddress on OSX/Linux should return "true" if, and only if, SO_REUSEADDR _and_ SO_REUSEPORT are "true."
2. Setting the value of ReuseAddress to "true" on OSX/Linux should set both SO_REUSEADDR and SO_REUSEPORT to "true."
3. Setting ReuseAddress to "false" on OSX/Linux should clear both SO_REUSEADDR and SO_REUSEPORT.

If there's ever a need to configure SO_REUSEADDR and SO_REUSEPORT separately, we'll need to add new items to the SocketOptionName enum. Maybe "ReuseAddressOnly" and "ReusePort?" But let's cross that bridge when we get there....

@stephentoub, @CIPop, any comments? @pgavlin, do you have any thoughts here?

[CIPop]

I agree with the ExclusiveAddressUse plan on platforms that have a different default behavior.

Regarding ReuseAddress and a potentially new enum value ReusePort: these are advanced socket options that, in my mind, map directly to the OS layer's implementation. I think that developers expect different behaviors as well as different options for each platform.
I agree that if we aim at simplifying porting apps from Windows then the above plan is what we want.
In the long run, this change might lead to confusion for UNIX developers which won't expect ReuseAddress to affect two different SO_REUSE* options

Maybe it would be easier if:

1. we added ReusePort as an option and enhance the documentation to point to setsockopt for Windows/*NIX.
2. we add a new public method/property that would act as a PAL for applications that just want this scenario to work cross-plat without caring which options get set on different platforms.

[christianhuening]

Maybe it would be easier if:

1. we added ReusePort as an option and enhance the documentation to point to setsockopt for Windows/*NIX.
2. we add a new public method/property that would act as a PAL for applications that just want this scenario to work cross-plat without caring which options get set on different platforms.

Sounds like a good alternative to me! With an enhanced documentation, this should not overly confuse people.

[christianhuening]

@ericeil Can you estimate when a "fix" for this scenario might be available? I am depending on this to work in order to be able to continue my project and would like to get an idea of how long it will take. If I can do anything to help, I gladly will!

[christianhuening]

Damn hit "close and comment" mistakenly. sorry everybody.

[ericeil]

@christianhuening, it just occurred to me that your scenario was setting ExclusiveAddressUse = _false_, which my proposal would not address (it'll still throw on Unix).

I am implementing a Multicast application, which requires me to allow to bind multiple sockets to the same port on a machine.

I wonder if you need to set ExclusiveAddressUse at all? I'm guessing you really need to set just ReuseAddress=true. You mention that this did not work for you on OSX; could that be because of the lack of SO_REUSEPORT support?

[christianhuening]

@ericeil Yeah, that's why I think point 2 from your proposal would work for me:

Setting the value of ReuseAddress to "true" on OSX/Linux should set both SO_REUSEADDR and SO_REUSEPORT to "true."

Setting ReuseAdress = true is all that's needed on Windows, but does not work on OSX / Linux, So I think, yes, adding SO_REUSEPORT support would be enough.

Can I somehow try this out? Is there any way to set that option to the socket I am using?

[ericeil]

Can I somehow try this out? Is there any way to set that option to the socket I am using?

I'm working on a PR for this now; assuming all goes well, this should be merged early next week.

In the meantime, the only way I can think of to try this would be to use reflection to get the socket's file descriptor (via private field access), and then manually p/invoke to setsockopt.

[mika76]

I seem to be getting this exception still on .NET Core 2?

[idubnori]

I've observed the similar issue which seems to related to this. onovotny/Zeroconf#96
It's on ARM32 (Raspberry Pi).

System.Net.Sockets.SocketException: Operation not supported
at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, String callerName)
at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue, Boolean silent)
at System.Net.Sockets.Socket.SetSocketOption(SocketOptionLevel optionLevel, SocketOptionName optionName, Int32 optionValue)
at System.Net.Sockets.Socket.set_ExclusiveAddressUse(Boolean value)
at System.Net.Sockets.UdpClient.set_ExclusiveAddressUse(Boolean value)
-Zeroconf/Platforms/netstandard/NetworkInterface.cs#L94

[winkmichael]

Still seeing this in mono 4.8 and .net core 2 on OSX