Runtime: Expose GetPipePath on System.IO.Pipes.PipeStream

[casperOne]

Add a new System.IO.Pipes.PipeStream.PipePath public property (read only) so developers can see what is the valid platform-specific normalized path of their pipe instance.

Make the System.IO.Pipes.PipeStream.GetPipePath static method public so developers can create valid pipe paths for a specific server name and pipe name combination.

Rationale and Usage

PipePath

Developers currently don't have a public API to check what is the path that represents their pipe instance in the current platform.

If we add this property, with only a get property accessor, they can consult the pipe name and can pass it across processes and across platforms.

The property could be accessed like this:

NamedPipeServerStream server = new NamedPipeServerStream(pipeName: "MyPipeName");
string serverPath = server.PipePath;

NamedPipeClientStream client = new NamedPipeClientStream(serverName: ".", pipeName: "MyPipeName");
string clientPath = client.PipePath;

// The value for both serverPath and clientPath would be:
// Windows  =   \\.\pipe\MyPipeName
// Unix     =   /tmp/CoreFxPipe_MyPipeName

GetPipePath

The existing GetPipePath static method is used internally to generate a valid platform-specific normalized pipe path using the passed server name and pipe name.

Developers currently don't have a public API to determine if a server name + pipe name combination can form a valid pipe path for the current platform. The only available workaround is to copy the GetPipePath code, which is not ideal: if our code changes, their code will be outdated.

If we expose this method, we can call it like this:

string path = PipeStream.GetPipePath(serverName: ".", pipeName: "MyPipeName");
// The value of path would be:
// Windows  =   \\.\pipe\MyPipeName
// Unix     =   /tmp/CoreFxPipe_MyPipeName

Proposed API

The internal method already exists, we only need to make it public:

namespace System.IO.Pipes
{
    ...
    public abstract partial class PipeStream : Stream
    {
        ...
        public virtual string PipePath { get; }
        public static string GetPipePath(string serverName, string pipeName);
    }
    ...
}

Classes that inherit from PipeStream can optionally override PipePath.
The NamedPipe*Stream classes should override it, while the AnonymousPipe*Stream classes may not (pending open question):

    public sealed partial class NamedPipeClientStream : PipeStream
    {
        ...
        public override string PipePath { get { ... } }
        ...
    }

    public sealed partial class NamedPipeServerStream : PipeStream
    {
        ...
        public override string PipePath { get { ... } }
        ...
    }

    public sealed partial class AnonymousPipeClientStream : PipeStream
    {
        ...
        public override string PipePath { get { ... } }
        ...
    }

    public sealed partial class AnonymousPipeServerStream : PipeStream
    {
        ...
        public override string PipePath { get { ... } }
        ...
    }

Details

• The GetPipePath method is static and currently none of the classes that inherit from PipeStream is overriding this method.

• We should add unit tests that perform the following checks on the GetPipePath method:

  • Windows:
  • The returned string has been normalized to the format: \\<serverName>\pipe\<pipeName>
  • Throws ArgumentOutOfRangeException if the returned pipe path is equal to the anonymous pipe name, which is reserved.
  • Throws the Path.GetFullPath exceptions if their conditions are met with the passed normalized formatted sting.
  • Unix:
  • If the pipe name is rooted, the pipe name string itself is returned.
  • If the pipe name is not rooted, the pipe prefix + the pipe name string is returned.
  • Throws PlatformNotSupportedException if:
    
    
    
    • The server name is anything else besides . or the current machine's hostname.
    
    
    • The pipe name contains invalid path name characters.
    
    
    • The pipe name ends in a directory separator character when the pipe name is rooted.
    
    
  • Throws ArgumentOutOfRangeException if the returned pipe path is equal to the anonymous pipe name, which is reserved by Windows.

• We should decide if the PipePath property getter should return the default value (either string.Empty or null) or if it should throw.

Open Questions

• What should the AnonymousPipe*Stream.PipePath property getter return? Should it return the \\.\pipe\anonymous string in Windows (and whatever the equivalent constant string in Unix)? Should it return null or should it return string.Empty?
• What should the default value of PipeStream.PipePath be, in case a child class does not override it?
• If the Windows anonymous pipe is expected to be \\.\pipe\anonymous, then what is the Unix equivalent? Is it \\tmp\CoreFxPipe_anonymous?
• Should we use the name PipePath or should we use another name to keep it consistent with similar APIs that expose a path? Update: could we use LocalPath, as it it already being used by Uri?

Pull Request

Pending approval of this API proposal.

Updates

• Edited the description to the official API Proposal format, copied the original description into a comment for historical purposes.
• Answered question of virtual vs abstract: Making the property abstract would break classes inheriting from PipeStream.

[JeremyKuhne]

I don't see a problem with exposing this on NamedPipeServerStream. @carlossanlop can you help get this to a more formal proposal?

[carlossanlop]

@casperOne thank you for suggesting this API. I am going to copy your original description and paste it below, and then I will substitute your description with the official API proposal.

@JeremyKuhne whenever you get a chance, would you mind taking a look at the proposal?

@casperOne commented on Mar 15:

I believe that exposing GetPipePath (or rather, an instance property that exposes the pipe path for that PipeStream instance) would be beneficial for those using PipeStream.

I'm.currently using a NamedPipeServerStream, generating pipes to pass to a command-line application which is not .NET that I don't have control over (FFMPEG).

I'm developing this on Windows, but deploying to an Ubuntu container in the cloud, so I'd like to be able to not have to use platform-specific code to generate the pipe names to pass to the external process (and I imagine there are many others that would like to do this as well).

I can get around this for now, copying the internal GetPipePath code in the Unix and Windows-specific PipeStream implementations, but I do worry about getting out-of-sync with the PipeStream implementation.

[scalablecory]

• Should we use the name PipePath or should we use another name to keep it consistent with similar APIs that expose a path?

e.g. Uri.LocalPath.

[scalablecory]

• Should the PipeStream.PipePath property be abstract or virtual? This class is abstract anyway, so making the property abstract would ensure it always gets implemented by a child class. If it's made virtual, it can be overriden but can also have a default value if it is not overriden.

Making it abstract will break anyone currently implementing PipeStream.

[carlossanlop]

Thanks @scalablecory, you're correct. I updated the description.

[stephentoub]

What is the use case for this method? How will it be used? The underlying implementation for these types differs across platforms. I also don't understand the relevance of this to anonymous pipes.

[casperOne]

@stephentoub The original description (which had the reason) was copied and pasted into @carlossanlop here:

https://github.com/dotnet/corefx/issues/36069#issuecomment-524496416

[stephentoub]

You say you're trying to consume the pipes this creates from another process on Unix... but NamedPipeServer/ClientStream doesn't actually use Unix "named pipes". Can you clarify? This is part of my concern. Exposing this path leaks an implementation detail.

[casperOne]

@stephentoub I'm doing something like this:

https://stackoverflow.com/q/32584220/50776

Namely, I'm using NamedPipeServer to create the named pipe, then I am passing the path of the pipe to FFMPEG (something like \\pipe\.\...) where it writes it's output to.

I'm then consuming the data from the NamedPipeServer.

I'm developing this on Windows, but I'll ultimately deploy this on Unix.

[carlossanlop]

@stephentoub when you have an instance of a NamedPipe*Stream class, you cannot consult what its pipe path is, unless you know what the constructor arguments were when the instance was created. @casperOne explained his problem in the original description, which I also backed up in a comment.

I added a couple usage examples in the proposal. The usage is simple. Should I add more?

The implementation differs depending on the platform, yes. The named pipe path string format is different in Unix compared to Windows. I am of the opinion that we should make sure to explain this in the documentation: the string returned by the method depends on the platform where you call it.

Regarding anonymous pipe streams, it's my understanding that they do have a predefined name. In Windows, it's supposed to be \\.\pipes\anonymous. in Unix I am not sure, which is why I added this in the Open Questions section.

I looked at the code that creates the anonymous pipe path and I didn't see the string being used, but there are places were we explicitly forbid the usage of the word "anonymous" for a pipeName, precisely because it is reserved. I will ask @JeremyKuhne about this.

[stephentoub]

The usage is simple.

Is it? Help me understand exactly what you plan to do with this path on Unix, where it's currently actually a path for a Unix domain socket.

[casperOne]

@stephentoub I believe what you're saying is that on Unix, .NET is not using mkfifo for the named pipe, is that correct?

The 30k ft. view is this:

I want to tell an external process (FFMpeg in this case) to write it's output to something that I can utilize a producer/consumer model on; named pipes seems to serve this very well and the assumption (which may be incorrect on my part) may be wrong here.

If there is some other way, what would it be? I'm looking to avoid the following:

• Waiting for the full file to be written before processing the bytes; there's no reason to incur the storage cost on some interim medium (at least, explicitly, not thinking about paging out memory, etc.)
• Explicitly using some sort of interim storage; on Windows, named pipes is generally faster than saving to disk, and the assumption (again, which may be incorrect) was that it would be the same on Unix.

I'm open to approaching this a different way, but I need to be able to pass some sort of named file/handle to FFMpeg to tell it where to write.

[stephentoub]

.NET is not using mkfifo for the named pipe, is that correct?

Correct. It's not actually a "named pipe" at all; the semantics of named pipes on Windows are on Unix much closer to the semantics of Unix domain sockets than to those of Unix named pipes, so NamedPipeClient/ServerStream use domain sockets under the covers. Any path exposed would be for a domain socket, not a fifo.

[carlossanlop]

Is it? Help me understand exactly what you plan to do with this path on Unix, where it's currently actually a path for a Unix domain socket.

Ah, I was thinking usage example referred to how we are going to call the API. What we want to know is examples of scenarios where the API is going to be used. Rookie mistake :smile: , I appreciate the clarification. I believe @casperOne explained a usage scenario in his last couple of replies.

[casperOne]

@stephentoub Good news then! FFMpeg is able to read from Unix domain sockets:

https://lists.ffmpeg.org/pipermail/ffmpeg-user/2017-May/036134.html

With a list of protocols here:

https://lists.ffmpeg.org/pipermail/libav-user/2012-May/001901.html

That said, I'd need to get something from the named pipe in order to pass to FFMpeg.

Or are named pipes the wrong abstraction here (given the points above)?

[carlossanlop]

@casperOne I'll assign the issue to you to continue the API proposal discussion. Feel free to edit the proposal description as you see necessary; if you can also add your use case examples, that would be great.
It's now my understanding that the api-ready-for-review label is not yet needed so I will remove it for now.

[scalablecory]

What is the harm in exposing the unix domain socket path? This doesn't seem like an implementation detail that we will change in the future, as it would break future versions.

[stephentoub]

This doesn't seem like an implementation detail that we will change in the future, as it would break future versions.

It would break cross-version communication, as would something as trivial as changing the naming convention used. I'm not sure that would stop us in a major release if we had a good reason.

What is the harm in exposing the unix domain socket path?

If you want to use Unix domain sockets, why not just use Socket, which supports UDS way better than NamedPipeServerStream does?

My concern is that it's exposing an easily misinterpreted implementation detail. That doesn't mean we can't, but we better have a really good use case for it, as there is downside.

[eiriktsarpalis]

Is there a real use-case for making the PipePath property overrideable? If so, wouldn't that contradict what the GetPipePath method is trying to achieve?