Corefxlab: Support appending foreign buffers to pipe

[tmds]

Foreign buffers are buffers not allocated with the Pipe MemoryPool.

Use-cases for foreign buffers:

• custom buffers more appropriate for type of data to be sent
• send same buffer to multiple Pipes without copying the data in each Pipe

Comments:

• it needs to be possible to track the usage of the foreign buffer in order to know when it can be re-used
• some Transports may not be capable of sending foreign buffers (e.g. RIO) in which cases the buffer needs to be copied into the Pipe MemoryPool by the Transport

[mgravell]

/cc @Drawaes

This actually used to exist - it was called Append, took a ReadOnlyBuffer, and it is sorely missed. Strictly speaking, the pipe doesn't need to care where it comes from - it just needs to Preserve when accepting it, and Release when it is done (it already does the Release step). If it goes back to a memory pool: great! if it was actually a wrapped non-owned array, or a chunk from a completely unrelated pool: also fine!

Use cases:

• passing through a frame unchanged, common in many pipe processing daisy chains such as compression or encryption, or wrapped protocols that embed a payload in some other framing mechanism ("pipe over transparent web-socket" as an example I coded previously for "channels")
• passing chunks of pre-existing data unrelated to pipelines - for example, serializing a BLOB to a pipe in a serialization library where that means just "prefix with the length, then throw down the bytes" (note that this also extends to utf8 data that is stored as a BLOB rather than a string, for use with Utf8String)

Without this, "zero copy" isn't possible.

[Drawaes]

To be honest if Foreign buffers are a problem initially, then okay. But at the very least buffers from the same pool should be append able. Imagine say a protocol that reframes, at the moment you have to copy all the data across from one pipe to the other with either a small header added or removed, so as @mgravell says Zero copy is killed here.

I would prefer foreign buffers could be allowed also.

[Drawaes]

Another scenario for Append is to "stage" writes. Only a single thread can be "writing" to a pipeline at a time, which is fine. However sometimes you can have multiple threads building frames, you then serialize the write. However without Append this adds a copy to the processing.

Example

High performance trading applications often "stage" a buy and sell frame ready to send for many stocks. When something makes the decision to actually buy or sell you want to be able to just "append" that pre generated buffer without copying straight to the socket (or as quick to the socket as you can).

Today you don't see this scenario with ASP.NET because it tends to be a single thread servicing a single request/response at a time, however in Service to Service communications often a single pipe is used by many backend threads, and you want to keep the serialization of these to a minimum.

[ReubenBond]

This would be useful in Orleans where clients send request messages to a server which might then forward it on to the correct destination server. That's a common scenario for any distributed system which has a gateway/reverse proxy component (eg nginx, HAProxy, mongodb).

Currently we duplicate buffers when a message might be forwarded because of how our buffer pool works (no block-level ref counting). See https://github.com/dotnet/orleans/blob/a2e723da9b00638494c235f75fdca7c4fa8cb2e1/src/Orleans.Core/Messaging/IncomingMessageBuffer.cs#L181

[tmds]

Should this issue move to corefx repo?

[ahsonkhan]

Should this issue move to corefx repo?

Yes, please.

[muratg]

@tmds Closing this one. Please do the needful :)