Corefxlab: Thoughts on Span<T>

[davidfowl]

Some things I concluded about stack only spans:

• Unified representation of native and managed memory is super nice. Only the lowest level API needs to create the initial span. Everything else slices from it.
• Since you can't have span fields, you end up storing what I've been calling Span<T> promises. Basically all of the data you need to make a span on demand. You can expose the Span<T> as property and just computes the span from the span promise.
• C# not having generic pointers means you need to cast void* to specific T* when you want to do anything useful with native/unsafe code.
• Span needs pinning support. I the current prototype, there's no way to pin the span for interop. I happen to use either pinned byte[] or native memory in the pool so getting the unsafe pointer actually works for native interop.
• We should allow passing a Span<byte> to a native API directly. Like we do byte[].
• APIs that take (byte[], offset, length) are hard to interop with as you need to account for the Span representing native memory so there's no underlying array. We should look at this one. Some sort of bridge would be nice (for things like Stream).
• Not exposing the offset makes using RIO (winsock RIO extensions, basically zero copy networking on Windows) a little tricky just because you need to pass a buffer ID and an offset. It's easy to work around though.

/cc @KrzysztofCwalina @jkotas

[jkotas]

We should allow passing a Span<byte> to a native API directly
Span needs pinning support.

These are tracked in the uber Span<T> issue in CoreCLR repo - https://github.com/dotnet/coreclr/issues/5851

For pinning, the best way to do this is via property that returns the first element ref. So you would write:

fixed (int * p = &mySpanOfInt.UnsafeRef)
{
}

This should compile fine in C# 7 today (better name for the UnsafeRef property is welcomed).

Also, it may be nice to have a shortcut for it in C# that would compile into the above code.

[jkotas]

cc @jaredpar

[jaredpar]

Since you can't have span fields, you end up storing what I've been calling Span promises.

That's not quite specific enough. It's not possible to have Span<T> fields in class types, but it's completely fine to have it as a field inside a struct. That requires decent language support but it's very doable.

Span needs pinning support. I the current prototype, there's no way to pin the span for interop

Agree. Pinning was an important feature in the Midori version of this type and I expect we'll need it in our version too.

We should allow passing a Span to a native API directly. Like we do byte[].

In general I think PInvoke magic is bad. It's great in the hands of wizards but tends to trip up novices who don't understand all of the subtle rules which come along with it. I'd perefer the MCG approach where we let invocations get declared in higher level terms like Span<T> and spit out the correct blittable C# code.

Not exposing the offset makes using RIO (winsock RIO extensions, basically zero copy networking on Windows) a little tricky

I don't think there is anything necessarily wrong with exposing the offset.

[davidfowl]

That's not quite specific enough. It's not possible to have Span fields in class types, but it's completely fine to have it as a field inside a struct. That requires decent language support but it's very doable.

With the 2 Span plan (slow span and CLR span), would that work?

In general I think PInvoke magic is bad. It's great in the hands of wizards but tends to trip up novices who don't understand all of the subtle rules which come along with it. I'd perefer the MCG approach where we let invocations get declared in higher level terms like Span and spit out the correct blittable C# code.

If it was deeply integrated into the toolchain already I'd say sure but it's a .NET Native only thing that probably won't be integrated into the default msbuild pipeline for a while (correct me if I'm wrong).

I don't think there is anything necessarily wrong with exposing the offset.

That might be nice but I dunno if it makes the abstraction less pure.

[davidfowl]

cc @migueldeicaza

[jaredpar]

With the 2 Span plan (slow span and CLR span), would that work?

Yep. Should work fine.

That might be nice but I dunno if it makes the abstraction less pure.

Certainly less pure but functionality trumps purity IMHO. Perhaps we'll find a less direct way to expose it though (via an extension method, strongly worded property, etc ...)

[davidfowl]

@jaredpar @jkotas could we also support:

C# Span<byte> span = stackalloc byte[10];

[KrzysztofCwalina]

Yes! BTW, recently I have been thinking about renaming Span to Pointer. It seems like the main value prop of the type as it's currently designed is that it makes non-GC memory (native, stack, interior object) safe.

[jkotas]

Pointer (System.Reflection.Pointer) is taken...

[jaredpar]

@davidfowl or really any type of non-managed alloctation:

Span<byte> b = Marshal::CoTaskMemAlloc();

If you combine that with IDisposable it become a pretty C# -ish pattern

[KrzysztofCwalina]

@jkotas, System.Reflection.Pointer is non-generic.

[jaredpar]

I'm not a fan of Pointer. I think that's going to bring along too many bad memories / assumptions. In particular I feel htere is too much of as association with pointer and a single value where Span is specifically about a range.

Note: can't use Range eihter. That's the type I'm hoping to use in the compiler. Essentially consider the following:

string s = "hello world";
Span<char> s = s[0..5];

The way this works is the range operator 0...5 produces a value of type Range. It's a simple struct which has two fields: index and length. That is then passed to the string index operators and eventually picks

class string { 
  Span<char> this[Range r] => ...
}

[davidfowl]

What about Memory<T>?

[benaadams]

Probably could do Array<T> as it has no instantiation currently :grinning:

[KrzysztofCwalina]

Yeah, I thought about Array and liked it several months ago. But I think it would be confusing. The type is not super similar to _the array_.

Memory is an interesting option.

[benaadams]

Depends what you want it used for Memory<T> is a bit scary maybe at a level of T* or unsafe so people will shy away from using it as a default go to. Rather seeing as a similar level to T[].

Which means people will use ArraySegment<T> in preference to Memory<T> and ReadOnlyMemory<T> even though they are both actually safer than ArraySegement<T> as they only capture a portion of the array rather than giving naked access to the full array which ArraySegment does and one of them is even readonly! :smile:

So ignoring the native memory aspect they are safer than both ArraySegment<T> and T[], offset, length in common usage?

[mstrobel]

Span<T> may not be a particularly sexy name, but it's concise, accurate, and it doesn't clash with existing types. FWIW, when I first read about the proposal for Span<T> here, I actually thought, "nice, that's a good name."

Using Array<T> would be confusing for newbies and veterans alike, and I have to agree with @benaadams that Memory<T> comes off as less approachable. I'd avoid any names involving the words 'memory' or 'pointer'. Such names imply more constrained use cases related to unsafe or interop scenarios. In effect, they say, "you shouldn't be using me as part of a public API."

I would preemptively strike Buffer<T> and Blob<T> too, as they also imply more specific use cases.

[davidfowl]

this type is a type and memory safe representation of memory. It's basically a T* which is partially why it has all of the same bizzare restrictions and also unsafe operations.

[adamsitnik]

Why can't we use the initial name : Slice? Javascript, Rust, Go, C++ and Python use this name. Java don't have such thing.

[davidfowl]

I don't think this compares tbh. Slice also doesn't convey the pointer side of the story and that's one of the main benefits and reasons for the bizzare restrictions (stack only)

[mstrobel]

Why can't we use the initial name : Slice?

You cannot have a Slice method on a Slice<T> type because the names collide, and for consistency's sake it is desirable for all types with slicing methods to use the same method name. Since Slice is such an obvious and suitable name for a slicing method, we're left needing a different name for this type. I agree with this decision: if you can't use the name in both places, use it for the method.

this type is a type and memory safe representation of memory. It's basically a T* which is partially why it has all of the same bizzare restrictions and also unsafe operations.

I suppose this depends on what you consider to be the primary purpose of this type. I would say that the primary purpose is to expose a range of values for efficient random access. In many cases, perhaps even in the common case, that range will be a slice of some larger range. In such cases, the resemblance to a T* pointer is incidental, and the use of a pointer under the hood is an implementation detail. That Span<T> provides pointer-like functionality is useful, but I'm not sure that's the core concept that should be reflected in the name.

[buybackoff]

Having deeply touched html recently, Span is so natural to express the meaning of some piece of data in between of bigger data (not because of html, but semantics, the word conveys the intent very well). Memory is very confusing, because it makes me think about native memory and C world of unsafe pointers. Span is safe and not memory, but a safe abstraction around it - be it managed or native. Slice as a verb here fits naturally (method name), so when we do Slice() a Span we get another Span. My 2 c.

[davidfowl]

Look at how beautiful that is.

https://github.com/davidfowl/Channels/blob/master/test/Channels.Tests/WritableBufferFacts.cs#L302-L323

We're just leaning on different sides of the capability of Span<T>. You realize you can't use it like you use an array right? You can't store it in a field for example (unless you can guarantee it's only ever on the stack). You can't use it in generic arguments (because it has the same restrictions as a pointer). Maybe we should split Span<T> and Slice<T>/Memory<T>. I want the unification of managed and native memory in a type safe way. That may be different to "a piece of an array" that basically looks like an array.

[mstrobel]

Look at how beautiful that is.

Love it! Similar in concept to Java's 'direct' (unmanaged) version of ByteBuffer, but potentially far more efficient due to reduced copying.

I want the unification of managed and native memory in a type safe way. That may be different to "a piece of an array" that basically looks like an array.

Agreed on both points. But it looks like you've already built a nice suite of complex memory I/O on top of Span<byte> with your WriteableBuffer. Correct me if I'm wrong, but it looks like Span<T> already gives you the functionality you need, in addition to providing generalized array-like random access to some range of values. So what is there to split? What's missing from your point of view? The only thing we seem to actually disagree on is whether Span<T> or Memory<T> is a more appropriate name.

Okay, looking closer at your Channels project, I can see you depend on Span.Read<T> and Span.Write<T>, which are not included in this implementation. The version you depend on does indeed seem like it's doing the work of two separate types. Given that the memory I/O methods are missing from the current implementation, perhaps the coreclr guys agreed.

[mgravell]

@mstrobel the case of Read etc methods for Span<byte> is a bit of a special-case, conveniently addressed by extension methods against Span<byte> (not Span<T>). See SpanExtensions etc

[jzabroski]

Hi @davidfowl

This seems to be the thread to dump "thoughts on Span", so if I may contribute some of mine, here are my thoughts:

• Haskell has a seamless way to think of "threads of computation" across state (memory). It's the Pipes project, and on top of Pipes, they have Pipes.Parse. Pipes.Parse builds upon Pipes to add several features missing from the core concept of a pipe:
  
  
  
  • End-of-input detection, so that Parsers can react to an exhausted input stream
  
  
  • Leftovers support, which simplifies several parsing problems
  
  
  • Connect-and-resume, to connect a Producer to a Parser and retrieve unused input
  
  

In this way, a Parser is just a special kind of Pipe-Consumer, with several handy "extension methods" like zoom that allow you to delimit a Parser into a Lens'. Suppose you had a Span.ReadAll. zoom in Haskell would let you scope the ReadAll operation to a subset of the overall Span, chunking it into smaller semantic units, presumably with different read-write cache properties. Imagine lifting a whole output buffer into a template that you can then view through the Lens and only plug in your write-able properties. That is essentially what this abstraction gives you - a really high-level abstraction of what we can do today in .NET through string template interpolation.

Once you have the formal notion of a sub-set of a Span that can be written to, you can then implement a high-level library for donut caching just about anything.

[KrzysztofCwalina]

Thanks for all the feedback and discussion.