Corefxlab: Channels - TPL Dataflow

[grahamehorner]

I'm a little confused with regards to channels as the features look to be duplicating the dataflow? or is channels to replace dataflow? If channels is to replace dataflow is there a migration path/backwards compatibility planed?

[i3arnon]

Channels library is more focused on the specific scenario of handing data off between open-coded producers and consumers, and is optimized for that scenario, further expanding on the kinds of such buffers available and with implementations geared towards the relevant consumption models.

If these Channel interfaces were to become part of corefx, System.Threading.Tasks.Dataflow would likely take a dependency on System.Threading.Tasks.Channels as a lower-level set of abstractions

https://github.com/dotnet/corefxlab/tree/master/src/System.Threading.Tasks.Channels#relationship-with-tpl-dataflow

[grahamehorner]

@i3arnon but, dataflow has the following description:-

The Task Parallel Library (TPL) provides dataflow components to help increase the robustness of concurrency-enabled applications. These dataflow components are collectively referred to as the TPL Dataflow Library. This dataflow model promotes actor-based programming by providing in-process message passing for coarse-grained dataflow and pipelining tasks. The dataflow components build on the types and scheduling infrastructure of the TPL and integrate with the C#, Visual Basic, and F# language support for asynchronous programming. These dataflow components are useful when you have multiple operations that must communicate with one another asynchronously or when you want to process data as it becomes available.
....
The TPL Dataflow Library provides a foundation for message passing and parallelizing CPU-intensive and I/O-intensive applications that have high throughput and low latency. It also gives you explicit control over how data is buffered and moves around the system. To better understand the dataflow programming model, consider an application that asynchronously loads images from disk and creates a composite of those images. Traditional programming models typically require that you use callbacks and synchronization objects, such as locks, to coordinate tasks and access to shared data.

so in short dataflow & channels have duplicate features; which I feel should be broken out into shared library as apposed to duplication of functionality in different implementations?

[stephentoub]

so in short dataflow & channels have duplicate features; which I feel should be broken out into shared library as apposed to duplication of functionality in different implementations?

There is some overlap. But they're optimized for different things. If there are places where we could layer the dataflow library on top of channels, using primitives from the channels library to implement dataflow, we can explore that, but it would be as an implementation detail. The dataflow library has already shipped, so we won't be taking any breaking changes to its APIs in order to factor our shared portions, and there are sufficient differences in the APIs being exposed in the two libraries. The dataflow library is really focused on chaining together the "blocks" it exposes, whereas the channels library is really focused on using the core primitives directly as a hand-off between your own manually-written producers and consumers.

[stephentoub]

Closing as I think the questions have been answered. Please feel free to reopen if not.