Runtime: Task.WhenAll support for Tuples

[petroemil]

Task.WhenAll should have a (set of) overload(s) to support returning the wrapped Tasks' results in a strongly typed Tuple.

An oversimplified implementation of this would look something like this

public static async Task<(T1, T2)> WhenAll<T1, T2>(Task<T1> task1, Task<T2> task2)
{
    return (await task1, await task2);
}

... and the usage would look something like this

var (x, y) = await Task.WhenAll(AsyncMethod1(), AsyncMethod2());

I believe it would be relatively easy to implement but its value is huge as it allows to write simple but very efficient code that executes in parallel.

Think about initialization logic when we have to gather information from different parts of the system, doing a bunch of async calls and either await-ing them one-by-one or collect the Tasks, throw them in a Task.WhenAll and then extract the Results.

With Tuple being supported on a language level with decomposition, I think this would be a logical addition to the Task library.

[petroemil]

Sorry, just realized this is the wrong repo :)
Opened an issue in the CoreCLR repo - where it belongs: #15397

[mikedn]

Actually this is the right repo because this would be a new API addition and this kind of issues are discussed here, even if the implementation is in coreclr.

[petroemil]

Should I just reopen this then and close the other one in the CoreCLR repo?

[mikedn]

Should I just reopen this then and close the other one in the CoreCLR repo?

Yes.

[petroemil]

You are right, this particular method signature would hide the original WhenAll(params Task[] tasks) option in most cases, the signature for this use case should be unique. Maybe it could be called WhenEach for the sake of differentiation, or it could expect to receive its parameters wrapped in a Tuple.

[mikedn]

You are right, this particular method signature would hide the original WhenAll(params Task[] tasks) option in most cases, the signature for this use case should be unique. Maybe it could be called WhenEach for the sake of differentiation, or it could expect to receive its parameters wrapped in a Tuple.

Well, my (deleted) comment was mostly incorrect, somehow I read the signature as public static (T1, T2) WhenAll<T1, T2>(Task<T1> task1, Task<T2> task2).

Now, it's is still true that adding this overload would make existing code call it instead of WhenAll(params). It's not clear if there are any downsides in doing this. I suppose it may affect performance.

An interesting alternative might be for the language to support awaiting a tuple of tasks:
C# (x, y) = await (tx, ty);
Not sure if this is something that language designers would agree to. What's for sure is await works well until you need to do things in parallel. Then it becomes a bit clumsy and it wouldn't hurt to improve this somehow.

[petroemil]

I didn't dare to dream that big, I think introducing a new API is much simpler than introducing a new language feature, but I agree, if that syntax would be possible, that would be quite amazing.

[stephentoub]

An interesting alternative might be for the language to support awaiting a tuple of tasks

FWIW, that doesn't require a new language feature, e.g. you can just write a GetAwaiter extension method for a tuple:
https://github.com/dotnet/corefx/issues/16010
https://gist.github.com/jnm2/3660db29457d391a34151f764bfe6ef7

[mikedn]

FWIW, that doesn't require a new language feature, e.g. you can just write a GetAwaiter extension method for a tuple:

Ah, of course! Magic :smile:

[petroemil]

In this case, I guess my question got answered, there is already a pretty great solution that is being properly implemented.

There is one thing I noticed, that under the hood that GetAwaiter extension is using Task.WhenAll(params Task[]) overload, which means it has a bit of extra allocation to turn those parameters into an array. Maybe if this would be properly supported by the Task APIs, there could be a more efficient implementation underneath.

But still... this is pure brilliant.

[karelz]

Duplicate of dotnet/runtime#20166
... I think we can close it that way.

[jnm2]

I updated the gist. tuple.ConfigureAwait was causing only the first two tasks to be awaited due to a copy/paste error. 🤦‍♂️

[TylerBrinkley]

I think this should be reopened as dotnet/runtime#20166 focused on adding similar support on Tuple's directly but suffered from a lack of visibility and had to add support for the void returning Tasks. This proposal however avoids most of the issues pointed out in that thread.

Adding these Task.WhenAll overloads would make them immediately visible to potential consumers, which is where they'll look first. It doesn't suggest adding overloads for the void returning Tasks because that overload already exists. Lastly I propose only adding support for up to an arity of 5 which should cover the 95%+ use case and thus reducing the amount of code bloat in the BCL.

To avoid the source breaking change issue with the params array overloads I instead propose specifying the parameter as a tuple of tasks which has the somewhat satisfying attribute of being symmetrical to its output being both tuples.

A bonus to this API is that it closely matches the Promise.all TypeScript overloads that many developers are probably familiar with.

Proposed API

c# namespace System.Threading.Tasks { public class Task { public static Task<(T1, T2)> WhenAll<T1, T2>((Task<T1>, Task<T2>) tasks); public static Task<(T1, T2, T3)> WhenAll<T1, T2>((Task<T1>, Task<T2>, Task<T3>) tasks); public static Task<(T1, T2, T3, T4)> WhenAll<T1, T2>((Task<T1>, Task<T2>, Task<T3>, Task<T4>) tasks); public static Task<(T1, T2, T3, T4, T5)> WhenAll<T1, T2>((Task<T1>, Task<T2>, Task<T3>, Task<T4>, Task<T5>) tasks); } }

[jnm2]

@TylerBrinkley

and begged to add support for the void returning Tasks.

Not sure I'm correctly parsing what "begged to add support" means, but I don't think void-resulting tasks were left out of my proposal there. See the '3. Void-returning awaiting' section in https://github.com/dotnet/corefx/issues/16010#issuecomment-371006308.

[jnm2]

I have examples of needing to use arities higher than 10. Maybe we could go up to the next standard .NET BCL level of having the max arity be 16?

[TylerBrinkley]

Not sure I'm correctly parsing what "begged to add support" means, but I don't think void-resulting tasks were left out of my proposal there. See the '3. Void-returning awaiting' section in #16010 (comment).

I know you included the void-resulting tasks, what I meant was that if you supported awaiting tuples directly it would be confusing if tuples of void-resulting tasks weren't also supported.

I have examples of needing to use arities higher than 10. Maybe we could go up to the next standard .NET BCL level of having the max arity be 16?

I think those would fall in the under 5% of the time use case, a number which I pulled out of nowhere. :smile: I'm not opposed to adding higher arities so long as it doesn't derail the higher used lower arities due to code bloat concerns.

[TylerBrinkley]

@stephentoub @karelz Could you reopen this issue since dotnet/runtime#20166 was rejected? As I specified above, my proposal doesn't suffer from many of the issues that the other proposal had.

[scalablecory]

@TylerBrinkley I don't know if we get enough value from this for it to make sense. It seems like a lot of async code does not have this kind of concurrency.