Docs: New topics: A guided tour of the Task Asynchronous Protocol (TAP)

[BillWagner]

Async programming is a complicated topic. While async, await and the Task based protocols do a lot to make it more approachable, as developers get beyond the basic scenarios. MicrosoftDocs/azure-docs#5963 has a lengthy discussion that surfaces many of the questions developers have about async and await:

1. The distinctions between a synchronous method that returns an awaitable object (like Task) and an async method is subtle. Developers can be confused about when the async modifier is needed. Developers have concerns about possible performance implications of the async state machine.
2. Asynchronous programming and multi-threaded programming are related concepts. The nuances cause confusion for developers. Task based asynchrony vs. TPL (Task Parallel Library) vs. Threads. Developers need more clarity on the distinction between these tools and the scenarios they target.
3. More on async vs. parallel: Developers need more guidance on how async can increase throughput and efficient use of the threadpool resources.
4. SynchronizationContext is not needed for basic scenarios, but surfaces in many intermediate and advanced uses. Developers need more tutorials that show how this is used.
5. While on the subject of SynchronizationContext, developers have questions on context affinity and when ConfigureAwait is needed.

We have a topic on async and await and another section on async concepts. Some of the above topics are covered there, but may not be surfaced as well as needed.

I've added the discussion label to this issue. What else regarding async isn't covered yet that should be? As you comment, note that there are other issues related to async docs already created.

[Mike-E-angelo]

Awesome, thank you @BillWagner for creating this. I will start out with my own list of identified grievances which have been captured in a (well-ignored :wink:) UserVoice vote here. This will save me a lot of typing here. Yeah, I'm lazy, deal with it. 😆

My primary confusion at the moment is untangling the twine between TPL and async. @Rick-Anderson demonstrates this for me perfectly with a quote from https://github.com/MicrosoftDocs/azure-docs/issues/5963#issuecomment-382923582:

I don't think you want to go the TPL route. AFAIK, for real web apps, the async/await approach is the most productive and highly preformant.

So, by "TPL" that means using the Task promises, I am assuming? So if you use a simple Task, does that mean it is not as performant/magical as using an async/await keywords? It is worth noting that going the TPL route happens regardless (if I understand correctly) as both approaches here involve a Task promise, which is found in the TPL. Perhaps TPL means something other than Task Parallel Library? ALL THESE QUESTIONS AND SO MUCH MORE!!! 😆

The other meaningful goal here is to ensure best practices and performant behavior as the first question suggests. Consider the FunctionInvoker, which already has an async/await applied to it. It, in turn, could also have an async/await applied to it with a call from upstream, and even further if a call to _that_ also has async/await applied to it.

If the user also applies a async/await to their own custom code, that is at least three asynchronous state machines that are created magically in code, providing their own set of generated code and necessary overhead to complete them. The question is: is that really necessary? Shouldn't the default guidance and design pattern instead be:

_Do NOT use the async/await keywords unless absolutely necessary?_ Simply return a task instead if you can manage.

This is especially pertinent to framework code such as FunctionInvoker which will be called each and every time a request is made.

Consider that this not only reduces generated/necessary overhead necessary to execute the code (performance implications), but it also reduces the overall ceremony needed to write code, which makes the code more readable, which, incidentally, addresses one of my issues found above in the vote. (Yeah, I'm shameless, deal with it. 😄)

[Mike-E-angelo]

Actually @BillWagner I shouldn't be so weary of sharing that vote here. In addition to the items listed in the vote, I have kept track within the comments a running tally of all the instances where I have found TAP-based confusion caused in the MSFT ecosystem... even by MSFT employees, themselves.

[BillWagner]

One clarification:

This issue is about the Task - based Asynchronous Pattern (TAP), not the Task Parallel Library (TPL).

[Mike-E-angelo]

Ah, right. I think that speaks to items 2 and 3 in your list, @BillWagner. The Task is located in the System.Threading.Tasks namespace, and the first sentence out of TPL is "The Task Parallel Library (TPL) is a set of public types and APIs in the System.Threading and System.Threading.Tasks namespaces."

So it would seem that TAP is a subset of the TPL? We are calling it a "pattern" but it seems more like an API. I am not sure what makes this a pattern outside that we are using the API as designed, so again more confusion you will have to pardon here.

In any case, it does seem that the TPL is an older form of asynchronous/parallel functionality, of which Task is contained and considered part of it. It would further seem that all the magic, focus, and energy has exclusively been around the Task promise object, especially for the past few years. So my impression is that Task (or TAP) is the new TPL, or rather, the part of TPL that is worth learning and getting confused about. 😉

Speaking of which, I am not sure if you read the comments of that TAP link you just provided, but I am glad to see I am not the only one getting confused upsidedown over this whole matter. 😆

[svick]

@Mike-EEE

Do NOT use the async/await keywords unless absolutely necessary? Simply return a task instead if you can manage.

Stephen Cleary wrote a whole blog post on this topic, where he argues the opposite. The performance gains from avoiding async-await are relatively small and don't outweigh the pitfalls in the common case.

And even if the conclusion was that avoiding async-await should be recommended, I think the wording should be much weaker than what you're proposing.

This is especially pertinent to framework code such as FunctionInvoker which will be called each and every time a request is made.

Performance-critical code often uses different patterns than code that's not performance-critical. But that doesn't mean such patterns should be generally recommended, especially in a guide to make asynchronous programming "more approachable" (i.e. one that's not aimed at framework developers).

---

BTW, the overhead of async-await improved in .Net Core 2.1, which I think decreases the importance of avoiding that overhead. For example consider this benchmark:

```c#
public class Bench
{
static async Task M()
{
await Task.Yield();
}

[Benchmark]
public async Task WithAwait()
{
    await M();
}

[Benchmark]
public Task WithoutAwait()
{
    return M();
}

}
```

The results are:

| Method | Toolchain | Mean | Error | StdDev | Gen 0 | Allocated |
|------------- |-------------- |---------:|----------:|----------:|-------:|----------:|
| WithAwait | .NET Core 2.0 | 5.427 us | 0.1600 us | 0.4716 us | 0.1488 | 472 B |
| WithoutAwait | .NET Core 2.0 | 4.931 us | 0.1252 us | 0.3571 us | 0.0763 | 256 B |
| WithAwait | .NET Core 2.1 | 2.459 us | 0.0487 us | 0.0616 us | 0.1030 | 329 B |
| WithoutAwait | .NET Core 2.1 | 1.840 us | 0.0364 us | 0.0618 us | 0.0648 | 216 B |

Notice how the version with one more await on .Net Core 2.1 (preview 2) is faster than the version without the await on .Net Core 2.0.

[Mike-E-angelo]

LOL @svick that article is awesome. Thank you for sharing it in addition to the great information about .NET Core and benchmarks. I laugh here because apparently I have already read that article, posting a comment under it while complaining about async then as much as I am now. 🤣

I hope everyone here can appreciate the sheer amount of documentation, discussion, dissent, and confusion this topic has caused. We talk about pitfalls in approaches, but the biggest one of all in my view is the sheer amount of _cost_ it consumes in learning it, followed by the sheer amount of subsequent cost in relearning it, as you learned that you didn't learn it correctly the first time (or second or _third_ time as it is in my case).

Finding incredibly valuable and relevant information such as Mr. Cleary's excellent resource above proves to be challenging and sparse. For instance, I had no idea that _eliding_ was a word. This is especially compounded by the fact that I have clearly already read about the subject (that reflects more on me, I know, but work with me here). These articles and the articles they reference should be incorporated in this new documentation somehow, if possible. I may be in the minority in feeling this way, of course.

For instance, eliding appears to be a topic to itself, demonstrating that there are two camps who favor the differing approaches: ones that prefer to elide (Toub), and ones that do not (Cleary). Even Cleary admits to being in the former, but of course is now in the latter. Or at least he was in 2016. 😉 There does seem to be a desire towards eliding to make code simpler and easier to read, but of course there are pitfalls. The REAL answer to this problem it would seem is to provide framework intervention to mitigate these pitfalls so both camps can enjoy their approaches without these associated risks, but this of course is outside of the scope of this issue.

In any case, very valuable post, @svick. It is very much appreciated from my point of view.

Since I have your incredibly knowledgeable mind here, perhaps you can answer one last outstanding question of mine in regards to eliding: is there anything magical about the async keyword that enlists a method into parallelism? For instance, in an Azure Function (or ASP.NET Core) context, if the method does not contain async/await, does this keep it from participating in the renowned throughput gains and benefits that are within the ASP.NET Core runtime? From what I understand it is the Task that really matters, not the async keyword.

[svick]

@Mike-EEE

is there anything magical about the async keyword that enlists a method into parallelism?

No, async is not magic and it's not really related to parallelism either. If you have a framework like ASP.NET, that can execute your methods in parallel (e.g. because multiple requests arrive at the same time), then it usually supports parallelism whether you use async, Task, or neither. It's just that when you use async properly, the degree of parallelism can be higher, because async lets the framework reuse the thread that is executing your code.

In theory, the framework could find out if a method is async by detecting whether it has the AsyncStateMachine attribute applied. But I don't think frameworks do that.

For instance, in an Azure Function (or ASP.NET Core) context, if the method does not contain async/await, does this keep it from participating in the renowned throughput gains and benefits that are within the ASP.NET Core runtime? From what I understand it is the Task that really matters, not the async keyword.

To be completely accurate, it's not really about either. What matters is that you don't block the thread unnecessarily (e.g. when doing IO operations) and that your method returns a future that the framework you're using understands.

For example, ASP.NET Core MVC has a surprisingly (at least to me) robust system for working with controller action return types, which supports any awaitable type.

This means that the following two controller actions would both take advantage of the same "async" benefits, even though the second one doesn't use async or Task (full code):

```c#
public async Task AsyncTask()
{
var client = new WebClient();
var content = await client.DownloadStringTaskAsync("http://google.com");

return content.Length;

}

public CustomAwaitable CustomAwaitable()
{
var result = new CustomAwaitable();

var client = new WebClient();
client.DownloadStringCompleted += (s, e) => result.SetResult(e.Result.Length);
client.DownloadStringAsync(new Uri("http://google.com"));

return result;

}
```

(Keep in mind that the above code is just an example to explain the concept, I'm not suggesting that you should write code like that.)

The big advantage of async-await is that it makes writing asynchronous code much simpler.

Of course, frameworks other than ASP.NET Core might have much more limited support for futures, like only supporting Task.

[Mike-E-angelo]

OK... that pretty much aligns with my understanding of this, @svick. Thank you again for taking the time to explain this. I know it takes time to write all this out (it does for me, at least 😏), so it is appreciated. I suppose the upside of investing time into this thread is that it should find its way into the documentation at some point, that is its purpose after all.

The only remaining question I have is in regards to your statement here:

It's just that when you use async properly, the degree of parallelism can be higher, because async lets the framework reuse the thread that is executing your code.

Are you saying that while assuming that this is done via Task or any other IAwaitable (duck-typed or otherwise) promise object? That's sort of my driving factor here. I have heard so much about Asp.NET Core's performance being able to account for millions (or equivalent number) of requests per second. I am assuming this is only enabled/possible while using asynchronous processing via the promise types you are describing. If it is, then does using async/await keywords affect this throughput? My understanding is that it does _not_ impact throughput (but in fact adds a slight overhead as your benchmarks demonstrate), so that is what I am looking to definitively understand here. That is, the async/await keywords are not required to reach those metrics, and might actually impact them negatively when load reaches that sort of scale.

[nhwilly]

+1 for Cleary. Definitely "clearied" it up for me. 😀

Best explanation I've seen.

[Mike-E-angelo]

Wow, did the venerable @davidfowl just ghost this thread? I thought what you provided was valuable. FWIW, I am also interested in running ASP.NET Core performance metrics in regards to eliding at scale. I would greatly appreciate any pointers in the right direction to start hunting this down.

[davidfowl]

Yea, I misread the comment made by @svick 😏

[Mike-E-angelo]

Oh. Then I meant _formerly_ venerable @davidfowl. 🤣

[gortok]

Right way to do Async when database is involved synchronously (not all clients support async API) in a REST API.

[davidfowl]

Right way to do Async when database is involved synchronously (not all clients support async API) in a REST API.

You're kinda screwed there. There is no right way then, there are just trade offs of which is the least bad way.

[gortok]

@davidfowl I think a talk of the approaches and trade offs would be helpful; we have taken a variety of approaches over the life of our codebase. Uncontrolled parallelization+database client side connection pooling with long writes (Think data warehousing) has turned into a situation where our async message bus code causes database issues like connection pool exhaustion to the point that a temporary approach is to go sync (though that has outsized performance impacts).

[davidfowl]

Sounds like back pressure isn’t properly being applied. Unbounded parallel execution is bad in both cases but in the synchronous case you’re using the blocked thread as the thing stopping you from doing more work. The same can be done in the async case without burning a thread.

[olugt]

I followed this from the other issue about Azure Functions. I am commenting only on the aspect about when to use async and await versus Task (with or without async-await).

What I know is that if you're returning Task.FromResult(somethingToReturn) the method can simply return Task<TypeOfSomethingToReturn> without using async and await keywords. This is also because not all long-running operations inherently have Task somewhere being used. For example, a for loop that takes too long doesn't use Task.

So, you would ideally need async-await keywords when you have in use within the async-await method a method that returns a Task.