How does C# async/await relates to more general constructs, e.g. F# workflows or monads?
async-await, c#, f#, monads
Solution
The asynchronous programming model in C# is very similar to asynchronous workflows in F#, which are an instance of the general monad pattern. In fact, the C# iterator syntax is also an instance of this pattern, although it needs some additional structure, so it is not just simple monad.
Explaining this is well beyond the scope of a single SO answer, but let me explain the key ideas.
Monadic operations. The C# async essentially consists of two primitive operations. You can `await` an asynchronous computation and you can `return` the result from an asynchronous computation (in the first case, this is done using a new keyword, while in the second case, we're re-using a keyword that is already in the language).
If you were following the general pattern (monad) then you would translate the asynchronous code into calls to the following two operations:
Task<R> Bind<T, R>(Task<T> computation, Func<T, Task<R>> continuation);
Task<T> Return<T>(T value);
They can both be quite easily implemented using the standard task API - the first one is essentially a combination of `ContinueWith` and `Unwrap` and the second one simply creates a task that returns the value immediately. I'm going to use the above two operations, because they better capture the idea.
Translation. The key thing is to translate asynchronous code to normal code that uses the above operations.
Let's look at a case when we await an expression `e` and then assign the result to a variable `x` and evaluate expression (or statement block) `body` (in C#, you can await inside expression, but you could always translate that to code that first assigns the result to a variable):
[| var x = await e; body |]
= Bind(e, x => [| body |])
I'm using a notation that is quite common in programming languages. The meaning of `[| e |] = (...)` is that we translate the expression `e` (in "semantic brackets") to some other expression `(...)`.
In the above case, when you have an expression with `await e`, it is translated to the `Bind` operation and the body (the rest of the code following await) is pushed into a lambda function that is passed as a second parameter to `Bind`.
This is where the interesting thing happens! Instead of evaluating the rest of the code immediately (or blocking a thread while waiting), the `Bind` operation can run the asynchronous operation (represented by `e` which is of type `Task<T>`) and, when the operation completes, it can finally invoke the lambda function (continuation) to run the rest of the body.
The idea of the translation is that it turns ordinary code that returns some type `R` to a task that returns the value asynchronously - that is `Task<R>`. In the above equation, the return type of `Bind` is, indeed, a task. This is also why we need to translate `return`:
[| return e |]
= Return(e)
This is quite simple - when you have a resulting value and you want to return it, you simply wrap it in a task that immediately completes. This might sound useless, but remember that we need to return a `Task` because the `Bind` operation (and our entire translation) requires that.
Larger example. If you look at a larger example that contains multiple `await`s:
var x = await AsyncOperation();
return await x.AnotherAsyncOperation();
The code would be translated to something like this:
Bind(AsyncOperation(), x =>
Bind(x.AnotherAsyncOperation(), temp =>
Return(temp));
The key trick is that every `Bind` turns the rest of the code into a continuation (meaning that it can be evaluated when an asynchronous operation is completed).
Continuation monad. In C#, the async mechanism is not actually implemented using the above translation. The reason is that if you focus just on async, you can do a more efficient compilation (which is what C# does) and produce a state machine directly. However, the above is pretty much how asynchronous workflows work in F#. This is also the source of additional flexibility in F# - you can define your own `Bind` and `Return` to mean other things - such as operations for working with sequences, tracking logging, creating resumable computations or even combining asynchronous computations with sequences (async sequence can yield multiple results, but can also await).
The F# implementation is based on the continuation monad which means that `Task<T>` (actually, `Async<T>`) in F# is defined roughly like this:
Async<T> = Action<Action<T>>
That is, an asynchronous computation is some action. When you give it `Action<T>` (a continuation) as an argument, it will start doing some work and then, when it eventually finishes, it invokes this action that you specified. If you search for continuation monads, then I'm sure you can find better explanation of this in both C# and F#, so I'll stop here...
Problem
The C# language design have always (historically) been geared towards solving specific problems rather then finding to address the underlying general problems: see for example http://blogs.msdn.com/b/ericlippert/archive/2009/07/09/iterator-blocks-part-one.aspx for "IEnumerable vs. coroutines": We could have made it much more general. Our iterator blocks can be seen as a weak kind of coroutine. We could have chosen to implement full coroutines and just made iterator blocks a special case of coroutines. And of course, coroutines are in turn less general than first-class continuations; we could have implemented continuations, implemented coroutines in terms of continuations, and iterators in terms of coroutines. or http://blogs.msdn.com/b/wesdyer/archive/2008/01/11/the-marvels-of-monads.aspx for SelectMany as a surrogate for (some kind of) Monads: The C# type system is not powerful enough to create a generalized abstraction for monads which was the primary motivator for creating extension methods and the "query pattern" I do not want to ask why has been so (many good answers have been already given, especially in Eric's blog, which may apply to all these design decisions: from performance to increased complexity, both for the compiler and the programmer). What I am trying to understand is to which "general construct" the async/await keywords relate to (my best guess is the continuation monad - after all, F# async is implemented using workflows, which to my understanding is a continuation monad), and how they relate to it (how they differ?, what is missing?, why there is a gap, if any?) I'm looking for an answer similar to the Eric Lippert article I linked, but related to async/await instead of IEnumerable/yield. Edit: besides the great answers, some useful links to related questions and blog posts where suggested, I'm editing my question to list them: - A starting point for `bind` using await - Implementation details of the state machine behind await - Other details on how await gets compiled/rewritten - Alternative, hypothetical implementation using continuations (call/cc)