How can I POST an HTTP request and wait for a callback without blocking the current thread?

.net, asynchronous, c#, httplistener, task-parallel-library

Solution

`async`-`await` together with `TaskCompletionSource` pretty much were made for this.

The sender side creates a `TaskCompletionSource`, adds it to a dictionary (with key being the id of the request), makes the request and returns the `TaskCompletionSource`'s `Task`.

The receiver then looks into the dictionary to find the right `TaskCompletionSource`, removes it from there and sets its result.

The caller of the sender method will `await` the returned `Task`, which will asynchronously wait for the receiver to process the callback call.

In code, it could look something like this:

// TODO: this probably needs to be thread-safe
// you can use ConcurrentDictionary for that
Dictionary<int, TaskCompletionSource<Result>> requestTcses;

public async Task<Result> MakeApiRequestAsync()
{
   int id = …;
   var tcs = new TaskCompletionSource<Result>();
   requestTcses.Add(id, tcs);
   await SendRequestAsync(id);
   return await tcs.Task;
}

…

var result = await MakeApiRequest();

var context = await _httpListener.GetContext();

// parse the response into id and result

var tcs = requestTcses[id];
requestTcses.Remove(id);
tcs.SetResult(result);

Problem

We have a .NET application that uses an HTTP based API where we `POST` a request to a third party HTTP endpoint (that is not under our control) and it calls us back at a later time on an HTTP endpoint that we give it; something like: ``` WebRequest request = WebRequest.Create(urlToMethod); request.Method = @"POST"; request.Headers.Add(@"Callback", "http://ourserver?id="+id ); ``` We make thousands upon thousands of these calls and so we'd like to be as effecient as possible (in terms of speed/memory/threads etc.) As far as the callback code is concerned, we have a type that acts as a listener; this is how we start it up: ``` _httpListener = new HttpListener(); _httpListener.Prefixes.Add(ourServer); _httpListener.Start(); _httpListener.BeginGetContext(callback, null); ``` When the server calls us back, it hits our `callback` method which looks something like this: ``` HttpListenerContext context = _httpListener.EndGetContext(result); HttpListenerResponse httpListenerResponse = context.Response; httpListenerResponse.StatusCode = 200; httpListenerResponse.ContentLength64 = _acknowledgementBytes.Length; var output = httpListenerResponse.OutputStream; output.Write(_acknowledgementBytes, 0, _acknowledgementBytes.Length); context.Response.Close(); var handler = ResponseReceived; if (handler != null) { handler(this, someData); } ``` So we have a single instance of this listener (_which internally uses `HttpListener`) and for every response it gets, it informs all of the subscribers on the `ResponseReceived` event. The subscribers (possibly hundreds of them) only care about data associated with their particular `id`. The `subscriber`s look something like: ``` _matchingResponseReceived = new ManualResetEventSlim(false); _listener.WhenResponseReceived += checkTheIdOfWhatWeGetAndSetTheEventIfItMatches; postTheMessage(); _matchingResponseReceived.Wait(someTimeout); ``` It's that last line that's bugging me. We post the message but then block the whole thread waiting for the `Listener` to get a response and call our event handler. We'd like to use `Task`s but doesn't seem like it'll give us much if we're blocking a whole thread waiting for the callback. Is there a better (more TPL friendly) way of achieving this so that no threads are blocked and we get fire off more requests simultaneously?

Original source