Asynchronously posting to Azure Queues

Question

I try to enqueue messages in Azure Queues asynchronously like this: ``` private async Task EnqueueItemsAsync(IEnumerable messages) { var tasks = messages.Select(msg => _queue.AddMessageAsync(new CloudQueueMessage(msg), null, null, null, null)); await Task.WhenAll(tasks); } ``` If I get it right this says "start enqueuing one item after the other without waiting them to get posted, keep a reference for each task and then wait until all get posted". This code works fine in most cases, but for a large number of items (5000), it starts enqueuing and then throws a timeout exception (after having enqueued ~3500 items). I solved it by waiting each one to finish before continuing with the next one ``` private async Task EnqueueItemsAsync(IEnumerable messages) { foreach (var message in messages) { await _queue.AddMessageAsync(new CloudQueueMessage(message), null, null, null, null); } } ``` Can anyone explain why this happened? Exception: `System.AggregateException` which wraps…

Accepted Answer

A queue in Azure is designed have a throughput of 2000 messages per second.

See: Azure Storage Scalability and Performance Targets

When your application reaches the limit of what a partition can handle for your workload, Azure Storage will begin to return error code 503 (Server Busy) or error code 500 (Operation Timeout) responses. When this occurs, the application should use an exponential backoff policy for retries. The exponential backoff allows the load on the partition to decrease, and to ease out spikes in traffic to that partition.

Asynchronously posting to Azure Queues

Solution

Problem