WCF is slow when reliable session is ON and with burst async request

.net, nettcpbinding, wcf, ws-reliablemessaging

Solution

Found a partial workaround for the problem from the below links:

- http://blogs.msdn.com/b/endpoint/archive/2011/05/04/wcf-scales-up-slowly-with-bursts-of-work.aspx

- http://support.microsoft.com/kb/2538826

With the workaround (using the WorkerThreadPoolBehavior), the measured throughputs are as follows:

- Async + NetTcpBinding/Reliable throughput -> 474 call/s (2.1 ms/call) ... improved but not satisfactorily

- Async + WsHttp/Reliable throughput -> 7856 call/s (0.13 ms/call) ... no change

- Sync + NetTcpBinding/Reliable throughtput -> 2110 call/s 0.47 ms/call) ... degraded

Note that the case 1 above is improved significantly from 70 ms/call. However, it still lags from case 2. And for case 3, introducing WorkerThreadPool behavior cause performance degradation from 0.25 ms/call to 0.47 ms/call.

Problem

For experiments, I created a simple "Hello World" WCF service and client using .NET 4.5 on VS2012. The server is hosted on a console application and use net.tcp binding. I wrote the client code to send a burst of async requests (a total of 700 requests) to the server. Everything went fine until I turned ON the Reliable session feature of the service. After it is ON, the service suddenly ran very slow and it took almost a minute on my machine to complete the 700 requests. I tried to fine tune the Concurrency and Throttling parameters (see below) but it didn't help. Does anyone know why this happen ? Is there anyway to avoid this ? The slowness didn't happen if I turned OFF the Reliable session feature, or if I made the service call synchronous. So I think it may relate to the way WCF handles pending requests in WS-ReliableMessaging mode. EDIT: Also this didn't happen when I chaned netTcpBinding to wsHttpBinding. This is very weird because in this case wsHttpBinding is much faster than netTcpBinding. EDIT: Running Perfmon.exe on the server side shows that the "Thread Count" gradually increase from 8 to beyond 100 in the above case. EDIT: Some measured throughput on my PC (local network). See that the performance of case 1 is very sluggish and practically useless. - Async + NetTcpBinding/Reliable throughput -> approx. 14 call/s (70 ms/call) - Async + WsHttp/Reliable throughput -> 7957 call/s (0.12 ms/call) - Sync + NetTcpBinding/Reliable throughtput -> 3986 call/s (0.25 ms/call) Below are the codes and configuration for the server and client I used in the experiments. Server: ``` using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using System.ServiceModel; using System.ServiceModel.Description; [ServiceContract] public interface IHelloService { [OperationContract(IsOneWay=false)] string SayHello(string name); } [ServiceBehavior(ConcurrencyMode=ConcurrencyMode.Multiple, InstanceContextMode=InstanceContextMode.PerSession)] public class HelloService : IHelloService { public string SayHello(string name) { String s = string.Format("Hello {0}", name); return s; } } namespace WcfServer { class Program { static void Main(string[] args) { Uri baseAddress = new Uri("net.tcp://localhost:8080/hello"); using (ServiceHost host = new ServiceHost(typeof(HelloService), baseAddress)){ // Open and listen host.Open(); Console.WriteLine("The service is ready at {0}", baseAddress); Console.WriteLine("Press <Enter> to stop the service."); Console.ReadLine(); // Close the ServiceHost. host.Close(); } } } } ``` Client: ``` using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using System.ServiceModel; using WcfClient.WcfServer; namespace WcfClient { class Program { static async Task PrintNameAsync(HelloServiceClient client, int cnt) { string s = await client.SayHelloAsync(string.Format("-- {0} --", cnt)); Console.WriteLine(s); } static void Main(string[] args) { HelloServiceClient client = new HelloServiceClient("HelloService", "net.tcp://10.20.61.13:8080/hello"); List<Task> tasks = new List<Task>(); for(int i=0; i < 700; i++){ Task t = PrintNameAsync(client, i); tasks.Add(t); } Task.WhenAll(tasks).Wait(); client.Close(); } } } ``` Server's App.config: ``` <?xml version="1.0" encoding="utf-8" ?> <configuration> <startup> <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" /> </startup> <system.serviceModel> <bindings> <netTcpBinding> <binding name="HelloServiceBinding"> <reliableSession ordered="true" enabled="true" /> <security mode="None" /> </binding> </netTcpBinding> </bindings> <behaviors> <serviceBehaviors> <behavior name="HelloServiceBehavior"> <serviceMetadata policyVersion="Policy15" /> <serviceDebug includeExceptionDetailInFaults="true" /> <serviceThrottling maxConcurrentCalls="1000" maxConcurrentSessions="1000" maxConcurrentInstances="1000" /> </behavior> </serviceBehaviors> </behaviors> <services> <service behaviorConfiguration="HelloServiceBehavior" name="HelloService"> <endpoint address="net.tcp://localhost:8080/hello" binding="netTcpBinding" bindingConfiguration="HelloServiceBinding" name="HelloService" contract="IHelloService" /> <endpoint address="mex" binding="mexTcpBinding" contract="IMetadataExchange" /> </service> </services> </system.serviceModel> </configuration> ``` Client's App.config: ``` <?xml version="1.0" encoding="utf-8" ?> <configuration> <startup> <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" /> </startup> <system.serviceModel> <bindings> <netTcpBinding> <binding name="HelloServiceBinding" sendTimeout="00:01:00"> <reliableSession enabled="true" /> <security mode="None" /> </binding> </netTcpBinding> </bindings> <client> <endpoint address="net.tcp://localhost:8080/hello" binding="netTcpBinding" bindingConfiguration="HelloServiceBinding" contract="WcfServer.IHelloService" name="HelloService"> </endpoint> </client> </system.serviceModel> </configuration> ```

Original source