Some thoughts around Service Throttling on WCF Services

  • Article

One of this we had to work on an issue around how to implement the Service Throttling Behaviour in a set of highly accessible WCF Services, basically we want that the maximum load on each service to be controlled.

Of course, that we were aware that the effects of these settings are dependent on the transport (different in several of those services) and concurrency mode of the service (also different).

So, lets try by start to discuss the relevant configurations here.

 

MaxConnections

This sets a value that controls the maximum number of connections to be pooled for subsequent reuse on the client and the maximum number of connections allowed to be pending dispatch on the server

On the client side MaxConnections determines the maximum number of connections to be pooled for subsequent reuse.

On the service side MaxConnections determines the maximum number of connections allowed to be pending dispatch.

This configuration setting has different meaning depending upon if you are on the client side or the service side.

We really need to be carefull here, using a significant high number could generate a large queue and then generate some timeouts, so you need to take that in consideration when opted in by a high value.

Other point is that, in our case, we didn't  know the value of this configuration on the client (it could vary), if it’s the default, then that means that they can pool 10 connections to the Service, meaning that the connection could be cut-off from the client side.

 

MaxConcurrentInstances | MaxConcurrentCalls | MaxConcurrentSessions

These ones are those configurations that you can use when implementing the behavior “Service Throttling”

Accordingly with our documentation:

MaxConcurrentInstances, specifies the maximum number of InstanceContext objects in the service.

It is important to keep in mind the relationship between the MaxConcurrentInstances property and the InstanceContextMode property.

If InstanceContextMode is PerSession the resulting value is the total number of sessions. If InstanceContextMode is PerCall, the resulting value is the number of concurrent calls.

If a message arrives while the maximum number of InstanceContext objects already exist, the message is held until an InstanceContext object closes.

MaxConcurrentCalls specifies the maximum number of messages actively processing across all the dispatcher objects in a ServiceHost object.

Each channel can have one pending message that does not count against the value of MaxConcurrentCalls until Windows Communication Foundation (WCF) begins to process it.

MaxConcurrentSessions property specifies the maximum number of channels a ServiceHost can accept.

Each listener object can have one pending channel that does not count against the value of MaxConcurrentSessions until WCF accepts the channel and begins processing messages on it.

This property is most useful in scenarios that make use of sessions.

Please bear in mind that this configuration is impacted by the capacity of the client to create multiple proxies to connect to the Service, or else this configuration will have no impact at all.

A very important piece of information here is that a trace is written every time the value of this properties is exceeded. The first trace is written as a warning.

So, if this is the case you can face an issue of our clients  be disconnected, in that case we should be seeing some warnings on your Event Viewers on our Servers.

Another important point, that need to be discussed here would be, as previously referred, the Instance Mode to be used.

InstanceContextMode is set using the [ServiceBehavior] attribute and controls how WCF's ServiceHost how WCF will manage instances of our service class, here you have three options:

    • Single - Use a single instance of the class for all calls
    • PerCall - Create a new instance of the class for each call
    • PerSession (default) - Create a new instance of the class for each session that is initiated

In most cases, we should set this value to Single so that we aren't allocating objects for nothing (note the caveat with ConcurrencyMode see below). The default, however, is PerSession.

ConcurrencyMode is also set using the [ServiceBehavior] attribute and controls the threading behavior that WCF will use when working with our classes, again.

ConcurrencyMode has three options:

    • Single (default) - Only allow a single thread per service instance
    • Reentrant - Only allow a single caller per service instance (avoids double counting a single caller and potentially deadlocking if you call back into the service)
    • Multiple - Allow multiple threads to access the service instance

The default options for ConcurrencyMode (Single) and InstanceContextMode (PerSession) don't offer the best performance.

Also, some other aspects that we need to consider is that "strange" fact that WCF sessions are not web sessions.

Unlike web sessions, which are managed by the server and generally tracked using http cookies, WCF sessions are initiated by the client proxy and don't end until they time out or the client sends an explicit request to abandon the session. Also, since each proxy instance initiates its own session, a user that makes a few requests at once could potentially be using multiple sessions at once.

So, if possible it would very useful understanding our client implementations to check if they are closing and disposing their proxies.

Another configuration that could also be taken in consideration here would be listenBacklog.

It’s important  to understand that when you create a server-side socket and start listening on it you need to specify how many pending connections you’re willing to buffer.

There is also a very important part of this discussion that could impact the capability of our WCF Service in servicing our clients, and that would be the configurations related with the timeouts.

 

OpenTimeOut & CloseTimeOut

This configuration is pretty much straightforward, it defines how much time it waits for the communication channel to be opened/closed.

This time would include the time necessary to all the necessary security handshakes, protocol negotiations, etc.

On the server side it will also include the time needed for the connection setup/teardown of the WCF Callbacks.

Important note: If a connection endpoint cannot be established at all (i.e. the service is not reachable), the WCF Service does not throws a Timeout Exception, instead, it throws a EndpointNotFoundException immediately.

 

SendTimeOut & ReceiveTimeOut

Send Timeout:  

Client-side call timeout, default 1 minute. Each call must be completed within the specified time, or they will trigger TimeoutException.

In TCP communications, the two sides to establish a connection, the client if you need to send a message to the server, it sends packets to a server, the server must be given a receipt to respond, the client can be declared after receiving the response sent successfully. Similarly, the server may also take the initiative to send a message to the client. These two actions, respectively , for the client and server are sending (Send), while for the other terms are to receive (Receive).

However, in the WCF field SendTimeout in the Send and the TCP-Send concept is similar, but totally not the same thing in essence. SendTimeout the Send is a client initiated call.

In the WCF client, you can take to SendTimeout with proxy.InnerChannel.OperationTimeout value. Here, proxy is automatically generated service proxy class, a XxxClient example.

ReceiveTimeout

And SendTimeout Similarly, ReceiveTimeout the Receive is initiated by the server-side callback (Callback).

Binding only apply when two-way communication, such as WSDualHttpBinding, NetTcpBinding, NetNamedPipeBinding, ...

ReceiveTimeout is a server callback (Callback) timeout, default 10 minutes. Each callback must be completed within the specified time, otherwise the server side trigger TimeoutException.

Hope that this information helps.

 

Some references

Optimizing WCF Web Service Performance

https://msdn.microsoft.com/en-us/library/ee377061(v=bts.10).aspx

There you can get more guidance on the above topics:

  • Implement serviceThrottling behavior for backend WCF services
  • Increase the default values for the NetTcpBinding.ListenBacklog and NetTcpBinding.MaxConnections properties in the backend WCF service web.config file
  • Eliminate ASP.NET httpModules that are not required to run WCF Web services
  • Use the WCF Module/Handler Registration tool to configure WCF modules/handlers and improve scalability of IIS 7.5/7.0 hosted WCF services