Concurrent Channel Performance

  • Article

Being thread-safe is different than being concurrent. The channel interfaces are thread-safe so that multiple callers can use them at the same time without getting garbled messages. However, if multiple callers try to send messages on a single channel at the same time then a few different things might happen.

- The channel might be fully concurrent and process the sends in an overlapped fashion.

The channel might be partially concurrent and process a limited number of the sends at a time. The other sends are blocked until one of the earlier requests completes.
  • The channel might be singly concurrent and process the sends in a sequential fashion.

A connection-oriented channel like TCP tends to be singly concurrent. Interleaving messages requires having a sophisticated framing protocol and the performance cost is rarely worth handling this special case. Datagram channels on the other hand tend to be at least partially concurrent. For example, HTTP in theory is fully concurrent to any degree that you'd like but the HTTP specification recommends limiting clients to two connections to any particular server, resulting in partial concurrency.

Whenever you have less than full concurrency you run the risk of deadlock with certain application designs. That's because the logical operation order of the application might say that caller 1 must complete a send before caller 2 can complete, but the physical operation order of the channel says that caller 1 cannot start a send before caller 2 completes.

I've seen customers encounter this problem when using HTTP and callbacks. The original request is waiting for the callback operation to complete but the callback thread cannot make any progress because all of the connections are tied up by existing callers, such as the original request. This is a case where the performance gain of sharing connections has led to unsafe behavior. The solution is to increase the concurrency of the system so that at least one call from any of the ongoing chains of operations can complete (there can be callbacks on top of callbacks in complicated knots so the problem is not as simple to solve as increasing the concurrency factor to two). Concurrency can be increased directly at the networking level or by isolating networking resources into separate pools at the application level.

Next time: Windows and UPN Format Credentials