TCP Throttling

As I mentioned on the 30th anniversary of IP, an early and fundamental split was made in TCP to distinguish point-to-point messaging from end-to-end messaging. The split is based on a philosophy that the communication endpoints should be in control and that there should be a minimal amount of functionality and responsibility distributed along the communication path in between.

The end-to-end philosophy shows up in how TCP systems perform throttling. Duplication of throttling makes the system more complicated. Duplication of throttling also makes the system less efficient because more throttling decisions have to be made by intermediaries that have less information about the system than the endpoints. TCP uses flow control to establish how fast data can be transmitted from the sender to the receiver. Rather than having the communication path adjust the flow rate, TCP uses congestion control algorithms to infer how the sender should change its transmission strategy to best use the network.

In the basic implementation of TCP, the data stream is overlaid by a transmission window that defines which packets can be sent. As packets are received and acknowledged, the transmission window slides along the data stream. The endpoints grow and shrink the size of the transmission window as needed to adjust the flow rate. Initially the transmission window is very small because the state of the communication path is unknown. However, the window size adapts itself over time to an optimal rate of transmission. Some of the changes to the TCP/IP stack in Windows Vista were to accelerate the rate of adaptation and reduce the amount of time spent using a sub-optimal transmission rate.

The end-to-end philosophy also shows up in how the WCF TCP channel avoids trying to do throttling. The service knows more about how it's going to use the network than the networking stack does. Consequently, if you want to control how quickly things are sent and received, the best place to control that is in the application. The only bottleneck in the TCP channel is how quickly it can pass information back and forth between the application and the wire. The default quota settings for the channel are sufficient for networks up to 100 Mbps. You'll want to increase the ConnectionBufferSize quota from its default of 8 KB to something like 64 KB on faster networks.

Note that if the throttling is desired for fairness between applications rather than within an application, then the best place to do the throttling is outside of the applications. Again, this follows the principle that the decision should be made where there's the most information.

Next time: Building on Custom Cookie Handling