Throttling, DSCP, and 802.1p with Traffic Control

In his introductory post about the legacy Traffic Control (TC) API, Gabe discussed the host-based model that TC provides. In this post, we will see how Traffic Control APIs can be used to achieve the following for TCP/IPv4 and UDP/IPv4 traffic sent from a host:

  • Throttle (rate-limit) outgoing traffic
  • Add DSCP value in layer-3 (IPv4) header
  • Indicate that an 802.1p tag value should be added in layer-2 header

At the bottom of the post, we’ve provided a link to the Networking Connect site to download full source and binaries to a tool which implements all of the above functionality.

The following are the steps involved:

1. The first step in the process is to obtain a handle to the Traffic Control subsystem through a call to TcRegisterClient() .

2. Next, make a call to TcEnumerateInterfaces() using the registration handle obtained in step #1. This call returns a list of all TC enabled interfaces on the system. Iterate through the list to find the interface(s) on which you want to prioritize and/or throttle outgoing traffic.

3. For each interface of interest, issue a TcOpenInterface() using the pInterfaceName from the corresponding TC_IFC_DESCRIPTOR for that interface from the list returned in step#2. Store the handle returned by TcOpenInterface() ; let’s call it the ifchandle.

4. At this point, create a TC Flow and add it to the interface(s) of interest.

a. Create the TC Flow:
A TC flow is a way of describing various QoS characteristics to be applied to a set of packets and is represented by a TC_GEN_FLOW structure. The following code snippet shows how to create a TC Flow given the DSCP value, 802.1p value and the throttle rate.

BOOL CreateFlow(PTC_GEN_FLOW * _ppTcFlowObj, USHORT DSCPValue, USHORT OnePValue, ULONG ThrottleRate)
{
BOOL status = FALSE;

  //
// Flow Parameters
//
ULONG TokenRate = QOS_NOT_SPECIFIED;
ULONG TokenBucketSize = QOS_NOT_SPECIFIED;
ULONG PeakBandwidth = QOS_NOT_SPECIFIED;
ULONG Latency = QOS_NOT_SPECIFIED;
ULONG DelayVariation = QOS_NOT_SPECIFIED;
SERVICETYPE ServiceType = SERVICETYPE_BESTEFFORT;
ULONG MaxSduSize=QOS_NOT_SPECIFIED;
ULONG MinimumPolicedSize=QOS_NOT_SPECIFIED;
PVOID pCurrentObject;
PTC_GEN_FLOW _pTcFlowObj = NULL;
int Length = 0;

  //
// Calculate the memory size required for the optional TC objects
//

  Length += (OnePValue == NOT_SPECIFIED ? 0:sizeof(QOS_TRAFFIC_CLASS)) + (DSCPValue == NOT_SPECIFIED ? 0:sizeof(QOS_DS_CLASS));

  //
// Print the Flow parameters
//

  printf("Flow Parameters:\n");
DSCPValue == NOT_SPECIFIED ? printf("\tDSCP: *\n"):printf("\tDSCP: %u\n", DSCPValue);
OnePValue == NOT_SPECIFIED ? printf("\t802.1p: *\n"):printf("\t802.1p: %u\n", OnePValue);
ThrottleRate == QOS_NOT_SPECIFIED ? printf("\tThrottleRate: *\n"):printf("\tThrottleRate: %u\n", ThrottleRate);
TokenRate = TokenBucketSize = ThrottleRate;

  //
// Allocate the flow descriptor
//
_pTcFlowObj = (PTC_GEN_FLOW)malloc(FIELD_OFFSET(TC_GEN_FLOW, TcObjects) + Length);

  if (!_pTcFlowObj)
{
printf("Flow Allocation Failed\n");
goto Exit;
}

  _pTcFlowObj->SendingFlowspec.TokenRate = TokenRate;
_pTcFlowObj->SendingFlowspec.TokenBucketSize = TokenBucketSize;
_pTcFlowObj->SendingFlowspec.PeakBandwidth = PeakBandwidth;
_pTcFlowObj->SendingFlowspec.Latency = Latency;
_pTcFlowObj->SendingFlowspec.DelayVariation = DelayVariation;
_pTcFlowObj->SendingFlowspec.ServiceType = ServiceType;
_pTcFlowObj->SendingFlowspec.MaxSduSize = MaxSduSize;
_pTcFlowObj->SendingFlowspec.MinimumPolicedSize = MinimumPolicedSize;

  //
// Currently TC only supports QoS on the send path
// ReceivingFlowSpec is legacy and ignored
//

  memcpy(&(_pTcFlowObj->ReceivingFlowspec), &(_pTcFlowObj->SendingFlowspec), sizeof(_pTcFlowObj->ReceivingFlowspec));
_pTcFlowObj->TcObjectsLength = Length;

  //
// Add any requested objects
//
pCurrentObject = (PVOID)_pTcFlowObj->TcObjects;
if(OnePValue != NOT_SPECIFIED)
{
QOS_TRAFFIC_CLASS *pTClassObject = (QOS_TRAFFIC_CLASS*)pCurrentObject;
pTClassObject->ObjectHdr.ObjectType = QOS_OBJECT_TRAFFIC_CLASS;
pTClassObject->ObjectHdr.ObjectLength = sizeof(QOS_TRAFFIC_CLASS);
pTClassObject->TrafficClass = OnePValue; //802.1p tag to be used
pCurrentObject = (PVOID)(pTClassObject + 1);
}

  if(DSCPValue != NOT_SPECIFIED)
{
QOS_DS_CLASS *pDSClassObject = (QOS_DS_CLASS*)pCurrentObject;
pDSClassObject->ObjectHdr.ObjectType = QOS_OBJECT_DS_CLASS;
pDSClassObject->ObjectHdr.ObjectLength = sizeof(QOS_DS_CLASS);
pDSClassObject->DSField = DSCPValue; //Services Type
}

  DeleteFlow(_ppTcFlowObj);
*_ppTcFlowObj = _pTcFlowObj;
status = TRUE;
Exit:

  if(!status)
{
printf("Flow Creation Failed\n");
DeleteFlow(&_pTcFlowObj);
}
else
printf("Flow Creation Succeeded\n");

  return status;
}

b. Add the Flow on the interface:
After obtaining a TC_GEN_FLOW structure with the desired characteristics using a function similar to the one above, issue a call to TcAddFlow() with the ifchandle (obtained in step #3) and a pointer to the TC_GEN_FLOW object (obtained in step #4a). Store the handle returned by TcAddFlow() ;let’s call it the flowhandle.

5. The next step is to create a TC Filter and add it to the TC Flow created above.

a. Create the TC Filter:
A TC Filter is a way of describing which packets to apply the QoS characteristics to. The QoS characteristics defined in the TC_GEN_FLOW will only apply to the packets matching the filter(s) associated with the Flow.
The following code snippet describes how to create a TC Filter given the destination address, the destination port, and the protocol (TCP,UDP or IP).

BOOL CreateFilter(PTC_GEN_FILTER * ppFilter, SOCKADDR_STORAGE Address, USHORT Port, UCHAR ProtocolId)
{

  BOOL status = FALSE;
USHORT AddressFamily = Address.ss_family;
PTC_GEN_FILTER pFilter = NULL;
PIP_PATTERN pPattern = NULL;
PIP_PATTERN pMask = NULL;

  if(AddressFamily != AF_INET)
goto Exit;

  //
// Allocate memory for the filter
//
pFilter = (PTC_GEN_FILTER)malloc(sizeof (TC_GEN_FILTER));
if(!pFilter)
goto Exit;

  ZeroMemory(pFilter, sizeof(TC_GEN_FILTER));

  //
// Allocate memory for the pattern and mask
//
pPattern = (PIP_PATTERN)malloc( sizeof(IP_PATTERN));
pMask = (PIP_PATTERN)malloc( sizeof(IP_PATTERN));
if(!pPattern || !pMask)
goto Exit;

  memset ( pPattern, 0, sizeof(IP_PATTERN) );
pPattern->DstAddr = ((SOCKADDR_IN *)&Address)->sin_addr.s_addr;
pPattern->tcDstPort = htons(Port);
pPattern->ProtocolId = ProtocolId;
memset ( pMask, (ULONG) -1, sizeof(IP_PATTERN) );

  //
// Set the source address and port to wildcard
// 0 -> wildcard, 0xFF-> exact match
//
pMask->SrcAddr = 0;
pMask->tcSrcPort = 0;

  //
// if the user specified 0 for dest port, dest address or protocol
// set the appropriate mask as wildcard
// 0 -> wildcard, 0xFF-> exact match
//
if(pPattern->tcDstPort == 0)
pMask->tcDstPort = 0;

  if(pPattern->ProtocolId == 0)
pMask->ProtocolId = 0;

  if(pPattern->DstAddr == 0)
pMask->DstAddr = 0;

  pFilter->AddressType = NDIS_PROTOCOL_ID_TCP_IP;
pFilter->PatternSize = sizeof(IP_PATTERN);
pFilter->Pattern = pPattern;
pFilter->Mask = pMask;

  //
// Delete any previous instances of the Filter
//
DeleteFilter(ppFilter);
*ppFilter = pFilter;
status = TRUE;

  Exit:
if(!status)
{
printf("Filter Creation Failed\n");
DeleteFilter(&pFilter);
}
else
printf("Filter Creation Succeeded\n");

  return status;
}

b. Adding the Filter to the TC Flow:
Once a TC Filter structure is obtained using a function similar to the one above, issue a call to TcAddFilter() passing the flowhandle obtained in step #4b and a pointer to the TC_GEN_FILTER structure obtained in step #5a. Store the filter handle returned by TcAddFilter() ;let’s call it filterhandle.
You can add multiple filters on the same flow causing different sets of packets matching each filter to get the same QoS characteristics applied to them.

6. At this point, your application is applying QoS on all matching outgoing packets as specified in the TC Filter and TC Flow. Finally, once your purpose is served, make sure you call the respective close calls on all the open TC handles – TcDeletefilter() , TcDeleteFlow() , TcCloseInterface() and TcDeregisterClient() .

You can download the full source and binaries of a simple command line tool – tcmonlite, which takes the filter and flow parameters as input, creates TC Flow and Filter, and configures the QoS subsystem with them using the Traffic Control API. All the outgoing traffic on the system matching the filter gets the desired QoS characteristics as long as tcmonlite is running. Go to the Microsoft Connect website, choose Available Connections on the left-hand side of the page, and select Windows Networking from the available connections (bottom half of the page). On the left-hand side of the Windows Networking page, choose Downloads, and select TCMonLite.

This tool can be used in conjunction with the NDIS LWF driver to detect 802.1p tags in the Ethernet header and DSCP in the IP header of packets. Let us know what you think!

-- Hemant Banavar