TFS Proxy - Unsung Hero

  • Article

1. Overview

Source files download operations could be pretty resource expensive operations and could impact overall TFS instance performance. A proper implementation of TFS proxy (for source code download operations) can help to increased TFS instance performance also reduce total time to complete the download operations. TFS proxy can also help to reduce total operational cost, increased TFS user’s productivity by decreasing total download operation time.

Our team provides support on software development tools to multiple groups within company. Some of the TFS implementations are the largest implementation of on-premises Team foundation servers.

Total databases sizes on all the instances is 110+ TB.

Some of the limitations which can severely impact TFS server performance:

  1. Large number/size files downloads from TFS server
  2. Location of TFS server, TFS proxy server
  3. Frequent cleanup of TFS proxy server content
  4. Users/automations not using correct proxy server name

Primary benefits of TFS proxy:

  1. Load on the main server is reduced; this is really the main benefit and the reasoning behind the recommendation for everyone to use proxies whether remote or not
  2. Performance is increased for users in remote sites, particularly for files that are retrieved multiple times or by more than one person

Introductory material on this feature can be found here.

Below proposed architectures provides details around possible solutions to make sure that TFS proxies are used and also is maintainable without impacting end users.

2. Proxy server hardware Recommendations

  1. At-least one server in each region (like US, Europe, Singapore, India, China)
  2. Standard web server/file server SKU
  3. Data caching drive should be at-least 20-30% of total content, this will reduce possibility of frequent cleanup.
  4. You will need about 2 IOPS per file, per second - so, say 1K downloads per sec requires 2KIOPS - 20 spindles or a couple SSDs.

3. Proxy Architecture

During the proxy setup evaluation phase we have considered following options;we have also done a proof of concept (POC) for all 4 options and based on the POC have documented advantages/disadvantages of each option to make an informed decision.

3.1. (Option#1) Proxy Setup Via GTM

Global Traffic Manager (GTM) is one of the standard ways to redirect traffic based on the client location.

This architecture gives us advantage of being able to redirect the traffic based on the client and in the case of one server failure, the client can perform the Get operation from the nearest available other proxy.

This architecture has other advantages, such as manageability and reliability. Recognizing that our team manages 750+ team project collections, when the proxy server name changes we do not have to make changes on all the team project collection, we just need to make changes on the virtual IP (VIP) level without affecting user’s configuration/settings. 

3.1.1. Architecture

 

3.1.2. Configuration on team project collection

This configuration is one time task.

To register a proxy for a specific team project collection execute the following command

  1. You must have TFS administrator permission to setup proxies on project collection.
  2. Open command prompt
  3. Change default directory to TF.exe location directory
  4. Execute below command

tf proxy /add <<ProxyGTMURL>> /default:global /collection:<<Collection URL>>

Note: Replace <<>> text with appropriate values.

tf proxy command & Permission details are available here 

3.1.3. User configuration

  1. Open command prompt
  2. Change default directory to TF.exe location directory
  3. Execute below command

tf proxy /configure /Collection:<<TeamProjectCollectionName>> 

3.1.4. Advantages 

  1. End user & machines never have to update the proxy configure
  2. If the proxy servers (Name or physical server) changes there is no impact on the end-user
  3. If TFS admin decided to add new servers in the current architecture, we do not have to configure servers URL on 750+ TPC databases
  4. Manageability of the proxies’ setup is easy. 

3.1.5. Limitations/Disadvantages

  1. Virtual IP (VIP) & GTM options may not be available in all geographical locations..

3.2. (Option#2) Proxy Setup Via AD site name

Proxy configuration via AD-Site name is another supported way to configure proxy for users in various locations.

The advantage of this solution is that users don’t have to explicitly configure a proxy for each user which increases the possibility of proxy usage more than in the current state. Since the majority of the Get operation will be served by the proxy it will reduce the load on the TFS instance and help to improve overall system performance. 

3.2.1. Architecture

  

3.2.2. Configuration on team project collection

To register a proxy for a specific team project collection execute the following command

  1. You must have TFS administrator permission to setup proxies on project collection.
  2. Open command prompt
  3. Change default directory to TF.exe location directory
  4. Execute below command

tf proxy /add <<ProxyURL>> /default:site /site:<<AD-SITE-NAME>> /collection:<<Collection URL>>

Note: Replace <<>> text with appropriate values.

Note: If you have multiple team project collection or site names you can script out the implementation to run the above mentioned commands for all team project collection.

3.2.3. User configuration

  1. Open command prompt
  2. Change default directory to TF.exe location directory
  3. Execute below command

tf proxy /configure /Collection:<<TeamProjectCollectionName>> 

3.2.4. Advantages 

  1. End user & machines never have to update the proxy configure
  2. If the proxy server changes (rename or new) there is no impact on the end-user
  3. If TFS admin decided to add new servers in the current architecture, we do not have to configure servers URL on 750+ TPC databases
  4. Manageability of the proxies’ setup is easy. 

3.2.5. Limitations/Disadvantages

  1. In the case of a proxy server name change the change needs to be executed on 750+ TPC databases
  2. For the best use of the proxy functionality TFS admin should configure all possible site names from all users’ (across the world).
  3. Maintainability cost of this option is very high as the accuracy of the information is user dependent and Admins may not have visibility of new users/team additions on different locations.

3.3. (Option#3) Hybrid setup with GTM & AD site name

In the hybrid solution we are going to merge best of option #1 & Option #2.

For the locations where GTM is supported (Like Redmond, Dublin, Singapore) we will add the proxy name as GTM URL with AD-Site name.

For the locations where GTM/VIP is not available we will use the server friendly name (alias name for server, can be created via DNS) and will add the TFS proxy using friendly name. In future if the server need to be replaced we just have to change the pointer (with new server IP) to replace the server. There will not be any configuration change for users.

3.3.1. Architecture

  

3.3.2. Configuration on team project collection

To register a proxy for a specific team project collection execute the following command

  1. You must have TFS administrator permission to setup proxies on project collection.
  2. Open command prompt
  3. Change default directory to TF.exe location directory
  4. Execute below command

tf proxy /add <<ProxyURL>> /default:site /site:<<AD-SITE-NAME>> /collection:<<Collection URL>>

Note: Replace <<>> text with appropriate values.

3.3.3. User configuration 

  1. Open command prompt
  2. Change default directory to TF.exe location directory
  3. Execute below command

tf proxy /configure /Collection:<<TPCURL>> 

3.3.4. How the setup works on client side

  1. TFS Client calls AD to get his site name.
  2. Client gets response from AD "Your site name is SITE-NAME."
  3. Client makes a call to TFS to check the configured proxy for SITE-NAME
  4. Client gets response from TFS with proxy URL for SITE-NAME
  5. Client usages the proxy URL for file download operations. 

3.3.5. Advantages 

  1. End user & machines never have to update the proxy configure
  2. If the proxy server changes (rename or new) there is no impact on the end-user
  3. If TFS admin decided to add new servers in the current architecture, we do not have to configure servers URL on 750+ TPC databases
  4. Manageability of the proxies’ setup is easy.

3.3.6. Limitations/Disadvantages

  1. For the best use of the proxy functionality TFS admin have to configure all team project collections for all possible site names.

 

Special thanks to Rémi Lemarchand and Philip Kelley for guidance and contribution.

Disclaimer: Above information is provided based on proxy implementation done in my team, Users/TFS admins should test the scenarios for best possible options.