September TFS DevDiv Dogfood Statistics

Brian Harry

Dogfood activity continues to be brisk but growth has slowed some in the last month.  I think the ramp from the onboarding of so many Orcas feature crews is behind us and the growth will be slower until we bring on more PUs. Overall performance and availability have been good but we continue to struggle with some issues.  I wanted to let you know about some changes that are coming in Orcas… Version control – We’re nearing completion of our Version Control Orcas “Perf & Scale” feature crew.  Early results are in and they look very promising.  There are three sets of work here:

• More efficient path handling – We found a better way to organize the indexes and working folder mappings to allow SQLServer to generate better query plans. Testing this with a fairly “typical” DevDiv enlistment, the time to do a “tf get” when relatively few files have changed has dropped by more than a factor of 2.  Some operations (like get of a single folder) have improved by as much as 50X.  By and large we should see all version control operations get faster and the worse they are today, the more “better” they should get.
• Streaming of result sets – Today we have a problem that the mid tier will run out of memory if anyone runs an operation that returns hundreds of thousands of rows. We’ve worked around this today by making our tools break up large operations into smaller chunks but it’s not fool proof and creates problems of its own.  In this feature crew we’ve added support to the mid-tier to stream results to the client as we get them.  As a result the most common “large” operations on the server never use a substantial amount of memory regardless of how big the operation is.  We are also working to make the client work the same way.  To help understand the urgency, we are currently measuring when the client “hits the memory wall”.
• Working folder mapping improvements – A few limitations on workspace mappings have created a problem for internal use. They’ve caused workspace definitions to become unwieldy and have detrimentally affected performance.  In this feature crew, we implemented 2 improvements – “one level mappings” and “maps under cloaks”.  One level mappings allow you to map the contents of a folder without mapping any child folders, “maps under cloaks” allow you to cloak a folder and then uncloak one or more folders somewhere under it.  Between these two, our workspace definitions should get much simpler.

Work item tracking – We’ve been working on the problem that people occasionally experience long (30-60 second) freezes.  It looks like this comes from view rebuilding that happens whenever an admin change is made that can affect people’s security settings.  It is the result of the number of work item fields we have defined and the number of distinct users that use our system.  We have a work item tracking feature crew going and are hopeful that we can reduce these pauses to sub-second. These issues are unlikely to have major effects on all but the very largest teams with very large code bases.  Nonetheless, everyone will see some performance improvements and I figured you might want to hear about some of the work we are doing for Orcas and the impact it will have. Lest you think, this is all we are doing for Orcas – not at all.  We have a bunch more stuff in the oven that will be applicable to a wider audience and I’ll be talking about that in the coming months. Here’s the latest dogfood stats…  The numbers that stand out to me here are – we passed 50 million files and 200 million local version rows.   Users

• Recent users: 745 (down 30)
• Users with assigned work items: 2,219 (up 80)
• Version control users: 1,589 (up 60)

Work items

• Work items: 121,793 (up 6,500)
• Areas & Iterations: 6,517 (up 150)
• Work item versions: 965,781 (up 50,000)
• Attached files: 38,095 (up 2,000)
• Queries: 11,469 (up 400)

Version control

• Files/Folders: 55,390,284/11,158,809 (up 7M/2M)
• LocalVersion: 201M (up 25M)
• Total compressed file sizes: 295.7G (up 22G)
• Workspaces: 2,676 (up 150)
• Shelvesets: 4,022 (up 600)
• Checkins: 124,488 (up 8,000)
• Pending changes: 669,544 (down 150,000)

Commands (last 7 days)

• Work Item queries: 741,584 (down 250,000)
• Work Item updates: 82,064 (up 65,000)
• Work Item opens: 275,870 (up 13,000)
• Gets: 34,959 (down 210,000)
• Downloads: 14.6M (up 1M)
• Checkins: 3,126 (up 15)
• Uploads: 105,607 (down 113,000)
• Shelves: 523 (up 125)

As always, thanks for listening…

Brian

Brian Harry Corporate Vice President, Cloud Developer Services

Follow