Using huge files with SharePoint

  • Article

One of our customers in Microsoft Consulting Services (MCS) was curious about the change in performance characteristics of SharePoint Servers when used with huge files (around 100 MB). They were currently allowing up to 20 MB and before allowing more to the business users they wanted to test their current infrastructure (2 front-end servers, 1 back-end app server and 1 DB server). Since it is confidential I won't give details about the infrastructure but still the below information will give you an idea about using SharePoint with huge files.

Testing Methodology

I used the OOB load test that comes with the open source project "SharePoint 2007 Test Data Population Tool" (https://www.codeplex.com/sptdatapop) to first test the environment without the huge documet usage. And then in parallel I wrote another test that simply reads/writes 100 MB documents in SharePoint and monitored the performance counters on all of the servers. The second test used around %1 of the total users in the first test (and this was our customers guess about the huge file usage)

Findings

  • The average request/sec dropped around %18
  • The average response time was quite similar (since less pages were processed by our test rig)
  • CPU usage on all of the servers were quite similar except a slight increase in the front-end servers
  • Available memory was again quite similar (slight decreases in the front end servers)
  • SQL Server disk I/O (reads, writes and disk queue length) had some significant spikes and the averages had some significant (and for our scenario acceptable) differences
  • Network interfaces in the front-end servers had some significant difference

These results might not be the same in your environment and I strongly suggest performing similar tests before coming to a conclusion. In our case the current infrastructure was quite capable of handling huge files but if we wanted to further increase the performance I'd personally start looking at the SQL physical discs first.

Oh by the way please also consider your overall disk capacity before allowing huge files.. Since SharePoint does not work like simple network shares you need to be very careful on disk capacity planning.