Uploading large file to IIS 7.5 or 8 using file input element

  • Article

I've recently worked on a very interesting file upload issue where my customer was hitting a 2 GB upload limit using Internet Explorer, IIS 7.5 and a simple file upload form with a file input element.
After doing some research and a couple of tests, I was able to build a simple "POC" project showing how to upload up to 4 GB using the following configuration:

  • Internet Explorer 10 client

  • Windows 8 / IIS 8

  • Application configured to run in .Net 4.5 Classic Pipeline

  • Web.Config configured as follows :

     

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.web>
<httpRuntime maxRequestLength="2147483647" />
<httpModules>
<add name="UploadModule" type="UploadModule,UploadModule"/>
</httpModules>
</system.web>
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="4294967295"/>
</requestFiltering>
</security>
</system.webServer>
</configuration>

 

  • simple upload.aspx test page :

     

<!DOCTYPE html>
<html lang="en" xmlns="https://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8" />
<title>Upload Test</title>
</head>
<body>
<form name="frm" action="upload.aspx" enctype="multipart/form-data" method="POST">
<h1>Choose file and click upload</h1>
<input type="file" id="SourceFile_1" name="SourceFile_1" size="40" />
<br />
<div>
<input type="submit" value="Upload" />
</div>
</form>
</body>
</html>

[private void Application_BeginRequest(object theSender, EventArgs theE)]

    {
HttpApplication httpApp = theSender as HttpApplication;
HttpContext context = ((HttpApplication)theSender).Context;
IServiceProvider provider = (IServiceProvider)context;
HttpWorkerRequest httpWorkerReq = (HttpWorkerRequest)provider.GetService(typeof(HttpWorkerRequest));
long receivedBytes=0;
long initialBytes=0;
byte[] buffer = new byte[10 * 1024 * 1024];

        if (httpApp.Request.HttpMethod == "POST")
{
// get the total body length
UInt32 requestLength = (UInt32) httpWorkerReq.GetTotalEntityBodyLength();
// Get the initial bytes loaded
if (httpWorkerReq.GetPreloadedEntityBody() != null)
receivedBytes = httpWorkerReq.GetPreloadedEntityBody().Length;
if (!httpWorkerReq.IsEntireEntityBodyIsPreloaded())
{
// Set the received bytes to initial bytes before start reading
do
{
// Read another set of bytes
initialBytes = httpWorkerReq.ReadEntityBody(buffer, buffer.Length);
// Update the received bytes
receivedBytes += initialBytes;
System.Diagnostics.Trace.WriteLine("#bytes read: " + receivedBytes.ToString());
}
while (initialBytes > 0);
}
System.Diagnostics.Trace.WriteLine("Request Length=" + requestLength.ToString() + " Total bytes read=" + receivedBytes.ToString());
}
}

 

If you use the above settings/pages and upload a large file (nearly 3GB in this example), you should see the following in DebugView :

 

I believe the 4 GB upload barrier using input type=file element is impossible to exceed for the following reasons :

  • there is a 4 GB upload limit in Internet Explorer: https://blogs.msdn.com/b/ieinternals/archive/2011/03/10/wininet-internet-explorer-file-download-and-upload-maximum-size-limits.aspx
    If you try to upload more than 4 GB with IE10 and above sample, IE will simply refuse to upload anything (you won't even see a POST request being sent!)

  • requestFiltering doesn't allow to specify more than 4 GB for maxAllowedContentLength

  • maxRequestLength is expressed in kilobytes and the limit specified above is nearly 2 TB. ASP.NET 2.0 doesn't allow a value greater than 2097151 KB (approx. 2 GB) and trying to set a greater value will fail with the following error :
    "The value for the property 'maxRequestLength' is not valid. The error is: The value must be inside the range 0-2097151"

  • If the application is running under the NET 4.5 Integrated Pipeline, upload will not work above 2G and the following error will be sent by IIS: "HTTP 400.0 – Bad Request ASP.NET detected invalid characters in the URL.".

    Debugging of the error a little bit further shows that its cause is the following stack and exception:

     

    0:034> !clrstack

    0000003a5f72e048 000007fb7e99811c [HelperMethodFrame: 0000003a5f72e048] System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32, IntPtr)
    0000003a5f72e130 000007fb58087ab1 System.Web.Hosting.IIS7WorkerRequest.ReadRequestBasics()
    0000003a5f72e1d0 000007fb5806ee45 System.Web.Hosting.PipelineRuntime.InitializeRequestContext(IntPtr, Int32, System.Web.
    Hosting.IIS7WorkerRequest ByRef, System.Web.HttpContext ByRef)
    0000003a5f72e240 000007fb5806e45f System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
    0000003a5f72e3d0 000007fb5806e2e2 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr,Int32)
    0000003a5f72e420 000007fb587cb781 DomainNeutralILStubClass.IL_STUB_ReversePInvoke(Int64, Int64, Int64, Int32)
    0000003a5f72e668 000007fb66959863 [ContextTransitionFrame: 0000003a5f72e668]

    0:034> !dso

    0000003A5F72DEC0 0000003893849348 System.ArithmeticException

    When the integrated pipeline is used, we go though webengine code (webengine4!MgdGetRequestBasics) which doesn't support more than 2 GB content-length and a System.ArithmeticException exception is raised which subsequently cause the HTTP 400 error. With the classic pipeline, we don't use webengine4 but the old ASPNET_ISAPI model and we don't hit the above issue.

If you need to upload more than 4 GB (or 2 GB in case ASP.NET 4.5 integrated pipeline or ASP.NET 2.0), I believe you'll need to use specific client and server code in order to use chunked-encoding and read data using GetBufferlessInputStream.

To perform the upload tests, I've used a quite recent PC (4 processor machine with 16 GB of RAM, 1 GB/s Lan, SSD storage, all the tests were made locally). With such configuration, the upload of 3 GB takes less than 30 seconds. Beyond memory/hardware requirements and impacts (upload of very large file clearly puts a lot of "pressure" on the server machine), my test scenario was not really realistic. In a real world scenario, I should have probably to tweak some settings at various levels (IE, http.sys, IIS, asp.net, etc…etc). Therefore, even if it is technically possible to upload up to 4 GB using above scenario, you may want to consider other means to build a more "robust" upload (using range request for example or using other protocols (FTP, WEBDAV…etc)).

Happy Uploading!

Emmanuel Boersma