Troubleshooting a 404.13 issue using IIS and Failed Request Tracing logs

  • Article

Consider the following scenario:

You are running an Exchange 2010 in your corporate network and you would like to have your clients running in external network (say, from home) access their e-mails in Outlook (using Outlook Anywhere).

Outlook --> Reverse Proxy --> Exchange 2010 CAS Servers

Clients will also be able to access their e-mails by using Outlook Web Access (OWA).

To achive this you can run a reverse proxy capable application such as TMG and publish your Exchange CAS Server to the internet for Outlook Anywhere (OA) and OWA roles. As an alternative you can use IIS 7.0 or later and use Application Request Routing as reverse proxy.

You have already tried TMG and know that your clients are able to access their e-mails from their Outlook clients (using Outlook Anywhere in the background) or using web based OWA.

However, you decided to use IIS 7.5 and Application Request Routing as reverse proxy and you set up the environment using the instructions from the following article:

Reverse Proxy with URL Rewrite v2 and Application Request Routing
https://www.iis.net/learn/extensions/url-rewrite-module/reverse-proxy-with-url-rewrite-v2-and-application-request-routing

When you test, however, you see that the clients can use OWA successfully but they cannot send or receive e-mails using Outlook. When you troubleshoot the issue you see the following error in the client side:

An RPC error was thrown by the RPC Runtime process. Call Cancelled Error.

As Outlook Anywhere uses RPC over HTTPS to connect Exchange CAS Servers, this error message should be coming from RPC process and it is quite a generic error message saying that there was an error in connection to the Exchange server.

As OWA works there should not be a name resolution issue in either client or server side. As Outlook Anywhere was working when TMG is used as reverse proxy, there should not be a configuration issue on Exchange side or the networking or firewall related problems between IIS and the Exchange CAS Servers.

Instead there should be an issue on IIS side either with some IIS or Application Request Routing related configuration.

Troubleshooting the issue:

I assume you understood that why we decided to troubleshoot the IIS side. If you are not sure please read the scenario above again.

When you need to troubleshoot an issue on an IIS related issue, it is always best to check IIS logs first to see the big picture and this scenario is not an exception to that. When we check the IIS logs we see 404.13.

404.13 means content length too large.

The HTTP status codes in IIS 7.0 and in IIS 7.5 Knowledge Base Article at https://support.microsoft.com/kb/943891 explains it:

404.13 - Content length too large. The request contains a Content-Length header. The value of the Content-Length header is larger than the limit that is allowed for the server.

The same article also points to another article which covers a similar issue:

Error message when you visit a Web site that is hosted on a server that is running Internet Information Services 7.0: "HTTP Error 404.13 - CONTENT_LENGTH_TOO_LARGE"
https://support.microsoft.com/kb/942074

That mentions about a similar issue and explains the reason in Cause section:

This problem occurs because the client request contains a Content-Length header that is larger than the value that is specified for this header in the maxAllowedContentLength property in the ApplicationHost.config file.

It also explains the solution in Resolution section advising increasing the maxAllowedContentLength property of requestLimits setting in the applicationHost.config file.

But we still need to know the value of the Content-Length header that the client (Outlook Anywhere) request contains so we can set the correct value in maxAllowedContentLength property.

You can get the Content-Length value of a request in a lot of ways. Some options are below:

  • We can collect a network trace on the client side or IIS and investigate the request headers. However, it may be difficult to run network trace tools on either side. Besides, if the traffic is made over HTTPS then it won’t help because the traffic would be encrypted and we couldn’t see the headers.
  • We can run Fiddler on IIS and set up server’s proxy to Fiddler’s proxy address so we can just get the HTTP and HTTPS traffic. Fiddler can decrypt the HTTPS traffic so we would have what we look for. However it may be a tricky job to run Fiddler as proxy in a critical server so you may not want to choose this option.

Fortunately, IIS has a great troubleshooting feature called Failed Request Tracing. I don’t explain what it is but I will give a brief summary about it from Official IIS site:

Troubleshooting Failed Requests Using Tracing in IIS 7
https://www.iis.net/learn/troubleshoot/using-failed-request-tracing/troubleshooting-failed-requests-using-tracing-in-iis

Request-based tracing provides a way to determine what exactly is happening with your requests and why, provided that you can reproduce the problem that you are experiencing. Problems like poor performance on some requests, or authentication-related failures on other requests, or the server 500 error from ASP or ASP.NET can often be difficult to troubleshoot--unless you have captured the trace of the problem when it occurs.

In most of the scenarios when using Failed Request Tracing you need to follow the steps below:

  • Enable Failed Request Tracing.
  • Set up a rule in Failed Request Tracing.
  • Reproduce the problem.
  • Disable Failed Request Tracing.

The same article contains an example usage of Failed Request Tracing which creates a rule for 404.2 HTTP Status Code and uses WWW Server provider and logs its Security related processes. Don’t get confused with these details at the moment but I suggest you to follow the example from that article to understand what those details mean.

Setting Up Failed Request Tracing Rule and analyzing the log:

In our scenario we only need to see Request headers and Content-Length propery so actually we don’t need to decide which providers we should enable, and we can keep the default setting which enables all the providers.

Note: In this scenario it wouldn’t hurt to collect more data than we need, but, in some cases it might mean a lot of unncessary data and you might be lost in a lot of unncessary information to examine or the size of the logs would be so large that you might have issues either collecting or analyzing the data.

To set up the rule, open the Failed Request Tracing feature and set 404.13 as status code when defining the trace condition. See a sample screenshot below:

After you create the rule you can reproduce the issue. To reprodcuce the issue we open Outlook in the client and try to receive or send e-mails. When the same issue happens and request fails because of content-length related issue, a 404.13 status code will be catched by our rule and a log will be created with the request details. When we open the log file in Internet Explorer we will see a lot of details about the request and the way it is processed in IIS. To find the request headers, I prefer switching the Compact View section in the page and search for GENERAL_REQUEST_HEADERS keyword. In our case the headers will be similar as seen below:

Headers="Cache-Control: no-cache
Connection: Keep-Alive
Pragma: no-cache
Content-Length: 1073741824
Accept: application/rpc
Authorization: ***********************
Host: ***********************
User-Agent: MSRPC
"

The one we are looking for is the Content-Length property and its value and it is 1073741824 bytes which is 1 GB exactly.

This sounds a very large request and I don’t know if it is how Outlook clients work or not but this is out of the scope of this blog post.

Solution

Please remember the article we found when we troubleshoot the issue:

Error message when you visit a Web site that is hosted on a server that is running Internet Information Services 7.0: "HTTP Error 404.13 - CONTENT_LENGTH_TOO_LARGE"
https://support.microsoft.com/kb/942074

The solution explained in that article is to increase the maxAllowedContentLength property of requestLimits setting in the applicationHost.config file. Obviously you can also use Request Filtering icon in IIS manager to manage its settings including the maxAllowedContentLength property.

The solution to our problem is, obviously, to set up a larger value for maxAllowedContentLength property:

Applies To:

IIS 7.0, IIS 7.5, IIS 8.0

--
AMB