IIS SEO Tip - Do not stress your server, limit the number of concurrent requests

  • Article

The other day somebody ask me if there was a way to limit the amount of work that Site Analysis in IIS SEO Toolkit would cause to the server. This is interesting for a couple of reasons,

  • You might want to reduce the load that Site Analysis cause to your server at any given time
  • You might have a Denial-of-service detection system such as our Dynamic IP Restrictions IIS module that will start failing requests based on number of requests in a certain amount of time
  • Or If you like me have to go through a Proxy and it has a configured limit of number of requests per minute you are allowed to issue

In Beta 1 we do not support the Crawl-delay directive in the Robots exclusion protocol; in future versions we will look at adding support this setting. The good news is that in Beta 1 we do have a configurable setting that can help you achieve this goals called Maximum Number of Concurrent Requests that you can configure.

To set it:

  1. Go to the Site Analysis Reports page
  2. Select the option "Edit Feature Settings..." as show in the next image
    EditFeatureSettings
  3. In the "Edit Feature Settings" dialog you will see the Maximum Number of Concurrent Requests option that you can set to any value from 1 to 16. The default value is 8 which means at any given time we will issue 8 requests to the server.
    MaxConcurrentRequests