One way to Reduce the Size of IIS Log Files


Question:


Hi David


We’re seeing the usual trauma of massive log files as the result of increased traffic. We’ve recently crossed the line from where this was inconvenient to where it has become a serious problem. Our “quick and dirty” solution was to post-process our logs and strip out all logged requests for images. While this hack solution is getting us by for right now – I would prefer something more elegant, sustainable and scalable.


I have been trying to figure out if there is any way to simply not log *.gif, *.jgp, *.css and *.js. So far I cannot seem to figure this out.


Any help would be really appreciated!


Answer:


IIS supports fine-grained control of which resource access results in a log entry. You simply need to set the DontLog property at the corresponding URL-level, and IIS will ignore logging requests to that URL or virtual directory. See the following blog entry for related details.


However, there is no configuration syntax to direct IIS to simply never log access to *.gif, *.jpg, *.css, and *.js regardless of location… but that should not be a big deal for a well-organized website. Presumably, you have all your resources organized and categorized across your website for efficient resource management – for example, all images are available under a /images virtual directory, all stylesheets under a /stylesheet virtual directory, and all client-side Javascript under a /client-scripts directory.


If your website content is organized, then you can simply set “DontLog” for the /images, /stylesheets, and /client-scripts virtual directories, and none of those resources will get logged by IIS.


Otherwise, you have no choice but to set DontLog property as a IIsWebFile for every single scattered resource on your web server. While IIS supports direct configuration of DontLog on a granular per-URL scope via the logic of “If URL Resource has DontLog set to true, then do not log access to it”, you really want a meta-configuration-logic of “If the URL Extension is X, Y, Z, then set DontLog to true.”


Ideally, you want to execute your meta-configuration-logic via an add-on extensibility module to either the IIS Configuration System or IIS Core Web Server. However, in this case, decision to Log a request is static, so you cannot use IIS Core Web Server extensibility mechanisms like ISAPI Filter to dynamically add your logic. IIS Configuration System does not support this rare form of extensibility, either…which means your only alternative is to write a script that automatically (but statically) updates the DontLog properrty of URLs your website on your behalf.


//David

Comments (6)

  1. TKW says:

    Would your solution of not logging, for instance, /images allow an attacker to issue GET /images/*exploitcodehere* commands without being logged?

  2. yman says:

    What exploit code would that be ?

    It couldnt be server side script vuln since it wouldn’t be in the image code. Are you talking about a future possible IIS vuln ?

  3. David.Wang says:

    TKW – in general, "not logging" a request made to the server, no matter how it is done, opens a potential security issue when it comes to repudiation. The server performed operations for a request and it is not logged – how does one prove what really happened on that request?

    So, yes, my solution does carry the caveat that access to /images will not be logged, but presumably a well-organized website will also split static files like images, client-side script, CSS, etc from dynamic server-side cgi-bin stuff, and you can lock down the directories containing static files to not allow server-side execution, thus reducing your attack surface to only IIS static file handler.

    Besides, for the really security conscious, they can use IIsWebFile to control exactly WHICH files are not logged. Trying to filter by *.jpg, *.gif, *.css, and *.js is simply syntactic sugar.

    //David

  4. David.Wang says:

    yman – TKW is simply pointing out that by turning off logging for the entire /images/ namespace, that any exploits which use that namespace will not be logged.

    In other words, TKW is just noting the security ramifications of applying DontLog with a "wide brush".

    //David

  5. Maurits says:

    Log files tend to compress well. It may be worthwhile to script a task that runs compact.exe on individual log files after their time window has elapsed.

  6. manvendra says:

    david can u help me urgent

    i have various log file and want to extract particular

    url information

    how i do that

    please help me