Thoughts on IIS Memory Recycling for 3rd party Applications

Sigh… it seems that the Application Health Monitoring features added in IIS6 are merely used by VARs to cover up for their own mistakes instead of leaving it in the user’s control as a crutch to fall back on when 3rd party web applications fail.


We have a CRM application which runs on IIS6. The application has been crashing intermittently with the message “The server is temporary busy, please try again after a moment.”

The manufacturer of the program had us adjust the default IIS AppPool properties as follows

[x] Recycle Worker processes (in minutes): 1440
[x] Recycle worker processes (number of requests): 35000

[x] Maximum virtual memory (in megabytes): 600
[x] Maximum used memory (in megabytes): 500

Changing these settings has tremendously improved the situation but we are still occasionally getting “The server is temporary busy” message.

I would go back to the manufacturer to ask for additional guidance but I got the impression they were just changing settings without knowing exactly what they were doing.

So my question is that given the server has 2GB of RAM, can we tweak these values further to eliminate this problem. What settings are recommended and why?

Also would adding additional RAM to the server help? Any assistance or documentation anyone could point me to regarding these settings would be greatly appreciated.

Thank You,


Actually, if you want to eliminate this problem, you need to insist that your application manufacturer debug their CRM application to determine the cause of “the server is temporarily busy…” error. Only after determining the cause of the issue through debugging can one:

  • produce a fix for the bug 
  • determine a work-around for the bug


In other words, without knowing the cause of the issue, you have no idea whether tweaking any server parameters will help, much less eliminate, the issue. This means that it does not help to ask for “recommended values” because there are never any generally recommended values for an application – it all depends on the application requirements, status, resource footprint, etc – and since the bug is “unknown”, the recommendation is also “unknown”. The same goes for changing system hardware like adding more RAM – once again, without understanding the issue, making changes is like gambling – and why gamble when the application manufacturer is supposed to support you in getting their CRM application working?

And if your concern is that the application manufacturer has no idea what they are doing with tuning/debugging their own application… then I wonder how you can trust their CRM application at all. The same group of people supporting the CRM application probably also wrote the CRM application – and if they cannot get you proper support for their application, then how can you rely on them long term?

In general, if someone tells you that “to run my application, you have to tweak THESE Application Pool Health Monitoring metrics”, it means that their application is not written well enough to stay running.


Right now, what it sounds like is that either this CRM application has some bug inside of it, either inside the program logic or its related configuration, and this bug eventually leads to the “server is temporarily busy…” error. The vendor either has no idea about the bug or knows about it but does not want to fix it.

There are two general ways to “resolve” a bug:

  1. Find a work-around to avoid the bug
  2. Fix the actual bug

Right now, it sounds like the vendor does not want to do #2 to truly eliminate the problem, so they distract you by telling you to tweak IIS AppPool Properties in an attempt to do #1. The problem with #1 is that it does NOT eliminate the problem – the bug is still there in the application you are running – and without debugging the issue, you cannot identify the problem and has no idea whether setting changes actually work-around the problem.

But… the vendor managed to get you distracted and wishfully thinking that YOU are actually empowered to resolve THEIR bug by simply tweaking IIS settings, adding more RAM, or otherwise tuning your system – when you have no idea what issue is being worked around. Meanwhile, the bug is still there in the application you are running…

Does this make any sense?

For example, adding more RAM does not solve logical bugs in an application – at best, it delays the problem. Suppose the issue is a memory leak. By adding more RAM, it simply means it takes longer for the application to consume all of your server’s memory; it does not fix the leak such that the application never consumes more memory than it needs. Sure, you can recycle the process more frequently to periodically “free” up the memory, but recycling destroys in-process state and caches which have down-stream performance and reliability effects… all depends on the application’s architecture. You merely substitute one unknown problem for another and do not make progress towards eliminating problems.

Remember, AppPool Health Monitoring Metrics are just that – generic metrics to determine the health of the application, and if deemed unhealthy by those metrics, recycle the worker process to clear away stale application state and start afresh. It is like rebooting the application.

In other words, it is meant as a possible crutch to keep the application running while support personel debug the issue and provide a fix. It is not meant as the “solution” to buggy software because it simply consumes one of YOUR defensive trump cards against buggy applications.

In general, the best solution to buggy software is to identify the issue and get it fixed, and this is what I suggest you insist your application manufacturer perform. Here are some useful blog entries for this endeavor:


Comments (11)

  1. Robert Moir says:

    Sadly, I think people on both sides of the support fence who confuse the workaround to a known problem with the actual solution are all too common.

    It’s perfectly understandable in a way, but for a lot of people, if there isn’t an actual error dialogue box up on the screen then there is no problem at all, and making the dialogue box stop appearing is therefore seen as fixing the problem. And before you know it, a temporary tweak stays in place for a year, someone’s line of business database has no end of junk written to it because the underlying problem is still there, and the whole game is over.

  2. Jeff Parker says:

    Yeah I agree, sometimes I do wonder how some apps get built and on the market. We had a purchased app once the server admins came and got me to help diagnose. IIS was throwing a login prompt. If a user tried to log in. It wouldn’t work eventually fail. if an admin logged in it worked. The vendor was Change this setting, change that setting. When they got me involved. I simply asked what directory is trying to be accessed or written to. No one even the vendor didn’t know. So I fired up filemon from sysinternals and told them ok hit the page. File mon lit up with all kinds of security access violations. I looked and then shook my head. Aparently the vendor for some reason was writing out config changes  to the application in the C:Documents and SettingsAll UsersApplication Data directory . Now of course on a locked down server why on earth would you allow a web app to write something there was beyond me. Basically it was a problem they needed to fix and make it right for the rest of the world. They seems to think the account should be an admin of the server.  You find a lot of wacky things like that in vendor apps.

  3. David.Wang says:

    Jeff – hehe, it is nice to have experiences like this be known… because for too long, IIS (and indeed Microsoft as a whole) has favored "having something just work" instead of "having something work securely".

    Of course, I think that right now, we are tilting too much over to security and making things more difficult (for example of this, look at User Access Control [UAC] in Windows Vista – ugh!!!) and there will be another backlash…

    But hey, the world will get to be more safe after all this… or will it. Training users to blindly click "Yes" to any Windows dialog box because otherwise things may not work is NOT progress… except for the Phishers, maybe. 😉

    Yes, Computers are difficult to use and not simple to use like a Car, but Computers are a general purpose tool able to do lots of things; Cars just get you from point A to B (ok, maybe a couple of other more inauspicious uses… 😉 ).


  4. Jon Diamond says:

    One thing that I have recently been introduced to memory fragmentation caused by IIS response buffers… it could be a similar thing if the demand on the server is very high then due to the way .net allocates memory in chunks the IIS rsponse buffers can cause out of memory errors and possibly server unavailble messages… there is a Microsoft KB article on this somewhere…

    I guess what I’m saying is it’s probably an app bug but you never know.


  5. David.Wang says:

    Jon – You will have to provide that KB article… because:

    1. IIS does not buffer responses, so I do not know what you mean by "IIS response buffers". IIS buffers are all ACACHE’d to not cause memory fragmentation

    2. ASP.Net does buffer responses and can have such problems under high load, but .Net is NOT IIS.

    I am not saying that IIS or Microsoft software do not cause memory fragmentation – we are talking about software, after all. I am just saying that people give Microsoft a lot less credit than it deserves and lumps all 3rd party Application issues in with "Microsoft" because it all happens on Windows.


  6. Jon Diamond says:

    To be fair I’m a massive Microsoft supporter and I’d look at everything possible before blaming IIS or dotnet.

    This problem was something that as a dotnet developer I’d have blamed on the app 99 times out of 100… all I’m saying is it’s something to consider especially on heavily hit sites.

    The KB article is

    Basically the response buffers fragment the heap and the CLR can’t allocate another 64MB contigious block resulting in an OutOfMemoryException even with shed loads of free memory.


  7. Jon Diamond says:

    I take your point though that dotnet is not IIS but then neither is the worker process but that doesn’t stop there being a recycle worker process setting in IIS6 as discussed at the top of this thread 😉

  8. David.Wang says:

    Jon – Thanks for providing the KB article.

    Yup, this issue is with ASP.Net and not IIS.

    The problem is that ASP.Net ends up using memory in a way that fragments memory, and having that fragmented memory pinned on a native-call to IIS (such as sending a response over a slow connection) simply exacerbates the implementation flaw in ASP.Net. IIS is just the helpless victim here. 🙂


  9. Jon Diamond says:

    …. and maybe the third party app 😉

    Thanks for the discussion and your insight, I’ll be tuning into your blog more often.


  10. David.Wang says:

    Jon – hehe, ah, well, w3wp.exe is almost always the helpless victim on IIS6. 😉

    Just look at its work contract: It is told to run everything with NO control of what that actually is… and Windows always report "w3wp.exe" as the Faulting Process, not the code that is run. It is like the ultimate scapegoat.

    You should see our Watson OCA buckets when we debug those crash reports you send in for w3wp.exe…


  11. Santhosh says:

    Please install Service PACK 2 in server you problem of IIS memory will be eradicated.

    Try and let me know even we were facing same issues. Now it is sorted out.