In my opinion BLOBCache is a must have in most MOSS environments. It is known that BLOBCache and Web Garden are not the best of friends, and according to Optimizing Office SharePoint Server for WAN environments don´t use Web Garden if you have BLOBCache in your environment. The main reason for this is that only one process can acquire the lock to this cache and if the process serving the request is not the one who acquired the lock then you can imagine you won´t get the result you were expecting.
So far so good, but there is another setting that can also get you into trouble even if you don´t have a Web Garden scenario.
IIS 6.0 has a metabase property called DisallowOverlappingRotation and according to documentation (DisallowOverlappingRotation Metabase Property)
“The DisallowOverlappingRotation property specifies whether or not the World Wide Web Publishing Service (WWW Service) should start up another worker process to replace the existing worker process while it is shutting down."
The default value for this property is FALSE (see table below) which means that WWW Service is allowed to start a new worker process when the old one is shutting down.
XML Data Type
WMI Data Type
ADSI Data Type
ABO Data Type
ABO Metabase Identifier
Now you can imagine there is a frame in time where you have multi-instance because both processes still exist (this is like Web Garden but just for a short period of time). When this new process starts it will try to acquire the lock to the cache but what happens if the process that is shutting down hasn´t yet released this lock? The new process won´t be able to acquire and it won´t be able to use this cache to improve the performance of your website. You will notice a decrease in performance until you do an IISReset.
Also in documentation about this property we can read the following
“The value of this property should be set to true if the worker process loads any application code that does not support multi-instance.”
My advice is that if your environment makes use of BLOBCache also set IIS 6.0 property DisallowOverlappingRotation to TRUE.