W3SVC 1310 and who changed my worker process affinity


 


 


Randomly, IE will show follow error when customer accessing their web application. Meantime, ASP.NET event 1310 will recorded as well. According to KB 821268 and MSDN, this is the recommended setting for a two CPU system (Dual core is considered as 2 CPU).


 


Here is what we see in the browser:


 


Parser Error Message: The value for ‘minFreeThreads’ must be less than the thread pool limit of 100 threads.

Source Error:






 


Line 136:  </system.data>


Line 137:  <system.web>


Line 138:                 <httpRuntime minFreeThreads=”176″ minLocalRequestFreeThreads=”152″ />


Line 139:                 <processModel maxWorkerThreads=”100″ maxIoThreads=”100″ minWorkerThreads=”50″ />


Line 140:



Source File: C:\WINNT\Microsoft.NET\Framework\v2.0.50727\Config\machine.config    Line: 138


 


Also, we could see 1310 event in the application event log.


 


Event Type:       Warning


Event Source:    ASP.NET 2.0.50727.0


Event Category: Web Event


Event ID:          1310


Date:                2009/3/27


Time:                下午 02:54:23


User:                N/A


Computer:         ZW


Description:


Event code: 3008


Event message: A configuration error has occurred.


…..


…..


Exception information:


    Exception type: ConfigurationErrorsException


    Exception message: The value for ‘minFreeThreads’ must be less than the thread pool limit of 100 threads. (C:\WINNT\Microsoft.NET\Framework\v2.0.50727\Config\machine.config line 138)


 


Thread information:


    Thread ID: 7


    Thread account name: NT AUTHORITY\NETWORK SERVICE


    Is impersonating: False


    Stack trace:    at System.Web.HttpRuntime.FirstRequestInit(HttpContext context)


   at System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context)


   at System.Web.HttpRuntime.ProcessRequestInternal(HttpWorkerRequest wr)


 


Here is the MSDN suggested settings and its details description. As you can see, customer’s setting is exactly what we suggested.




























Configuration Setting


Recommended Value


maxconnection


12 * #CPUs


maxIoThreads


100


maxWorkerThreads


100


minFreeThreads


88 * #CPUs


minLocalRequestFreeThreads


76 * #CPUs


 


 






Then why this error happens? Based on MSDN, the configuration item in processModel will be automatically multiplied by the number of available CPUs, and MaxWorkerThreads must be equal or greater than minFreeThreads in httpRuntime. For customer’s setting, threads available to the worker process to service request is 100*2-176=24, which are 12 threads per CPU.

 







maxWorkerThreads


5 to 100


Configures the maximum amount of worker threads to be used for the process on a per-CPU basis. For example, if this value is 25 on a single-processor server, ASP.NET uses the runtime APIs to set the process limit to 25. On a two-processor server, the limit is set to 50. The default is 20. The value of maxWorkerThreads must be equal to or greater than the minFreeThread attribute setting in the <httpRuntime> configuration section.


 






 


Now, come back to the original question, why this error happens? The only factor can affect this is process affinity. Now, we are able to reproduce this problem easily.


1.    Create a web application contains hello.aspx


2.    Open IE and visit hello.aspx, the page loaded correctly.


3.    Open task manager, right click the w3wp.exe and select “set affinity”. Change the worker process’s affinity by un-check one CPU.


4.    Open the machine.confog or web.config, make any change and then un-do this change. Save the config file.


5.    Open IE and visit hello.aspx again, IE shows the errors and able to find 1310 in event log as well.


 


Here comes to the new question, who changed my worker process’s affinity? There will be 3 scenarios which may change worker process’s affinity.


1.    Other application changed IIS worker process’s affinity by call SetProcessAffinityMask, example Task Manager.


2.    User changed the worker process’s affinity by adsutil.vbs. This is done through W3SVC and finally, called SetProcessAffinityMask.


3.    The code running in the worker process itself changed the affinity by call SetProcessAffinityMask.


 


For scenario 1, I don’t know how to find out the culprit application. My Windows colleague send me a checked build of Kernel32.dll, with this build, any process call SetProcessAffinityMask will be crashed at once. I don’t have a chance to test this yet.


 


I finally find out the problem by attach debugger to worker process and W3SVC service and set breakpoint at SetProcessAffinityMask. Here is the call stack in the worker process’s debugger’s log. A 3rd party DLL used by customer’s web application is calling SetProcessAffinityMask!


 


# ChildEBP RetAddr 


kernel32!SetProcessAffinityMask(void * hProcess = 0xffffffff, unsigned long dwProcessAffinityMask = 1)


WARNING: Stack unwind information not available. Following frames may be wrong.


AStrangeComponent!DllCanUnloadNow+0x50a76


AStrangeComponent!DllCanUnloadNow+0x10f9a5


…………..


 


 


Regards,


 


Zhao Wei

Comments (1)

  1. Three months ago, I have a blog talking about worker process affinity . Now, we have a similar issue

Skip to main content