The default stack size of IIS starting with Windows Server 2003 was changed. Prior to 2003, it was set to 1 MB. This is also true of the aspnet_wp.exe process that Windows 2000 uses.
In 2003 and later, it is now set to 256 KB.
This can cause a process to run out of stack space if it has something that uses a recursive function.
This is talked about in our Knowledge Base here.
If you want to change this back to the 1 MB that it used to be, you can use editbin to do this.
EDITBIN /STACK:1048576 w3wp.exe
This will change the process. You can use dumpbin to view the sizes:
This is before making the change.