Does DEP/NX affect web applications?

A while a go when I posted about the .NET Framework 3.5 and 2.0 SP1 being available for download, Kima posed an interesting question in the comments. Kima asked whether the new enforcement of NX compatability by the C# compiler after the application of 2.0 SP1 would affect ASP.NET applications? Specifically - is aspnet_wp.exe declared as NX compatible or not?

First of all a few things need explaining. Number one, what is NX?

Well, NX is all about providing one more tool in the armoury to try and prevent malicious code from executing. Strictly speaking here, we are talking about Data Execution Prevention (DEP) which is a feature of Windows that will trigger a special exception if the CPU tries to execute any machine instructions in pages of the virtual address space that are not meant to contain any executable code. Now some CPU architectures provide support for this "out of the box" whereas some don't. And if yours doesn't, certain versions of Windows (since round about Windows XP SP2) implement DEP in software instead.

A detailed description of DEP can be found here:

A detailed description of the Data Execution Prevention (DEP) feature in Windows XP Service Pack 2, Windows XP Tablet PC Edition 2005, and Windows Server 2003

Now this question centres around the fact that the default DEP configuration for Windows is OptIn. Only those processes that declare themselves as having been made compatible with DEP have it applied to them.

So how does a process declare itself as being compatible with DEP? This boils down to having the appropriate bit set in the PE header for the EXE itself. That's it. One little bit is all it takes. The bit in question is in the DllCharacteristics field of the IMAGE_OPTIONAL_HEADER structure. The bit that has to be set is 0x100 which has the symbolic name IMAGE_DLLCHARACTERISTICS_NX_COMPAT. How can this be set? Well, normally it would be set by the linker when it generates the image although theoretically you could set it post link using EDITBIN.

Now after you install  .NET Framework 3.5 or apply 2.0 SP1, the C# compiler (csc.exe) is updated such that the images it produces have this bit set. Now managed assemblies are themselves always NX_COMPAT (provided they are using safe code at least) but if they use any non NX_COMPAT DLLs this could cause a problem.

But what about web applications? Well, when you run your web application your code is hosted inside either w3wp.exe (IIS6/7) or aspnet_wp.exe (IIS5/IIS5.1(XP)/IIS6-in-5.0-compat-mode). All you need to do to find out if an EXE is marked as NX_COMPAT is use dumpbin.exe to examine the appropriate bit of the PE header:

>dumpbin /headers aspnet_wp.exe | find "DLL characteristics"
            8000 DLL characteristics

>filever aspnet_wp.exe
--a-- W32i APP ENU  2.0.50727.1433 shp     33,280 10-24-2007 aspnet_wp.exe

So, the "DLL characteristics" of aspnet_wp.exe in 2.0 SP1 (version .1433) does not include the 0x100 bit. Therefore it is not "opting in" to DEP. And the same goes for w3wp.exe and dllhost.exe:

>dumpbin /headers dllhost.exe | find "DLL characteristics"
            8000 DLL characteristics

>dumpbin /headers inetsrvw3wp.exe | find "DLL characteristics"
            8000 DLL characteristics

Well, these are all "generic host processes". Their role in life is to act as a host process for application code, from 10 year old ASP applications using VB6 DLLs and legacy third party components through to cutting edge ASP.NET 3.5 AJAX and LINQ enabled applications.  If we marked the EXE as NX_COMPAT then DEP would be applied to these processes and many of these older applications would break.

One interesting point to note is that in Windows Vista SP1, Windows XP SP3 and Windows Server 2008 we have added some new APIs (which Michael Howard blogged about the other day) that allow a process to set the DEP policy for itself. The interesting thing is that on of the options is to enable DEP for the process but disable ATL thunk emulation which was the main reason that older ATL components did not sit comfortably with DEP.

So if you are happy that all the components in your web application(s) are compatible with DEP (and that is a big 'if') then you might be able to use these new APIs to enable DEP. Now I cannot confirm whether this is officially supported or recommended by Microsoft for web application in general or ASP.NET applications in particular but it is an interesting idea.

So the answer to the original question, "Does NX affect web applications?",  is, I think, no. The host process is not marked NX_COMPAT so on most 32-bit systems with a default configuration DEP will not apply to the process hosting your web application. Assemblies compiled as part of the ASP.NET dynamic compilation will of course get compiled by the C# compiler (if the use C#) so if 2.0 SP1 is present (which it always is if you are using 3.5) those assemblies are marked with NX_COMPAT:

>dumpbin /headers ClassLibrary1.dll | find "DLL characteristics"
             540 DLL characteristics

But, because they are loading into a process that is not opting in for DEP, DEP will no be applied since it applies at the process level not the assembly level.

Now, one thing I have not checked yet is whether w3wp.exe is marked as NX_COMPAT in Windows Vista and Windows Server 2008. But that will have to wait for another day....



Comments (3)
  1. hudabi says:

    Now, one thing I have not checked yet is whether w3wp.exe is marked as NX_COMPAT in Windows Vista and Windows Server 2008. But that will have to wait for another day….

    nice article

  2. goran.kimovski says:

    Hey Doug,

    I can confirm that w3wp.exe is marked NX_COMPAT on Vista, which make me suspect the flag was added in IIS7 and will likely stay there on Windows Server 2008 too. I believe Mike’s answer to my comment at confirms this too.

    I would expect many customers to find themselves in a situation where migrating from IIS6 to IIS7 proves to be painful. This will likely lead to many requests back to third-party component vendors that still rely on native code and use ATL prior to v8 to upgrade their projects to VS2005 or VS2008 and re-ship the binaries again. As you can see from Ed’s blog on the .NET 2.0 SP1 changes regarding the NX bit at people already start to feel the frustration. I suspect the IIS7 impact will be much bigger.

    In any case, I am off to upgrade my compiler 🙁 and I hope this info is useful to others facing the same problem.



  3. Mike Dimmick says:

    Just a quick note: it is the ATL thunks *themselves*, not the emulation, which are not DEP-compatible. (That’s because they are code, which recovers the ‘this’ pointer for a window when a window procedure is called, which is written to a part of the object itself, which is typically allocated on a heap.) The thunk *emulation* detects that the code that caused an access violation exception, due to being on an NX-marked page, was an ATL thunk and emulates the behaviour of that code. This emulation works in any process that’s opted into DEP. There’s a risk, though, that the thunk has been trashed by a buffer overflow and it could be exploitable (because the ‘this’ pointer points to the vfptr which points to the vtable, which in turn is a table of pointers to code).

    Disabling ATL thunk emulation means that legacy ATL code which used this behaviour won’t work, but it avoids the possibility that something that wasn’t an ATL thunk gets picked up as one, which could be an avenue for exploits. It’s a tricky balance between compatibility and security.

Comments are closed.

Skip to main content