Understanding DEP/NX

Despite being one of the crucial security features of modern browsers, Data Execution Prevention / No Execute (DEP/NX) is not well understood by most users, even technical experts without a security background.

In this post, I’ll try to provide some insight into how DEP/NX works, explain why you might encounter a DEP/NX crash, and convince you that turning off DEP/NX is almost never the right decision.

More than anything else, I hope you take away two important facts from reading this post:

  • In many cases where you encounter a DEP/NX crash, the browser would have crashed anyway.
  • The vast majority of DEP/NX crashes are caused by browser add-ons. If you run IE in No Add-ons Mode, it’s very unlikely that you will encounter a DEP/NX crash.


I’ll begin by providing some background information on DEP/NX and how the browser makes use of it.

What is DEP/NX?

DEP/NX is a feature of modern CPUs that allows marking of memory pages as Executable or non-Executable. This allows the CPU to help prevent execution of malicious data placed into memory by an attacker. If the CPU detects that it is about to jump to (begin execution of) data which is in a memory page which is not marked as Executable, the CPU will raise an exception which results in termination of the process.

Stated another way, if DEP/NX determines that if a potentially dangerous jump is about to be made, the process is intentionally “safely crashed” to prevent a potential security exploit. You can get more DEP/NX details here.

Checking Your Protection

You can see which processes are protected by DEP/NX using Task Manager’s Process tab. On Windows XP, you need to use Process Explorer instead. In either case, ensure that the “Data Execution Prevention box” is checked in the View > Select Columns menu, and a column in the process list will show the DEP/NX protection status.

Process Explorer showing DEP Permanent for iexplore.exe

As mentioned last year, Internet Explorer 8 enables DEP/NX protection by default. In IE7 and earlier, DEP/NX was disabled by default due to compatibility concerns that were resolved in IE8.

Opting-in to DEP/NX

Internet Explorer 8 uses the SetProcessDEPPolicy() API to enable DEP/NX. This provides the following benefits versus using the /NXCOMPAT linker flag:

  • It allows us to offer an Internet Control Panel checkbox and Group Policy option to disable DEP/NX if desired.
  • It enables DEP/NX on Windows XP SP3. The Windows XP loader does not check the NX Compatible bit.
  • It ensures that ATL_THUNK_EMULATION, an important compatibility feature, works properly.

Note: New applications without 3rd-party code compatibility concerns, targeted for use on Vista and later, should simply use the /NXCOMPAT linker flag.

Recognizing a DEP/NX Crash in Internet Explorer

When Internet Explorer 8 recovers from a DEP/NX-induced crash, it will not automatically recover the current tabs. This is a security measure designed to help prevent a malicious site from having multiple attempts to exploit a vulnerability. Instead of reloading the tabs, the browser will show the following error page:

Error page for DEP/NX Crash Recovery

Unfortunately, the nature of DEP/NX crashes makes it infeasible for the browser to “pin the blame” on the specific add-on that is responsible for the problem.

Why do DEP/NX Crashes Occur in the Real World?

Now, let’s take a look at why users encounter DEP/NX crashes in the real-world.

When the CPU is about to jump to a non-Executable memory page, there are three possible types of data in that page: malicious code, non-malicious code, and garbage data. I’ll discuss each of these in the following sections.

Jump Target: Malicious code

This is the scenario where DEP/NX shines. In this scenario, an attacker has put malicious data in memory that will be executed as x86 instructions if he can get the CPU to jump to it. The attacker then exploits some vulnerability to induce the CPU to jump to his data, typically using a memory-related vulnerability in an add-on or the browser itself.

In this scenario, the CPU notes that the attacker’s code is not in an executable memory page and prevents the interpretation of the attacker-supplied data as instructions. The attack is foiled and the user’s machine is protected. If not for DEP/NX, the attacker would have been able to execute his instructions and potentially infect the user’s machine with malware, steal their data, or achieve some other nefarious goal.

Now, the obvious next question is: What if the attacker can somehow get his data marked as executable?

The answer is that doing so is intentionally difficult. IE8 blocks the best known trick used to get the attacker’s data in an executable page. That means that the attacker must find some other way to get the memory page containing his instructions marked as executable.

The obvious choice would be for the attacker to call VirtualProtect() directly, passing PAGE_EXECUTE_READ as the flNewProtect flag. However, thanks to Address Space Layout Randomization (ASLR) it is difficult for the attacker to guess where the VirtualProtect function is in memory. If he guesses wrong (and he almost always will), the process will crash and not execute his attack instructions.

Jump Target: Non-malicious code

In this scenario, a browser add-on is designed in such a way that it expects to be able to execute data from memory pages which are not marked as executable, or otherwise makes a bad assumption.

There are a number of possible cases where this may happen.

Case #1: Code Generation

In the first case, the add-on (or the technology it is built upon) depends on the ability to execute dynamically generated instructions at runtime. Examples of this are the Java Virtual Machine (JVM) and the Active Template Library (ATL). These frameworks generate (“JIT compile”) executable code at runtime and jump to it. Older versions of these frameworks did not mark the memory pages containing the generated code as executable and would hence crash when DEP/NX was enabled. The Java team fixed this problem in the JVM years ago, and the ATL team also fixed this problem several versions ago.

Because ATL is so commonly used to build Internet Explorer add-ons, additional work was done to allow Windows to “emulate” the ATL Thunk code which violated DEP/NX, so that even if an add-on was compiled against an ancient version of ATL, ATL Thunk Emulation will ensure that the code runs properly inside Internet Explorer with DEP/NX enabled.

Case #2: Code Rewriting

In another common case, the add-on depends on “thunking” or modifying an existing Internet Explorer API or Windows function at runtime by rewriting the instructions in the existing function’s memory page. In order to accomplish this, the add-on uses VirtualProtect() to change the memory protection of the target page to allow Write, then updates that page with new instructions that point to some code that the add-on would like to have run inside the target function.

If the add-on fails to subsequently call VirtualProtect() to revert the memory protection back to allow Execute, the process will crash with a DEP/NX violation the next time that function is called.

More commonly, the add-on will later change the memory protection back to allow Execute, but the developer ignores the fact that it’s entirely unsafe to perform modification of shared code while any other threads are executing. While an add-on thread is modifying the code in a memory page, if any thread attempts to call any function in the same memory page, the process will crash. Internet Explorer makes extensive use of threads, so such crashes are likely if an add-on uses thunking.

Because timing is a critical factor here, the add-on may seem to “work fine” on one machine (e.g. a slower single-core machine) and always crash on another (e.g. a fast multi-core machine). This problem is just one of the major reasons why function thunking by Add-ons is not supported and is strongly discouraged.

Jump Target: Garbage data

In this scenario, inadvertent memory corruption has occurred such that the CPU is about to jump to arbitrary data somewhere in memory. This scenario is probably the most common source of DEP/NX crashes, particularly when the crash occurs at a seemingly random time, or when a browser tab is closed.

This arbitrary data isn't usually chosen by an attacker, and usually doesn’t even represent sensible x86 instructions. For instance, the jump may be to an address near 0x000000 where no code is loaded (near-null jump), if a virtual function was called off an object pointer which has been nulled. Or, the jump may be to some other address where code used to exist (stale pointer) but that memory was later freed and potentially reused for another purpose.

In this “garbage data” scenario, the process will almost always crash, even if DEP/NX were not enabled. That’s because the CPU is very unlikely to reliably execute arbitrary data as sensible x86 instructions. Most likely, the process will crash within a microsecond with an exception like “Access Violation”, “Invalid Instruction”,  “Divide by 0” or similar.

Attackers look for this type of memory corruption to use as an entry point in their attacks; they may, for instance, “spray the heap” with many copies of their malicious data, then trigger the memory corruption vulnerability with the hope that the CPU will jump into a copy of their malicious code.

Resolving DEP/NX Problems

Your best bet to resolve DEP/NX problems in Internet Explorer is to first confirm that the problem is caused by a buggy browser add-on. You can do this by running IE in No Add-ons Mode. After confirming that the problem is related to an add-on, you should use the browser’s Manage Add-Ons feature to disable unwanted add-ons and find updated versions of any add-ons that you wish to keep.

If you find that you’re encountering DEP/NX crashes in multiple software applications, it’s possible that you have malicious or buggy system software installed (e.g. malware or a buggy anti-virus product). You should check your system for malware and ensure that you install the latest updates for your system software.

Frequent DEP/NX crashes also suggest that your computer might have a hardware problem (e.g. bad system memory). To help rule out hardware failure, you can use the Windows Memory Diagnostic.


DEP/NX provides an important defense against malicious websites that may try to exploit vulnerabilities in your add-ons or web browser. By ensuring that you are running the latest version of add-ons and system software, you can improve your security and minimize the incidence of DEP/NX crashes. If you're currently using an older version (6 or 7) of Internet Explorer that does not have DEP/NX protections enabled by default, you should upgrade to IE8 as soon as possible.

Thanks for reading!


Comments (14)

  1. Matt Sharpe says:

    Interesting post.

    I always enable DEP for "all programs and services except those I select" via the System Properties.

    There’s an excellent program at GRC.com called SecurAble which tells you whether your CPU supports hardware DEP.

  2. EricLaw [MSFT] says:

    Over at //bernardodamele.blogspot.com/2009/12/dep-bypass-with-setprocessdeppolicy.html, Bernardo writes a nice post explaining how exploit code could attempt to circumvent DEP on Windows XP by calling SetProcessDEPPolicy.

    That post fails to note two critical points, however:

    1> SetProcessDEPPolicy can only be called once per process, so if the process itself calls the function (as IE8 does on startup) this attack won’t work.

    2> It’s broadly understood that DEP/NX isn’t very useful without ASLR. It’s ASLR that breaks the shellcode’s ability to reliably jump to the SetProcessDEPPolicy function to turn off DEP/NX protection.  

    The attack demo shown in the post is performed on Windows XP, where that pesky ASLR stuff isn’t available.

  3. Salim says:

    Thank you very much Eric for this realy very usefull and excellent article. I had for a week problems with DEP, but I was not sure, that it was not a virus. I followed your advices and solved my problem.

    Further I found a  Add-On: ie-translator, which crash my iexplorer 8.

    Thank you again


  4. EricLaw [MSFT] says:

    Note: In order to change the IE-specific DEP setting for IE7 or IE8, you must use the “Enable memory protection” checkbox inside the Internet Control panel. For historical reasons, IE’s InetCPL UI will not allow the user to set DEP/NX policy on a per-user basis. The checkbox is only enabled when IE/INETCPL is running as Administrator.

    I recently noticed that sometimes, after running IE as Admin, unticking the box, and restarting IE as standard user, DEP/NX is still on and the INETCPL box is still ticked.

    What happened?

    Further investigation (using Process Monitor to watch registry access to the DEPOff key) shows that on the affected machines, the registry key HKCUSoftwareClassesVirtualStoreMachineSoftwareMicrosoftInternet ExplorerMainDEPOff exists with a value of 0. That value is respected and forces DEP on.

    As mentioned, you must run IE as Administrator in order to change the value in the INETCPL. When you do that, it changes the *HKLM* value. However, it doesn’t touch the VirtualStore value, and if re-launch IE as non-admin, if you have the VirtualStore value, it is respected and you’ll find that the checkbox remains checked and IE still has DEP/NX enabled.

    The fix is to delete the bogus DEPOff key copied in the VirtualStore registry area.

  5. Linne says:

    In our company we have trouble with every client because of ActiveX-Plugins (since the update IE6 to IE8).

    Many ActiveX-Plugins (even microsoft support activeX control!) are crashing the IE8. We need the correct function of ActiveX-Plugins, for example for WebEx ActiveX-Plugins.

    But we don't want to disable DEP, because it's an important security function. I disabled all IE add-ins, but this doesn't help.

    What can I try or check else? I found many articles in the internet with users .

  6. @Linne: I'm not aware of *any* Microsoft ActiveX controls that have problems with DEP/NX. I'm also confused when you say "I disabled all IE Add-ins" and it didn't help– if you run IE with add-ons disabled, this prevents ActiveX controls from loading. If IE is crashing even when you run without add-ons, this suggests that some other software (likely anti-virus software) is crashing IE.

  7. Linne says:


    I had this problem with the active-x plugin which must be installed if you use windows update (the first time). Another important plugin is WebEx for our company.

    I didn't start the IE with the "without add-ons modus". I deactivated each add-on mannually to test the behaviour.

    I will test if our anti-virus software (SEP) is crashing the IE. Are there any entries (log etc.) where I could see this?

    Do you have example-links for active-x plugins which are "well programed" and are working with DEP an other which are not "well programed" and are crashing the IE? I'm thankful for every tip and information to solve this issue.



  8. Willy says:

    Wouldn't the setting "Enable memory protection to help mitigate online attacks" have been better under IE's Security?  Thus, under Local Intranet or Trusted zone it'd be off, but under Internet or Restricted it'd be on.  And we can add sites to the zones depending on our circumstances/observations.

    As is, I have to uncheck the setting; do what I need to resolve a user's issue; and re-check that setting.  And most sites I've come across are not telling/advising/recommending people to recheck that setting.  One or two users that's fine, but I could see where it could become a headache to support.  

    I had a user try to download SQL Server 2008 iso from Microsoft MSDN website, and the dep crash page came up.  After we unchecked that settings, we saw that it was trying to get the user to install a File Dowload Manager.  So it DOES happen with a Microsoft site–which we kind of thought was funny/ironic.

  9. EricLaw [MSFT] says:

    @Willy: The setting in question is process-based, not Zone-based, so it cannot be set on a per-zone basis. There should not be any add-ons that are incompatible with DEP at this point, and I've never encountered any Microsoft add-on with a problem which was truly DEP-related and not a case, as outlined above, where DEP/NX simply "dove in front of" a different type of crashing bug.

  10. Linne says:


    Well but we, any many other people have problems with the memory protection. The virus protection is not the reason, I was testing too.

    As discribed: "I had this problem with the active-x plugin which must be installed if you use windows update (the first time). Another important plugin is WebEx for our company."

    I would like to do further testings, so my questions again:

    – Are there any entries (log etc.) where I could see information to the crashes?

    – Do you have example-links for active-x plugins which are "well programed" and are working with DEP an other which are not "well programed" and are crashing the IE? I'm thankful for every tip and information to solve this issue.

  11. EricLaw [MSFT] says:

    The Windows Update ActiveX control is fully DEP-compatible. If you encounter DEP crashes with WindowsUpdate, either you have hardware failure (bad memory) or conflicting system software (e.g. antivirus) which is improperly thunking system APIs and introducing crashes.

    I have yet to find ANY ActiveX sample code which is not DEP-compatible. The only incompatible code I know of (e.g. ancient versions of the Sun Java Virtual Machine) are not open-source.

  12. carlher says:

    Hello Eric, do ActiveX controls that were developed using VB6 can cause DEP/NX crashes as well?

  13. Yuhong Bao says:

    Another advantage is that LCIE allows IE to run non-DEP capable add-ons and DEP capable add-ons in separate processes.

  14. @Yuhong: No such feature exists. DEP/NX is enabled by default for all processes (and should be left in that configuration). If you disable the feature, then it's off for all processes. Generally speaking, very few things aren't DEP-compatible these days.

Skip to main content