On enabling NX and ASLR for a module after the fact


A customer wanted to enable NX (also known as Data Execution Prevention, or DEP) and ASLR for some executables and DLLs. There are two ways of doing this:

  • Enable the options at link time by passing the linker command line options /NX­COMPAT and /DYNAMIC­BASE, and for good measure /HIGH­ENTROPYVA.
  • Build the modules the usual way, and then use the EDITBIN program with those same command line options to enable the features on the files in question.

For reasons the customer didn't provide (but which I can guess),¹ there are a handful of files that they cannot relink, so they are forced to use the EDITBIN approach for those files.

What the customer found was that both the linker and the EDITBIN approaches seemed to be fine with /NX­COMPAT. But the story was different with the /DYNAMIC­BASE flag.

Specifically, the linker approach was creating a larger file due to a new .reloc section. The file produced by the EDITBIN approach didn't have a .reloc section, and dumping the header reports Relocations stripped. Furthermore, when running EDITBIN, it generated the ominous warning message, "Warning LNK4259: '/DYNAMIC­BASE' is not compatible with '/FIXED'; image may not run." It appears that linking with the /DYNAMIC­BASE flag implicitly sets /FIXED:NO, but the EDITBIN command doesn't apply the same behavior, and it doesn't support the /FIXED:NO command line option.

The customer had the following questions:

  • Are the linker approach and the the EDITBIN approach for enabling /NX­COMPAT equivalent?

Yes, enabling NX via the linker /NX­COMPAT flag is equivalent to enabling it with EDITBIN. In both cases, they set the IMAGE_DLL­CHARACTERISTICS_NX_COMPAT bit in the IMAGE_OPTIONAL_HEADER's Dll­Characteristics.

  • Are the linker approach and the the EDITBIN approach for enabling /DYNAMIC­BASE equivalent?

The answer depends on what color glasses you're wearing. If you're wearing kernel-colored glasses, then yes, the two approaches are the same because both set the IMAGE_DLL­CHARACTERISTICS_DYNAMIC_BASE bit in the IMAGE_OPTIONAL_HEADER's Dll­Characteristics. But in reality, they are not the same, because of the behavior noted above with respect to relocations. If you ask the linker for /DYNAMIC­BASE, it will default to /FIXED:NO because /DYNAMIC­BASE means "My base address can be moved around", which is the opposite of /FIXED, which means "My base address cannot change."

If you say /FIXED, then the linker does not generate "relocations", which are the bits of information that describe what adjustments need to be made to your DLL in order to make it happy at its new location. Trying to turn on /DYNAMIC­BASE with EDITBIN on a binary that is /FIXED doesn't work because EDITBIN doesn't know how to regenerate the .reloc section.

If you want to enable /DYNAMIC­BASE, then you cannot link with /FIXED.

  • Are there any adverse consequences of mixing ASLR-enabled DLLs and non-ASLR-enabled DLLs in the same process?

There are no consequences beyond those already stated on the tin. The ASLR-enabled binaries will be subject to ASLR, and the non-ASLR-enabled binaries will not. You can mix and match freely within a process. Just be aware that the non-ASLR-enabled binaries will not be randomly relocated, which means that those binaries will not benefit from the security protections provided by ASLR.

  • Are there any adverse consequences of mixing NX-enabled DLLs and non-NX-enabled DLLs in the same process?

The NX setting is process-wide, and the process takes its NX state from the /NX­ENABLED state of the executable, not from any DLLs. It's yet another one of the module flags that are meaningless for DLLs. So mix it up in the DLLs all you want. Nobody will care, because the flag is ignored for DLLs anyway.

  • Are there any adverse consequences of mixing EXEs and DLLs which use different approaches for enabling NX and/or ASLR? For example, is there an unexpected interaction between an EXE which enabled ASLR with EDITBIN and a DLL that enabled ASLR with a linker switch? How can I tell whether an EXE or DLL has had its NX or ASLR bit set via the linker as opposed to the EDITBIN program?

There is no way to tell how the NX or ASLR attributes were enabled. Using the linker or EDITBIN both lead to the same binary. There's nothing that records who set the bit. As a result, it doesn't matter how you enabled NX and ASLR. The system behaves the same regardless of how you enabled them.

Bonus chatter: There is some text on MSDN (which got copied into a glossary on TechNet) which says

For a component to support ASLR, all components that it loads must also support ASLR. For example, if A.exe consumes B.dll and C.dll, all three must support ASLR.

Nobody is quite sure how that text got into the documentation, since it's not true. One of the original authors of that article surmises that perhaps what they were trying to say was "In order for a process to take full advantage of ASLR, the executable and all DLLs must support ASLR," but somehow the message got garbled during editing.

Given what we already know about how the loader shares pages in the face of ASLR, having a DLL be subject to ASLR in some processes but not others would create extra work for the loader, because it now has to be able to keep up to two copies of every DLL in memory: A randomly-located version for ASLR, and a non-relocated version for non-ASLR. It's extra work for no real benefit, so I can understand why they don't do it.

¹ I can think of a few reasons why the customer cannot relink all of the files. One possibility is that they don't have the source code any more. Another is that they have the source code, but they don't have the build tools any more. (For example, it may be built with a very old compiler that doesn't work any more.) Or they have the source code, and they have the tools, but they simply don't want to take the risk that relinking the file might result in an unexpected change to the program. (For example, if it was linked with an older version of the linker, and they have since upgraded to a newer version.)

Comments (12)
  1. Brian_EE says:

    Isn’t another reason that perhaps they are 3rd party DLLs?

    1. morlamweb says:

      Raymond mentioned that as the first possibility: the customer doesn’t have the source code for the binaries. They may have had it at one point until it got archived and they lost, the tape, or they’re 3rd part binaries and they never had the source in the first place. It’s a meaningless distinction. Either way, the customer doesn’t have the source.

      1. GregM says:

        “Raymond mentioned that as the first possibility”

        Sort of, but not really. “One possibility is that they don’t have the source code any more.” implies that they had the source code at one point and that they built it. If this had stopped before “any more” then I would agree that it would completely cover 3rd party DLLs. I had the exact same thought, and it fits right in at the beginning of the progression:

        – They never had the source code.
        – They don’t have the source code any more.
        – They don’t have the build tools any more.
        – They don’t want to take the risk.

  2. anai says:

    “Nobody is quite sure how that text got into the documentation, since it’s not true.”

    1) Am I currently protected by ASLR? – >>Yes
    2) Load Component A.exe (/DYNAMIC­BASE)
    3) Component A requires Component B.dll (/FIXED) so it loads
    4) Am I still protected by ASLR? – >>Not so much

    So does Component A “support ASLR”? – I think you could be kind to the authors and say “no”.
    That is, “support” could be read as “operate within the security mechanism and not break or weaken it”.

    1. philiplu says:

      I’m pretty sure that was the original meaning of the documentation – if any DLL in a process isn’t protected by ASLR, it’s much easier to take advantage of the fixed addresses to escalate a buffer overrun anywhere into remote code execution. In fact, I think I might be partly responsible for that text – I remember discussing this with folks over in the Windows team back when I was working on the Visual C++ runtime components of the various security mechanisms, in 2005 or so.

      1. Unfortunately, some customers interpreted the wording to mean “If B.DLL or C.DLL is /FIXED, then the /DYNAMICBASE attribute of A.EXE is ignored and it will not be subjected to ASLR.”

        1. Joshua says:

          I’m surprised that /DYNAMICBASE does anything with /FIXED set in the header. (OK I know that’s really !(relocations stripped) but that’s not the point.) It just seems like the kernel would conclude that it can’t relocate it and it got its base address anyway.

  3. It is left as an exercise to the reader to create a tool that can fixup the locations and create a .reloc section.

    1. 640k says:

      The author of editbin should be able to paste the linker source into the code base.

    2. Jonathan Wilson says:

      I recently wrote some code that needed to do exactly that (parts bits of code and data in the exe file and generate a bunch of relocations) and I can tell you its not possible to fully automate the process, you need to know in advance whether bits of data in the exe file are addresses or just plain data.

      1. Naturally; even IDA, hallowed be its name, still gets code vs data wrong occasionally when no obfuscation at all is being done, let alone when it is. Anything one of us mortals makes is bound to be worse.

  4. cheong00 says:

    On the other hand, since Delphi always include a .reloc section regardless of whether it’s EXE or DLL, using EDITBIN to alter “/DYNAMIC­BASE” flag for these binaries will work.

Comments are closed.

Skip to main content