/DYNAMICBASE and /NXCOMPAT

Hello, my name is Xiang Fan and I am a developer on the C++ Shanghai team.


Today I’d like to talk about two linker options related to security: /DYNAMICBASE and /NXCOMPAT.


 


These two options are introduced in VS2005, and target to improve the overall security of native applications.


 


You can set these two options explicitly in VS IDE:


 



 


These two options have three available values in the IDE: On, Off and Default.


 


They are set to “On” if you create native C++ application using VS2008 wizard.


When VS2008 upgrades projects created by older version of VC which doesn’t support these options, it will set them to “Off” after upgrade.


If you set them to “Default”, linker will treat it as “Off”.


 


After several years adoption, we plan to change the behavior of “Default” to “On” in VS2010 to reinforce the security. And we’d like to get your feedback.


 


Here are the detailed information about these two options:


 


1.      DYNAMICBASE


 


/DYNAMICBASE modifies the header of an executable to indicate whether the application should be randomly rebased at load time by the OS. The random rebase is well known as ASLR (Address space layout randomization).


This option also implies “/FIXED:NO”, which will generate a relocation section in the executable. See /FIXED for more information.


 


In VS2008, this option is on by default if a component requires Windows Vista (/SUBSYSTEM 6.0 and greater)


/DYNAMICBASE:NO can be used to explicitly disable the random rebase.


 


This article talks about ASLR: http://technet.microsoft.com/en-us/magazine/cc162458.aspx


ASLR is supported only on Windows Vista and later operating systems. It will be ignored on older OS.


 


ASLR is transparent to the application. With ASLR, the only difference is OS will rebase the executable unconditionally instead of doing it only when an image base conflict exists.


 


2.      NXCOMPAT


 


/NXCOMPAT is used to specify an executable as compatible with DEP (Data Execution Prevention)


Notice that, this option applies for x86 executable only. Non-x86 architecture versions of desktop Windows (e.g. x64 and IA64) always enforce DEP if the executable is not running in WOW64 mode.


 


Here is a comprehensive description of DEP:


http://support.microsoft.com/kb/875352


 


This option is on by default if a component requires Windows Vista (/SUBSYSTEM 6.0 and greater).


/NXCOMPAT:NO can be used to explicitly specify an executable as not compatible with DEP.


However, the administrator can still enable the DEP even if the executable is not specified as compatible with DEP. So you should always test your application with DEP on.


 


Windows Vista SP1, Windows XP SP3 and Windows Server 2008 add a new API SetProcessDEPPolicy to allow the developer to set DEP on their process at runtime rather than using linker options. See the following link for more details:


http://blogs.msdn.com/michael_howard/archive/2008/01/29/new-nx-apis-added-to-windows-vista-sp1-windows-xp-sp3-and-windows-server-2008.aspx


 


There are several common (and incomplete) patterns which are not compatible with DEP (See http://msdn.microsoft.com/en-us/library/aa366553.aspx for more information.)


a.       Dynamic code generated in heap or stack


If your application must run code from a memory page, it must allocate and set the proper virtual memory protection attributes.


 


The allocated memory must be marked PAGE_EXECUTE, PAGE_EXECUTE_READ, PAGE_EXECUTE_READWRITE, or PAGE_EXECUTE_WRITECOPY when allocating memory. Heap allocations made by calling the new, malloc and HeapAlloc functions are non-executable. An application can use the VirtualAlloc function to allocate executable memory with the appropriate memory protection options.


 


Another option is to pass HEAP_CREATE_ENABLE_EXECUTE when create the heap via  HeapCreate. Then the memory allocated by the subsequent HeapAlloc will be executable.


 


b.      Executable code in data section


They should be migrated to a code section


 


Security vulnerabilities are more exploitable than they would be if DEP were enabled. So you should always make your application DEP compatible and turn DEP on.


 


The following sample demonstrates the code which is not compatible with DEP.


It also shows two DEP compatible ways to run the code on heap.


Running code in data section or on stack almost always implies security holes. You have to put the code in code section or heap instead.


 


#include “windows.h”


#include <cstdio>


 


typedef void (*funType)();


unsigned char gCode[] = {0xC3}; // ”ret” instruction on x86


const size_t gCodeSize = sizeof(gCode);


 


// these are not DEP compatible


 


void RunCodeOnHeap()


{


    unsigned char *code = new unsigned char[gCodeSize];


    memcpy(code, gCode, gCodeSize);


 


    funType fun = reinterpret_cast<funType>(code);


    fun();


 


    delete []code;


}


 


// these are DEP compatible


 


void RunCodeOnHeapCompatible1()


{


    unsigned char *code = (unsigned char *)::VirtualAlloc(NULL, gCodeSize, MEM_COMMIT, PAGE_READWRITE);


    memcpy(code, gCode, gCodeSize);


 


    DWORD flOldProtect;


    ::VirtualProtect(code, gCodeSize, PAGE_EXECUTE_READ, &flOldProtect);


 


    funType fun = reinterpret_cast<funType>(code);


    fun();


 


    ::VirtualFree(code, 0, MEM_RELEASE);


}


void RunCodeOnHeapCompatible2()


{


    HANDLE hheap = ::HeapCreate(HEAP_CREATE_ENABLE_EXECUTE, 0, 0);


 


    unsigned char *code = (unsigned char *)::HeapAlloc(hheap, 0, gCodeSize);


    memcpy(code, gCode, gCodeSize);


 


    funType fun = reinterpret_cast<funType>(code);


    fun();


 


    ::HeapFree(hheap, 0, code);


    ::HeapDestroy(hheap);


}


 


INT DEPExceptionFilter(LPEXCEPTION_POINTERS lpInfo)


{


    // please check http://technet.microsoft.com/en-us/library/bb457155.aspx


    // for more information


 


    if (lpInfo->ExceptionRecord->ExceptionCode == STATUS_ACCESS_VIOLATION &&


        lpInfo->ExceptionRecord->ExceptionInformation[0] == 8) {


        return EXCEPTION_EXECUTE_HANDLER;


    }


 


    return EXCEPTION_CONTINUE_SEARCH;


}


 


int main()


{


    __try


    {


        RunCodeOnHeap();


        printf(“RunCodeOnHeap: OK\n”);


    }


    __except (DEPExceptionFilter(GetExceptionInformation()))


    {


        printf(“RunCodeOnHeap: Fail due to DEP\n”);


    }


    __try


    {


        RunCodeOnHeapCompatible1();


        printf(“RunCodeOnHeapCompatible1: OK\n”);


    }


    __except (DEPExceptionFilter(GetExceptionInformation()))


    {


        printf(“RunCodeOnHeapCompatible1: Fail due to DEP\n”);


    }


    __try


    {


        RunCodeOnHeapCompatible2();


        printf(“RunCodeOnHeapCompatible2: OK\n”);


    }


    __except (DEPExceptionFilter(GetExceptionInformation()))


    {


        printf(“RunCodeOnHeapCompatible2: Fail due to DEP\n”);


    }


}


 


Output:


 


cl test.cpp /link /nxcompat:no


 


RunCodeOnHeap: OK


RunCodeOnHeapCompatible1: OK


RunCodeOnHeapCompatible2: OK


 


cl test.cpp /link /nxcompat


 


RunCodeOnHeap: Fail due to DEP


RunCodeOnHeapCompatible1: OK


RunCodeOnHeapCompatible2: OK


 


In summary, “cl test.cpp” is equivalent to “cl test.cpp /link /nxcompat:no /dynamicbase:no” before VS2010. We plan to change it to “cl test.cpp /link /nxcompat /dynamicbase” in VS2010.


If you have any concerns about the default behavior change of these two options, don’t hesitate to give your feedback. Thanks!


 


Regards,


    Xiang