Protecting against Pointer Subterfuge (Kinda!)

When exploiting a buffer overrun vulnerability, the goal of an attacker is usually to change the flow of execution from the normal execution flow to a flow dictated by the attacker. Sure, he may want to whack a DWORD in memory or change a variable, but usually he wants to get shellcode running.


The classic attack is to whack sensitive data on the stack, most notably the function return address; but, there is more than one construct the attacker is interested in; other examples include exception handlers and pointers, especially pointers to code.


Here’s a quick (and highly contrived) code example:


void SomeFunc() {

  // do something



typedef void (*FUNC_PTR )(void);


int DangerousFunc(char *szString) {

  char buf[32];



  FUNC_PTR fp = (FUNC_PTR)(&SomeFunc);

  // Other code

  // Other code


  return 0;



If an attacker controls the szString argument to DangerousFunc, then of course, the stack-based buffer named buf is ready to get clobbered. You know the rest of the story. Let’s assume for a moment that the attacker can’t whack the function return address (perhaps szString isn’t long enough) that leaves the function pointer, fp, as a likely victim. If the attacker can overwrite the function pointer then he can make it point to another address, perhaps exploit code and wheh (*fp)() is called, the ‘sploit runs. So we need to protect the function pointer.


Oh, something I should add, we re-arranged the stack layout in VC++ 2003 and VC++ 2005, so buffers are higher in memory than non-buffers, so to whack fp, you’d need a buffer under-run. But humor me.


In Windows XP SP2 and Windows Server 2003 SP1 we added functions to the OS to help defend such pointers from attack. The functions are:


EncodePointer and DecodePointer

EncodeSystemPointer and DecodeSystemPointer


All these functions do is XOR a pointer with a random number; the first two functions use a random number held in every application’s  process information block, and the system versions use a value that is tied to each system reboot and is accessible to every application running on the box.


Now take a look at the following somewhat safer code:


int DangerousFunc(char *szString) {

  char buf[32];



  FUNC_PTR fp = (FUNC_PTR)EncodePointer(&SomeFunc);

  // Other code

  // Other code

  FUNC_PTR fp2 = (FUNC_PTR)DecodePointer(fp);

  if (fp2) (*fp2)();

  return 0;



Rather than using the pointer directly, the pointer is encoded and stays encoded until you’re about to use it. This makes is more difficult for an attacker, because he must overwrite the function pointer with a value that would calculate correctly after the XOR operation. Not impossible, but highly unlikely.


Which leads me to entropy. The per-process random number is derived from:


  • The higher portion of the system tick count (100 nano-second tick count since Jan 01, 1601) XOR

  • The lower portion the system tick count XOR

  • The number of interrupts since system boot XOR

  • The number of system calls since system boot.

The source code example I showed is somewhat contrived, but these pointer encoding functions become really useful if you have global or very long-lived pointers, for example, in Windows we use these functions to encode unhandled exception filter addresses and more.


You shouldn’t use it for every pointer you use, because that would simply bog the system down, and require a ton of code churn – rather, you should pick your battles. If you have long-lived function pointers in critical Internet-facing code that are not often accessed, I would seriously consider using these APIs.


Finally, the code samples I showed are of course, lousy. If you compile this code with VC++ 2005 you’ll be told that strcpy is bad and you should use something else, like strcpy_s. But because the buffer size is a constant, VC++ 2005 can automatically change the call from strcpy to strcpy_s.


(Thanks to Neill Clift, Shawn Hernan and Nitin Kumar Goel for their draft reviews)

Comments (10)
  1. ilja says:

    That’s pretty neat. Certainly raises the bar. But I think attacking this isn’t as unlikely as you said.

    lets assume an attacker controls a huge amount of memory, 512 megs of it for example (assuming ia32 !), then it doesnt really matter what he overwrites the pointer with, he gets a 1/4 chance of hitting his nop sled and shellcode.

    more realisticly, let’s assume an attacker controls 65535 bytes of memory starting at 0xaabb0000. then all he has to do is get the first 2 bytes rights and it’ll hit somewhere in the nopsled, which is about a 1/65535 chance, I think most attackers can live with that (assming that if the service he attacks restarts after it crashes). Also, if there are decent informations leaks (I mean a read-anything-anywhere infoleak) the attacker could just smash the function pointer XOR’ed with what just got leaked and gets control of execution flow the 1st try. If there is a not-so-decent info leak (attacker gets to leak the encoded pointer) there is still a high probability a he can hit his shellcode. Since the text segment is fairly predictable (or is .text randomized on windows these days) an attacker can probably figure out the 2 msb’s of the pointer (again, assuming ia32). hitting the shellcode probably wouldnt take more then 1 to a couple of 100 tries.

  2. Artem Frolov says:

    Hello Michael,

    Your link‘s%20Web%20Log.aspx does not appear to be working. And I would _really_ want to read it because automatic conversion to strcpy_s (almost wrote ‘strlcpy’ *wink*) is a really cool thing.

    Just to give you an example of a question that immediately popped up in my head (not that I am asking you to answer it): how elaborate is data flow analysis that supports this conversion?

    For example, if I allocate memory on heap, would compiler still convert to strcpy_s if expression is constant, may be evaluated at compile-time or live, like this:

    char* buf = (char*) malloc(n);

    strcpy(buf, arg); // = strcpy_s(buf, n, arg)??

  3. Alun Jones says:

    It’s a shame there’s no simple way for the compiler to insert a check on every function call to ensure that the pointer being called is one that has come from an entry point or loop in the original object code.

    I’ve amended your statement that "all data is evil" with a corollary of my own – "anything that isn’t in the source code is data" – to remind myself (and those that work with me) that data isn’t just what you get from files or the network; it’s mouse movements and keystrokes and window messages and timer callbacks and events and semaphores and … well, you get the idea – anything that isn’t hard-coded.

    Using EncodePointer / DecodePointer strikes me as a relatively simplistic step in the arms race against hackers, though – isn’t this still possibly going to put the decoded pointer on the stack? It’ll likely be evaluated in a register most of the time – but then you’re at the mercy of that unpredictable moment when a compiler optimisation quirk puts you in the situation of having a stack pointer available to overwrite again.

    I don’t have enough mathematical ability this early in the morning to figure out exactly what other operations might be possible to subvert in this – for instance, if the attacker has the ability not just to overwrite a pointer, but to add to it, or some other operation (obviously if he can XOR it, he’s got a win) – what pointer arithmetic operations might be necessary to avoid?

  4. Jesse says:

    Very cool, I’d never heard of these functions before. I also didn’t know about the stack layout change in the compiler… very nice!

  5. Steven Alexander Jr. says:

    In the code examples above, the function pointers are defined after the buffers which means that they will be lower in memory even before VS 2003/2005 rearranges the stack layout.

  6. Stoyan says:

    Jesus! 🙂 The Encode/Decode-Pointer functions smell like they were added to whitewash shit 🙂

  7. Windows Vista Beta 2 includes a new defense against buffer overrun exploits called address space layout…

  8. In a prior post, "Protecting against Pointer Subterfuge (Kinda!)" I described the algorithm we used to…

  9. Web Resources


    [SQL Server and Data Access] 2006 PASS Community Summit: Microsoft SQL…

  10. Cuarta Protección: Ocultación de Información Con las tecnologías vistas hasta el momento se ha pretendido,

Comments are closed.

Skip to main content