Understanding "Magic" Pointers and Offsets


With this blog post I try to explain how “magic” pointers and offsets work.


I just copied the term “magic” to refer to these kinds of pointers or offsets:


 


dd poi(0x129514 + 0x18) + 0x8 L2


du poi(0x0007de95)


du poi(poi(poi(0x129514 + 0x9c)) + 0x4)


dd poi(0x129514 + 0x34)


 


To use an analogy: it is similar to the “magic number” term we use for programs that access a value using a number instead of using a constant, like:


 


b = 3.14;  // Magic number.


 


Instead of:


 


const float PI = 3.14;


 


b = PI;


 


 


I decided to blog about it because sometimes I run into people that look at these and wonder how these expressions can return valid data and how to create them.


Actually it’s very simple!


 


Let me explain: The commands above display values or Unicode strings from specific locations.


These locations are selected based on pointers and offsets. These offsets usually come from structs (struct is the short term for structure. It’s a C/C++ reserved word) or classes. The structs or classes can be seen when using private symbols or source code access. To create them it’s necessary to have private symbols or source code access because you need to see the class or struct to be able to create a command that displays information for a specific field without using any symbol.


 


Ok, at this point you may wonder why not use the variable names instead of using offsets.


The answer is because sometimes you just don’t have private symbols.


Thus, when you use specific offsets from a specific address, you work around the lack of symbols.


This is great; therefore, I use this approach in some of my scripts. That’s why it works for you using just public symbols or no symbols at all, like this one here. The downside is that your command or script depends on the specific offsets and pointers. In other words, if the internal class or struct changes, you might need to change your command, or you won’t get the right information anymore.


 


This is funny: very often I see that inexperienced debuggers get impressed when they see a command using several layers of offsets.


Probably they don’t know the person who created the command memorized it because he uses it over and over. The person created it because he/she had access to the source code!


Yes, this is how we can create these kinds of commands using these magic pointers! :)


 


A lot of engineers at Microsoft, including myself, have a cheat sheet file where we save many “magic” pointers and offsets.


Some of the commands I have:


      Extracts PerfMon counters from mscorsvr/mscorwks.


      Extracts ASP page number being executed from an ASP call.


      Extracts ASP template from an ASP call.


      Extracts ASP source code from an ASP call.


      Extracts SQL command from OLEDB call.


      Extracts Connection string from OLEDB call.


      Extracts HRESULT from call stack.


      Extracts COM  Object from OLE32 call.


      Extracts Remote Server IP Address from Winsock call.


 


We do that because we might forget the specific offsets and have a situation where we don’t have private symbols access, for example, when working from home without being able to access the office computer or when working from a customer’s site.


 


When we have this situation, we just open our cheat sheet file and use the command to get the information we want without needing to read the source code.


 


Whenever you see this kind of “magic” pointer or offset you should consider it might not work with newer product versions. Keep it in mind, and you won’t be frustrated.


 


That said; let me exemplify how this “magic” pointer works and what might happen when the struct or class being used changes.


 


Source code for the application MagicPointer:


 


 


#include “stdafx.h”


#include <conio.h>


 


struct stData


                   {


                             TCHAR szName[21];


                             int nVersion;


                             TCHAR cLetter;


                             int nCode;


                             // int nCode2;


                             TCHAR szSomeSentence[51];


                   };


 


void DoSomething(stData* pst);


 


                            


int _tmain(int argc, _TCHAR* argv[])


{       


    stData* pstStruct = new stData();


         


    DoSomething(pstStruct); 


   


    _getch();


    delete pstStruct;


   


    return 0;


}


 


void DoSomething(stData* pst)


{


    _tcscpy_s(pst->szName, L“Test”);


    pst->nVersion = 5;


    pst->cLetter  = L‘D’;


    pst->nCode    = 3;


    _tcscpy_s(pst->szSomeSentence, L“Some Sentence”);


 


}


 


I used Visual C++ 2005 and compiled the application above using a Debug build.


When debugging this application using symbols, I have the structure below after executing the last line from DoSomething():


 


Local var @ 0x17fe44 Type stData*


0x005f5600


   +0x000 szName           : [21]  “Test”


   +0x02c nVersion         : 5


   +0x030 cLetter          : 0x44 ‘D’


   +0x034 nCode            : 3


   +0x038 szSomeSentence   : [51]  “Some Sentence”


 


Since I have symbols and source code access, I can see the offsets above for each field.


 


Let’s suppose I want to dump the szSomeSentence field.


I can use:


 


0:000> du @@c++(pst->szSomeSentence)


005f5638  “Some Sentence”


 


Or:


 


0:000> ?? (wchar_t*) pst->szSomeSentence


wchar_t * 0x005f5638


 “Some Sentence”


 


Cool, huh? I’m using the C++ sintax from Windbg.


 


 


 


I also know the address of the DoSomething() method:


 


004114f0 MagicPointer!DoSomething (struct stData *)


 


Now, let’s try the same process but without using symbols.


I reloaded the application and broke into the debugger when the _getch() was called.


 


 


As you can see below, the approaches above didn’t work because I cannot use the variable names anymore.


 


 


 


How can I overcome it? Once I know the offsets, since I can see the source code and private symbols, I can create a magic pointer.


 


First, I put a breakpoint when the function is about to return. At this point the data will be already assigned to the struct.


 


 


 


Then I dump the Unicode string, but this time using pointers and offsets, like:


 


bp 00411560          ß Break point when the method is about to end.


 


0:000> kvn


 # ChildEBP RetAddr  Args to Child             


00 0017fe3c 00411478 008d5600 00000000 00000000 MagicPointer!DoSomething+0x70 (FPO: [Non-Fpo]) (CONV: cdecl) [c:\development\my tools\personal blog\article #17\magicpointer\magicpointer\magicpointer.cpp @ 40]


01 0017ff48 00411bd6 00000001 008d1188 008d1278 MagicPointer!wmain+0x88 (FPO: [Non-Fpo]) (CONV: cdecl) [c:\development\my tools\personal blog\article #17\magicpointer\magicpointer\magicpointer.cpp @ 24]


02 0017ff98 00411a1d 0017ffac 776119f1 7efde000 MagicPointer!__tmainCRTStartup+0x1a6


03 0017ffa0 776119f1 7efde000 0017ffec 77c2d109 MagicPointer!wmainCRTStartup+0xd


04 0017ffac 77c2d109 7efde000 00178231 00000000 kernel32!BaseThreadInitThunk+0xe


05 0017ffec 00000000 0041108c 7efde000 00000000 ntdll!_RtlUserThreadStart+0x23


 


008d5600 is our struct.


 


Then I use:


 


0:000> du 008d5600 + 0x38


008d5638  “Some Sentence”


 


Here it is!


Even without using symbols for the MagicPointer application I can get the information because I know the offsets.


 


Now, what happens when we change the struct? It might break our magic pointer.


Let’s see…


 


Remove the comment from the commented field nCode2 and rebuild the application.


Reload Windbg and, without using symbols for the MagicPointer application, use the breakpoint above and the same instructions.


 


bp 00411560


kvn


 


Then use:


0:000> kvn


 # ChildEBP RetAddr  Args to Child             


WARNING: Stack unwind information not available. Following frames may be wrong.


00 0017fe3c 00411478 00365600 00000000 00000000 MagicPointer+0x11560


01 0017ff48 00411bd6 00000001 00361188 00361278 MagicPointer+0x11478


02 0017ff98 00411a1d 0017ffac 776119f1 7efde000 MagicPointer+0x11bd6


03 0017ffa0 776119f1 7efde000 0017ffec 77c2d109 MagicPointer+0x11a1d


04 0017ffac 77c2d109 7efde000 00177837 00000000 kernel32!BaseThreadInitThunk+0xe


05 0017ffec 00000000 0041108c 7efde000 00000000 ntdll!_RtlUserThreadStart+0x23


 


du 00365600 + 0x38


 


What happened?


 



 


Now we got garbage because the field szSomeSentence is not at the offset 0x38 anymore.


However, if you use symbols and the instructions I used before referring to variable names, it works without problems!


 


So we need to change our offset. Of course, as we have access to the source code and symbols, we can do that very easily and create an instruction that again dumps the szSomeSentence value when not using symbols. Actually we could get the same information by reverse engineering the application if you don’t have access to private symbols or source code. I’m not considering this approach here. We don’t need to do that if we have or had access to the source code.


 


Ok, so now we know there’s a new field. It’s an integer, so it uses a double word, 4 bytes on Win32.


What do we need to do? Shift our offset.


 


The new one is:


 


du <address> + 0x3c


 


Let’s try it…


 



 


It worked!


 


Of course, this is a very simple example. Usually we have classes that have properties that are pointers to structs and so on. The expressions become more complex although the principle is the same.


Actually, it sounds complex but it’s simple.


 

Comments (8)

  1. newsoft says:

    Men, you mean some people are trying to use WinDbg without any clue about structures and offsets ???

  2. Yes, it happens because not all Windbg users have programming knowledge. It easy to copy and paste a magic pointer from a blog and use it, however, it requires some basic programming knowledge to understand what’s going on under the hood, then everything makes sense.

  3. .rip says:

    Hello, Roberto

    How to set breakpoint on access to memory allowed for static variable using WinDbg ?

    WinDbg is attached to debugging process and SOS extension is loaded. Next, i execute command "!dumpclass" for getting values for static variables in class:

    0:000> !dumpclass 0x02ec4ae4

    Class Name : MyClass

    mdToken : 0x02000004 (c:windowsmicrosoft.netframeworkv1.1.4322temporary asp.net filesxxx63f6fcb8a907abcbassemblydl22f7d80d4712e8cae_ce72c501xxx.dll)

    Parent Class : 0x79b92ee4

    ClassLoader : 0x02949648

    Method Table : 0x02eb49d0

    Vtable Slots : 0x4

    Total Method Slots : 0xc

    Class Attributes : 0x100001 :

    Flags : 0x3000801

    NumInstanceFields: 0x0

    NumStaticFields: 0x2

    ThreadStaticOffset: 0x0

    ThreadStaticsSize: 0x0

    ContextStaticOffset: 0x0

    ContextStaticsSize: 0x0

    FieldDesc*: 0x02eb4924

           MT      Field     Offset                 Type       Attr      Value Name

    0x02eb49d0 0x4000001        0                CLASS     shared   static m_Tracer

       >> Domain:Value 0x0013f7a8:0x10338b50 <<

    0x02eb49d0 0x4000002      0x4       System.Boolean     shared   static m_bInited

       >> Domain:Value 0x0013f7a8:1 <<

    I wanted to set breakpoint "ba w" on memory address of static variable m_bInited, but i don’t know how find variable’s address. Can you help me?

  4. Hi,

    I think I know how to help you here. If you want to get the address from static variables you may use:

    !DumpHeap -mt 0x02eb49d0 <– MethodTable from m_bInited above

    If you use !DumpObject with the addresses you get you’ll see they correspond to the right field.

    I’m not sure if you are familiar with this link, but it’s helpful. It has all SOS commands:

    http://msdn2.microsoft.com/en-us/library/bb190764(vs.80).aspx

    Thanks

  5. .rip says:

    Hello, Roberto

    Thanks for answer… Unfortunately, I have tried many methods that getting addresses of static variables (methods specified in article http://msdn.microsoft.com/msdnmag/issues/05/05/JITCompiler/default.aspx). I use "dumpheap" when i want getting objects from the heaps, but static variable (ex. m_bInited) shouldn’t be stored in the heaps.

    Below i placed the output of command "dumpheap":

    0:000> !dumpheap -mt 0x02eb49d0

    ——————————

    Heap 0

      Address         MT     Size  Gen

    total 0 objects

    ——————————

    Heap 1

      Address         MT     Size  Gen

    total 0 objects

    ——————————

    Heap 2

      Address         MT     Size  Gen

    total 0 objects

    ——————————

    Heap 3

      Address         MT     Size  Gen

    total 0 objects

    ——————————

    total 0 objects

    :-(  but somehow "dumpclass" command takes static variables values in fact

  6. Hello .rip,

    I have an idea. Let me show you a real situation.

    I’m debugging one of my tools: Netwiz.

    2.0.50727.832 retail

    Workstation mode

    Stack:

    OS Thread Id: 0x14a4 (0)

    ESP       EIP    

    0012f314 7c90eb94 [InlinedCallFrame: 0012f314] System.Windows.Forms.UnsafeNativeMethods.WaitMessage()

    0012f310 7b087438 System.Windows.Forms.Application+ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Int32, Int32, Int32)

    0012f3b0 7b086e95 System.Windows.Forms.Application+ThreadContext.RunMessageLoopInner(Int32, System.Windows.Forms.ApplicationContext)

    0012f41c 7b086cd3 System.Windows.Forms.Application+ThreadContext.RunMessageLoop(Int32, System.Windows.Forms.ApplicationContext)

    0012f44c 7b0665fe System.Windows.Forms.Application.Run(System.Windows.Forms.Form)

    0012f45c 024601f9 NetWiz.Program.Main()

    0012f69c 79e7be1b [GCFrame: 0012f69c]

    I chose a method call.

    0:004> !name2ee *!System.Windows.Forms.Application.Run

    Module: 790c2000 (mscorlib.dll)

    ————————————–

    Module: 01f72380 (sortkey.nlp)

    ————————————–

    Module: 01f72010 (sorttbls.nlp)

    ————————————–

    Module: 01f52c24 (NetWiz.exe)

    ————————————–

    Module: 7a71e000 (System.dll)

    ————————————–

    Module: 7b444000 (System.Windows.Forms.dll)

    Token: 0x060012ca

    MethodDesc: 7b5ab910

    Name: System.Windows.Forms.Application.Run()

    JITTED Code Address: 7b0eed5c

    ———————–

    Token: 0x060012cb

    MethodDesc: 7b4b36e0

    Name: System.Windows.Forms.Application.Run(System.Windows.Forms.Form)

    JITTED Code Address: 7b0665d0

    ———————–

    Token: 0x060012cc

    MethodDesc: 7b5ab918

    Name: System.Windows.Forms.Application.Run(System.Windows.Forms.ApplicationContext)

    JITTED Code Address: 7b0eed94

    ————————————–

    Module: 7ae72000 (System.Drawing.dll)

    ————————————–

    Module: 648ec000 (System.Configuration.dll)

    ————————————–

    Module: 639ea000 (System.Xml.dll)

    I chose one of the MethodDesc above:

    0:000> !dumpmd 7b4b36e0

    Method Name: System.Windows.Forms.Application.Run(System.Windows.Forms.Form)

    Class: 7b46f078

    MethodTable: 7b46f0dc

    mdToken: 060012cb

    Module: 7b444000

    IsJitted: yes

    m_CodeOrIL: 7b0665d0

    0:004> !DumpClass 7b46f078

    Class Name: System.Windows.Forms.Application

    mdToken: 020001db (C:WINDOWSassemblyGAC_MSILSystem.Windows.Forms2.0.0.0__b77a5c561934e089System.Windows.Forms.dll)

    Parent Class: 790f8a18

    Module: 7b444000

    Method Table: 7b46f0dc

    Vtable Slots: 4

    Total Method Slots: 5f

    Class Attributes: 100101  

    NumInstanceFields: 0

    NumStaticFields: 12  <– Great 0n12 Static Fields.

         MT    Field   Offset                 Type VT     Attr    Value Name

    7a74da98  4001014      144 ….EventHandlerList  0   static 00000000 eventHandlers

    790f9244  4001015      148        System.String  0   static 00000000 startupPath

    790f9244  4001016      14c        System.String  0   static 00000000 executablePath

    790f8a7c  4001017      150        System.Object  0   static 00000000 appFileVersion

    790ffe7c  4001018      154          System.Type  0   static 00000000 mainType

    790f9244  4001019      158        System.String  0   static 00000000 companyName

    790f9244  400101a      15c        System.String  0   static 00000000 productName

    790f9244  400101b      160        System.String  0   static 00000000 productVersion

    790f9244  400101c      164        System.String  0   static 00000000 safeTopLevelCaptionSuffix

    79103c00  400101d      964       System.Boolean  1   static        1 useVisualStyles

    7b477e54  400101e      168 …ms.FormCollection  0   static 0290a4c4 forms

    790f8a7c  400101f      16c        System.Object  0   static 028c2af4 internalSyncObject

    79103c00  4001020      968       System.Boolean  1   static        0 useWaitCursor

    79103c00  4001021      96c       System.Boolean  1   static        0 useEverettThreadAffinity

    79103c00  4001022      970       System.Boolean  1   static        0 checkedThreadAffinity

    79103c00  4001023      974       System.Boolean  1   static        0 exiting

    790f8a7c  4001024      170        System.Object  0   static 028c2b00 EVENT_APPLICATIONEXIT

    790f8a7c  4001025      174        System.Object  0   static 028c2b0c EVENT_THREADEXIT

    You can see the class has 12 static fields and some of them are variables.

    Now I’m going to select the variable “exiting” that is System.Boolean:

    79103c00  4001023      974       System.Boolean  1   static        0 exiting

    0:004> !DumpHeap -mt 79103c00  

    Loading the heap objects into our cache.

    Address       MT     Size

    028d264c 79103c00       12    2 System.Boolean

    028d3478 79103c00       12    2 System.Boolean

    028d34f8 79103c00       12    2 System.Boolean

    I think when you tried to setup the breakpoints the objects were never used yet.

    Let me prove my theory. I restarted NetWiz and instead of clicking Next a few times I just broke into the debugger after loading the application:

    0:003> !DumpClass 7b46f078

    Class Name: System.Windows.Forms.Application

    mdToken: 020001db (C:WINDOWSassemblyGAC_MSILSystem.Windows.Forms2.0.0.0__b77a5c561934e089System.Windows.Forms.dll)

    Parent Class: 790f8a18

    Module: 7b444000

    Method Table: 7b46f0dc

    Vtable Slots: 4

    Total Method Slots: 5f

    Class Attributes: 100101  

    NumInstanceFields: 0

    NumStaticFields: 12

         MT    Field   Offset                 Type VT     Attr    Value Name

    7a74da98  4001014      144 ….EventHandlerList  0   static 00000000 eventHandlers

    790f9244  4001015      148        System.String  0   static 00000000 startupPath

    790f9244  4001016      14c        System.String  0   static 00000000 executablePath

    790f8a7c  4001017      150        System.Object  0   static 00000000 appFileVersion

    790ffe7c  4001018      154          System.Type  0   static 00000000 mainType

    790f9244  4001019      158        System.String  0   static 00000000 companyName

    790f9244  400101a      15c        System.String  0   static 00000000 productName

    790f9244  400101b      160        System.String  0   static 00000000 productVersion

    790f9244  400101c      164        System.String  0   static 00000000 safeTopLevelCaptionSuffix

    79103c00  400101d      964       System.Boolean  1   static        0 useVisualStyles

    7b477e54  400101e      168 …ms.FormCollection  0   static 00000000 forms

    790f8a7c  400101f      16c        System.Object  0   static 028ff260 internalSyncObject

    79103c00  4001020      968       System.Boolean  1   static        0 useWaitCursor

    79103c00  4001021      96c       System.Boolean  1   static        0 useEverettThreadAffinity

    79103c00  4001022      970       System.Boolean  1   static        0 checkedThreadAffinity

    79103c00  4001023      974       System.Boolean  1   static        0 exiting

    790f8a7c  4001024      170        System.Object  0   static 028ff26c EVENT_APPLICATIONEXIT

    790f8a7c  4001025      174        System.Object  0   static 028ff278 EVENT_THREADEXIT

    And…

    0:003> !DumpHeap -mt 79103c00  

    Loading the heap objects into our cache.

    Address       MT     Size

    Nothing on the managed heap!

    Every static variable is stored on the managed heap, regardless of whether it is declared within a reference type or a value type.

    They are like strings, always stored on the heap.

    But now I use the following breakpoint:

    ba r4 79103c00    <– Stops on read or write.

    Then I press ‘g’ and pass control to the application. Just after that I get:

    ModLoad: 605d0000 605d9000   C:WINDOWSsystem32mslbui.dll

    ModLoad: 77120000 771ab000   C:WINDOWSsystem32OLEAUT32.DLL

    ModLoad: 64890000 6498a000   C:WINDOWSassemblyNativeImages_v2.0.50727_32System.Configurationc86724b99cb7eb7830f5c29bf2e5133eSystem.Configuration.ni.dll

    ModLoad: 637a0000 63d02000   C:WINDOWSassemblyNativeImages_v2.0.50727_32System.Xmld79992ddeba22e1133e9d956346ac52aSystem.Xml.ni.dll

    Breakpoint 0 hit

    eax=c377e1ff ebx=00000000 ecx=79103c00 edx=002b3418 esi=0012d598 edi=002b3418

    eip=79e8575a esp=0012d50c ebp=0012d5b0 iopl=0         nv up ei pl zr na pe nc

    cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246

    mscorwks!TypeHandle::GetManagedClassObject+0x15:

    79e8575a 0f855e122800    jne     mscorwks!TypeHandle::GetManagedClassObject+0x47 (7a1069be) [br=0]

    Conclusion: I think the only reason you don’t see the static variables on the managed heap is because the method using it was not Jitted yet so there’s no instance of the object.

    It’s easy to prove that. You need to use a breakpoint after the moment the object was created then verify the heap and try your ‘ba’ breakpoint.

    Please, let me know if it helped.

    Thanks

  7. I really like using C/C++ expressions from WinDbg. It’s a natural way to extract information from C and

  8. I really like using C/C++ expressions from WinDbg. It’s a natural way to extract information from C and