How do you prevent the linker from discarding a function you want to make available for debugging?


We saw some time ago that you can ask the Windows symbolic debugger engine to call a function directly from the debugger. To do this, of course, the function needs to exist.

But what if you want a function for the sole purpose of debugging? It never gets called from the main program, so the linker will declare the code dead and remove it.

One sledgehammer solution is to disable discarding of unused functions. This the global solution to a local problem, since you are now preventing the discard of any unused function, even though all you care about is one specific function.

If you are comfortable hard-coding function decorations for specific architectures, you can use the /INCLUDE directive.

#if defined(_X86_)
#define DecorateCdeclFunctionName(fn) "_" #fn
#elif defined(_AMD64_)
#define DecorateCdeclFunctionName(fn) #fn
#elif defined(_IA64_)
#define DecorateCdeclFunctionName(fn) "." #fn
#elif defined(_ALPHA_)
#define DecorateCdeclFunctionName(fn) #fn
#elif defined(_MIPS_)
#define DecorateCdeclFunctionName(fn) #fn
#elif defined(_PPC_)
#define DecorateCdeclFunctionName(fn) ".." #fn
#else
#error Unknown architecture - don't know how it decorates cdecl.
#endif
#pragma comment(linker, "/include:" DecoratedCdeclFunctionName(TestMe))
EXTERN_C void __cdecl TestMe(int x, int y)
{
    ...
}

If you are not comfortable with that (and I don't blame you), you can create a false reference to the debugging function that cannot be optimized out. You do this by passing a pointer to the debugging function to a helper function outside your module that doesn't do anything interesting. Since the helper function is not in your module, the compiler doesn't know that the helper function doesn't do anything, so it cannot optimize out the debugging function.

struct ForceFunctionToBeLinked
{
  ForceFunctionToBeLinked(const void *p) { SetLastError(PtrToInt(p)); }
};

ForceFunctionToBeLinked forceTestMe(TestMe);

The call to Set­Last­Error merely updates the thread's last-error code, but since this is not called at a time where anybody cares about the last-error code, it is has no meaningful effect. The compiler doesn't know that, though, so it has to generate the code, and that forces the function to be linked.

The nice thing about this technique is that the optimizer sees that this class has no data members, so no data gets generated into the module's data segment. The not-nice thing about this technique is that it is kind of opaque.

Comments (22)
  1. skSdnW says:

    I sometimes do "if (!GetCurrentThreadId()) DoDebugThing(whatever);". This has more run-time overhead but also allows you to selectively execute the function.

    If you only care about VC:

    EXTERN_C void __cdecl ThisSeemsToWorkInVC(int x, int y)

    {

    #pragma comment(linker, "/include:"__FUNCDNAME__)

    }

  2. Joshua says:

    I just export the function. EXE images can export functions. I'm told DLL images can refer to them but I have no idea how this works and as far as I know only Cygwin uses this so they may have hacked the loader.

  3. Ben Voigt says:

    @Joshua: DLLs importing from the any other module, including the executable image, is supported by the OS itself (and not a recently added feature either).  Nothing cygwin-specific there.

  4. Matt says:

    Eww. Don't send it to a function that sets state and has a meaningful type like SetLastError. It's a type error at best and an app-compat fail waiting to happen. One day sending the pointer to SetLastError will do something crazy you didn't expect (like a shim hooking SetLastError to watching for a WINERROR range will happen to fire because ASLR put your function somewhere where it overlaps with that error-code range and then your app will explode and install malware and take your credit-card information).

    At best pass it to something that doesn't change state and which expects a PVOID like Ntdll!EncodePointer, rather than something that statefully sets a meaningful non-PVOID parameter like SetLastError.

  5. poizan42 says:

    @Ben Voigt, I'm not sure about how ASLR fit into this, but exe images used to not be relocateable (unless forcing the linker to generate a relocateable image ofc.), in which case loading it into another process will fail if there is already somthing allocated at the base address.

  6. Joshua says:

    Hmmm. Matt is correct. Too bad NtDll!EncodePointer is undocumented.

  7. Gabe says:

    Oh, are we trying to figure out the best Win32 function that you can pass a pointer to while minimizing any side effects?

    I'm going to nominate GetProcessId(). It will almost certainly fail, causing the last error to be set to something, but at least it will be something fairly predictable. It takes a HANDLE, but it's no problem to convert a pointer to a HANDLE.

    [This is a bad idea, because it will generate a STATUS_INVALID_HANDLE exception. -Raymond]
  8. skSdnW says:

    @Gabe: Why gamble when you can use "IsBadReadPtr(myptr,0)"? kernel32!EncodePointer is documented if you want to use that.

    [IsBadReadPtr is a bad idea because it will trigger security warnings in code analysis tools. -Raymond]
  9. Joshua says:

    @Matt: Ah there we go that's kernel32!EncodePointer which is fine if you run on anything newer than Windows 2000.

  10. At least the not-nice thing can be resolved with comments.

  11. Anonymous says:

    Ignoring for a moment that many of these architectures (alpha, mips, ppc) are not currently supported by NT-based Windows (and ia64 is not supported in the latest version), why not combine some of those (undecorated) architectures with an || in the preprocessor directive?  This makes it much less verbose:

    #if defined(_X86_)

    #define DecorateCdeclFunctionName(fn) "_" #fn

    #elif defined(_AMD64_) || defined(_ALPHA_) || defined(_MIPS_)

    #define DecorateCdeclFunctionName(fn) #fn

    #elif defined(_IA64_)

    #define DecorateCdeclFunctionName(fn) "." #fn

    #elif defined(_PPC_)

    #define DecorateCdeclFunctionName(fn) ".." #fn

    #else

    #error Unknown architecture

    #endif

    You could make it even shorter by putting the "_", ".", or ".." into another variable, maybe with "prefix" in the name…  Seeing DecorateCdeclFunctionName(fn) so verbosely and so many times is off-putting.

    [It's less verbose, but it's harder to manage when new architectures are added. -Raymond]
  12. Alex says:

    @poizan42: this feature may be useful for in-process DLLs, too.

  13. Craven Weasel says:

    I can handle the syntactical ambiguities of C for the most part. But while glancing over your post I stared at the line "ForceFunctionToBeLinked forceTestMe(TestMe);" for about two minutes, trying to figure out how you could keep a function from being linked out simply by the return type, until it occurred to me that you are defining a variable, not a function.

    Regarding the opaque-ness of this technique, you could always define a macro to hide the variable declaration behind a meaningful name. Perhaps play a trick with __LINE__ to generate a sufficiently unique identifier.

  14. Ben Voigt says:

    @poizan42 That creates difficulty in loading an image compiled as a main application into a process that has another main application.  It presents no difficulty whatsoever for modules importing functions from the first non-relocatable image in their process, which is the topic being discussed.

    Imports don't necessarily require loading a DLL; they are perfectly happy binding to modules already present, including the application module.

  15. voo says:

    @Matt: You are right, but we can still use the call with a simple change, since: "Bit 29 is reserved for application-defined error codes; no system error code has this bit set"

    So just or the result with (1 << 28) and we're golden.

  16. killer{r} says:

    if (!&GetLastError) SetLastError(PtrToInt(p));

    would be less intrusive, if only GetLastError will not become intrinsic one day

  17. Killer{R} says:

    Also a bit surprising that

    volatile void *pTestMe = &TestMe;

    is not helpful – no TestMe and no p in resulting module.

  18. Sven2 says:

    Could also do it like this:

    if (GetLastError() == PtrToInt(p)) SetLastError(PtrToInt(p));

    It would never change the error code and even the call to SetLastError is very unlikely to happen.

  19. mikeb says:

    It seems to me that Joshua's idea of simply exporting the function would be the best choice – there's no 'opaqueness' about it.  You're explicitly declaring that you want to make the function callable from 'something else'.  In this case the debugger is that something else.

    Is there a downside to that technique that's causing people to continue to look for another method that doesn't have some oddball side-effect possibility?

    [The downside of exporting it is that it becomes externally visible, and now you have an undocumented API that you have to preserve for compatibility reasons. It also means that there are now two files you need to keep in sync (the DEF file and the implementation). And you can't put #ifdef in a DEF file, so now you have the debugging function availble even in the non-debug build. -Raymond]
  20. alegr1 says:

    The worse thing the linker does in the debug build is identical COMDAT elimination (ICE). This causes all little stub functions to be merged. Then you can't set a breakpoint in just one of them, and it will mess any stack trace and disassembly which might refer to such functions.

  21. James Curran says:

    Also, couldn't we reduce this to just

        ForceFunctionToBeLinked(TestMe);

    (Constructing, but not storing the object). This is even more obvious what it's doing (if less obvious how it doing it).

    [You have to put the function call inside a function, and you have to make sure that enclosing function is itself not optimized out by the linker, so you're back to the original problem. -Raymond]

Comments are closed.

Skip to main content