The NT DLL Loader: FreeLibrary()

Let’s review the loader’s modus operandi and derive the (once again simple!) rules for what the heck is going on. When someone calls FreeLibrary(hInstance), the loader walks the closure of the dependencies for the module/instance in question and if their refcounts are not maxxed out (“pinned”), they are decremented.  Any modules whose refcounts are now…

6

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy - wrap up

Hopefully the culmination of these cautionary tales is clear: you’re walking a very fine line when you attempt to reenter the loader from within a loader callout.  You’re (more!) subject to ordering and cycle issues, you can force initialization to proceed on to your dependencies even if you’re not ready for them to call in…

2

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy - step 4 - ramifications of questionable quality

Last time I alluded to the world of hurt you’re in when the loader is reentered during DLL_PROCESS_ATTACH and the initialization of another DLL failed. The state of the affairs is pretty derivable from the clues that I’ve left behind in the series. First, we know that the loader tracks entry into the initialization function. …

1

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy - step 3 - quality requirements

Now we’re loaded for bear!  We understand how PEs which are either launched via CreateProcess() or loaded via LoadLibrary() are the roots of directed cyclic graphs.  Each new graph is turned into a linear initialization-order list where nodes further from the root are initialized prior to nodes closer to the root.  Cycles in the graph…

7

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy - step 2 - GetProcAddress()

Last time we pondered what does LoadLibrary() do when called inside of a DLL_PROCESS_ATTACH callout.  The answer was pretty simple and predictable – the only nuance is that the initializers are not run before returning. Now place yourself in the position of mythical developer Weve Stoods who evidently did most of the loader development in…

3

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy - step 1 - LoadLibrary()

So what happens if you call back into the loader when you’re inside a loader callout (DllMain) for DLL_PROCESS_ATTACH? I’ll be addressing teardown (DLL_PROCESS_DETACH) after completing the DLL_PROCESS_ATTACH series. The first issue is: what about LoadLibrary()?  I’ll address GetProcAddress() and FreeLibrary() later. We already know how LoadLibrary() works.  It walks the dependency graph starting from…

7

The NT DLL Loader: reentrancy - play along at home!

Anyone care to hazard a guess about what happens if you have the following code in your DllMain()?  Ignore the leak and the lack of error checking; focus on the what the loader’s behavior has to be… BOOL WINAPI DllMain(HINSTANCE hInstance, DWORD fdwReason, LPVOID lpvReserved) {    switch (fdwReason) {    case DLL_PROCESS_ATTACH:        SOME_FUNCTION_PTR_T pfn = (SOME_FUNCTION_PTR_T)…

12

The NT DLL Loader: DLL callouts (DllMain) - DLL_PROCESS_ATTACH deadlocks

The Windows DLL loader (I wasn’t around then but I assume some of this even comes from the days of 16-bit Windows) has a feature where a DLL may have an “entry point”. If a DLL has an entry point, the loader calls into it on certain significant events.  These events have identifiers associated with them:…

14

The NT DLL loader: dynamic unloads

To recap our story from last time: The NT DLL loader starts from some PE (either the main EXE or the DLL which is passed in to the LoadLibrary() API), walks the graph of static imports rooted with that first PE.  You can think of the loader as then building a linear ordered list of…

5

The NT DLL Loader: basic operation

Let’s start simply and consider the mythical vertical application (a topic itself for another day).  I’m not going to walk through what a PE is or a DLL or an EXE; if you don’t know, follow the link or take a gander around MSDN. Let’s call it ccalc.exe (console calculator – no funky graphics, etc….

6