Canonical Function Structure

Here is a proposed canonical structure for functions/procedures.  Clearly in some cases some sections would not be present.  What’s important is to understand the basic phasing of the work to be done and the separation of tasks.   int foo(      char *StringIn, // “in” parameter      size_t BufferLength, // “in” parameter      char Buffer[],…

5

The NT DLL Loader: FreeLibrary()

Let’s review the loader’s modus operandi and derive the (once again simple!) rules for what the heck is going on. When someone calls FreeLibrary(hInstance), the loader walks the closure of the dependencies for the module/instance in question and if their refcounts are not maxxed out (“pinned”), they are decremented.  Any modules whose refcounts are now…

6

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy – wrap up

Hopefully the culmination of these cautionary tales is clear: you’re walking a very fine line when you attempt to reenter the loader from within a loader callout.  You’re (more!) subject to ordering and cycle issues, you can force initialization to proceed on to your dependencies even if you’re not ready for them to call in…

2

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy – step 3 – quality requirements

Now we’re loaded for bear!  We understand how PEs which are either launched via CreateProcess() or loaded via LoadLibrary() are the roots of directed cyclic graphs.  Each new graph is turned into a linear initialization-order list where nodes further from the root are initialized prior to nodes closer to the root.  Cycles in the graph…

7

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy – step 2 – GetProcAddress()

Last time we pondered what does LoadLibrary() do when called inside of a DLL_PROCESS_ATTACH callout.  The answer was pretty simple and predictable – the only nuance is that the initializers are not run before returning. Now place yourself in the position of mythical developer Weve Stoods who evidently did most of the loader development in…

3

The NT DLL Loader: DLL_PROCESS_ATTACH reentrancy – step 1 – LoadLibrary()

So what happens if you call back into the loader when you’re inside a loader callout (DllMain) for DLL_PROCESS_ATTACH? I’ll be addressing teardown (DLL_PROCESS_DETACH) after completing the DLL_PROCESS_ATTACH series. The first issue is: what about LoadLibrary()?  I’ll address GetProcAddress() and FreeLibrary() later. We already know how LoadLibrary() works.  It walks the dependency graph starting from…

7

The NT DLL Loader: reentrancy – play along at home!

Anyone care to hazard a guess about what happens if you have the following code in your DllMain()?  Ignore the leak and the lack of error checking; focus on the what the loader’s behavior has to be… BOOL WINAPI DllMain(HINSTANCE hInstance, DWORD fdwReason, LPVOID lpvReserved) {    switch (fdwReason) {    case DLL_PROCESS_ATTACH:        SOME_FUNCTION_PTR_T pfn = (SOME_FUNCTION_PTR_T)…

12

The NT DLL Loader: DLL callouts (DllMain) – DLL_PROCESS_ATTACH deadlocks

The Windows DLL loader (I wasn’t around then but I assume some of this even comes from the days of 16-bit Windows) has a feature where a DLL may have an “entry point”. If a DLL has an entry point, the loader calls into it on certain significant events.  These events have identifiers associated with them:…

14

The NT DLL loader: dynamic unloads

To recap our story from last time: The NT DLL loader starts from some PE (either the main EXE or the DLL which is passed in to the LoadLibrary() API), walks the graph of static imports rooted with that first PE.  You can think of the loader as then building a linear ordered list of…

5