How you might be loading a DLL during DLL_PROCESS_DETACH without even realizing it


As you are I'm sure aware, you shouldn't be doing much of anything in your DllMain function, but you have to watch out for cases where you end up doing them accidentally.

Some time ago, I was investigating a failure which was traced back to loading a DLL inside DLL_PROCESS_DETACH. Wait, what kind of insane person loads a DLL as part of shutting down? Shouldn't you be cleaning up stuff, not creating new stuff?

The following is not the actual code, but it captures the same spirit:

INFO *CachedInfo;

BOOL WINAPI DllMain(HINSTANCE hinst, DWORD dwReason, void *pvReserved)
{
  switch (dwReason) {
  ...
  case DLL_PROCESS_DETACH:
    ...
    CoTaskMemFree(CachedInfo);
    ...
  }
  return TRUE;
}

There is some global variable that contains a pointer to memory that was allocated by CoTaskMemAlloc. In this case, I made it a cache, but the details aren't important. When the DLL is detached from the process, we free the cached memory so we don't have a leak. Since it is okay to pass NULL to the CoTaskMemFree function (it simply returns without doing anything), the cleanup code works even if we never called a function that put a value into the cache.

Except that this code ended up loading a DLL. The reason is delay-loading.

The authors of this DLL sped up its load time by marking OLEAUT32.DLL as a delay-loaded DLL, which means that it doesn't get loaded until somebody calls a function in it.

And in fact, nobody called a function from OLEAUT32. Ever.

"Hooray!" you shout. "We avoided loading OLEAUT32 altogether." After all, the fastest code is code that doesn't run.

Except that it does run. Right there. In your DLL_PROCESS_DETACH handler.

Since nobody called a function from OLEAUT32, the call to CoTaskMemFree was the first call to OLEAUT32 and therefore caused it to be loaded. From inside a DLL_PROCESS_DETACH handler.

Delay-loading is one of those features that is very convenient and saves you a lot of typing (namely, writing those stub functions yourself), but you also have to understand what's going on so you don't use it incorrectly. (In this case, a superficially-redundant if (CachedInfo != NULL) test needs to be inserted.)

Comments (12)
  1. Tim Kay says:

    >In this case, a superficially-redundant if (CachedInfo != NULL) test needs to be inserted.

    And a comment explaining why it really isn’t redundant, so I don’t come along and ‘fix’ it.

  2. Koro says:

    Isn’t unsafe to call anything not from KERNEL32 in DllMain anyway? In this specific case, OLEAUT32 might have had its DLL_PROCESS_DETACH sent already and cleaned up all of its global state, leading to undefined behavior in CoTaskMemFree…

    [If this is a dynamic detach, then OLEAUT32 will still be around. If it’s a termination detach, then yes, OLEAUT32 may not be there any more. -Raymond]
  3. Yuhong Bao says:

    Not only it is bad in DllMain, even outside DllMain it is quite inefficient to load an entire DLL just for one function that does effectively nothing in the case.

  4. Gabe says:

    Of course not releasing the memory because it’s a process termination gets you a spurious memory leak warning from memory checkers.

  5. porter says:

    I thought CoTaskMemFree was in OLE32.DLL, not OLEAUT32.DLL.

    [You’re right. Fortunately it doesn’t affect the underlying story. -Raymond]
  6. Anonymous Coward says:

    But considering you can’t check whether your DLL is dynamically unloaded or unloaded on termination, why would you call a function in a DLL that might not be present anymore?

    And if you impose that it must always be dynamically unloaded for some reason, you can also impose that users should call a function in the DLL that performs clean-up, and make it illegal to unload the DLL without calling that function (for example by raising an exception).

    In other words, I see no excuse to call functions from outside Kernel32 from DllMain. But since you wrote about it, I must be wrong. Please enlighten me.

    [The lpReserved parameter tells you whether you’re being unloaded dynamically or as part of process termination. The clean-up function is a good pattern, but it has its subtleties (race conditions) and sometimes you have to work within an existing model that doesn’t provide for it (e.g. COM inproc servers).-Raymond]
  7. Daev says:

    If you are writing a DLL in plain old C (or C++), and you use the ANSI C Standard atexit(f) function to indicate that function f() should run during the process’s exit-time cleanup, then in some circumstances the C run-time library may call f() during DLL_PROCESS_DETACH for your DLL.

    This is "how you might be writing code that goes into DllMain() without even knowing it."  So apply the same restrictive guidelines to atexit() functions that apply to DllMain.

  8. dgt says:

    shouldn’t the code just check that this is a process detach and *don’t* release the memory? it is going to be gone anyway.

    [s/process detach/process termination detach/ and I agree with you. -Raymond]
  9. Ivo says:

    Does the same restriction (“you can’t do anything interesting in DllMain”) apply for plug-ins or shell extensions? By the time the DLL loads the host must be fully operational and things like user32.dll should be already initialized, right?

    [Initialization order is only one of many issues. Here’s another. -Raymond]
  10. Anonymous Coward says:

    Ah! Thanks Raymond, I was fooled by the name lpReserved.

    So, is it a bug to link statically to or call LoadLibrary on an inproc server, because they expect to be unloaded dynamically and may perform necessary clean-up that it cannot do on process termination, or should well-written inproc servers clean-up when the total object count drops to zero?

  11. Worf says:

    So what happens if you’re shutting down Windows and the DLL the app is unloading requires loading another dll?

    At least, I assume since the error is "DLL Initialization Failed because this window station is shutting down" means the DLL can’t be loaded…

  12. Dave Sawyer says:

    Seen this myself several times.  Here’s a real groaner version.  They really tried to avoid calling their shutdown if they hadn’t been initialized…

    if (XYZDLL_Initialized())

     XYZDLL_Terminate();

    When it was delay loaded, XYZ.DLL gets loaded to call XYZDll_Initialized().  That returns false, but the damage is done.

Comments are closed.