Some reasons not to do anything scary in your DllMain


As everybody knows by now, you’re not supposed to do anything even remotely interesting in your DllMain function. Oleg Lvovitch has written two very good articles about this, one about how things work, and one about what goes wrong when they don’t work.

Here’s another reason not to do anything remotely interesting in your DllMain: It’s common to load a library without actual intent to invoke its full functionality. For example, somebody might load your library like this:

// error checking deleted for expository purposes
hinst = LoadLibrary(you);
hicon = LoadIcon(you, MAKEINTRESOURCE(5));
FreeLibrary(hinst);

This code just wants your icon. It would be very surprised (and perhaps even upset) if your DLL did something heavy like starting up a timer or a thread.

(Yes, this could be avoided by using LoadLibraryEx and LOAD_LIBRARY_AS_DATAFILE, but that’s not my point.)

Another case where your library gets loaded even though no code is going to be run is when it gets tugged along as a dependency for some other DLL. Suppose “middle” is the name of some intermediate DLL that is linked to your DLL.

hinst = LoadLibrary(middle);
pfn = GetProcAddress(hinst, "SomeFunction");
pfn(...);
FreeLibrary(hinst);

When “middle” is loaded, your DLL will get loaded and initialized, too. So your initialization runs even if “SomeFunction” doesn’t use your DLL.

This “intermediate DLL loaded for a brief time” scenario is actually quite common. For example, if somebody does “Regsvr32 middle.dll”, that will load the middle DLL to call its DllRegisterServer function, which typically doesn’t do much other than install some registry keys. It almost certainly doesn’t call into your helper DLL.

Another example is the opening of the Control Panel folder. The Control Panel folder loads every *.cpl file so it can call its CplApplet function to determine what icon to display. Again, this typically will not call into your helper DLL.

And under no circumstances should you create any objects with thread affinity in your DLL_PROCESS_ATTACH handler. You have no control over which thread will send the DLL_PROCESS_ATTACH message, nor which thread will send the DLL_PROCESS_DETACH message. The thread that sends the DLL_PROCESS_ATTACH message might terminate immediately after it loads your DLL. Any object with thread-affinity will then stop working since its owner thread is gone.

And even if that thread survives, there is no guarantee that the thread that calls FreeLibrary is the same one that called LoadLibrary. So you can’t clean up those objects with thread affinity in DLL_PROCESS_DETACH since you’re on the wrong thread.

And absolutely under no circumstances should you be doing anything as crazy as creating a window inside your DLL_PROCESS_ATTACH. In addition to the thread affinity issues, there’s the problem of global hooks. Hooks running inside the loader lock are a recipe for disaster. Don’t be surprised if your machine deadlocks.

Even more examples to come tomorrow.

Comments (24)
  1. R says:

    I notice the errors themselves weren’t checked for expository purposes :)

    // error checking deleted for expository purposes

    hinst = LoadLibrary(you);

    hicon = LoadIcon(you, MAKEINTRESOURCE(5));

    FreeLibrary(hinst);

  2. Shane King says:

    Sometimes doing these kind of "immoral" things is the only way though. Several times I’ve had to do ill advised things in DllMain because I absolutely need to ensure the dll doesn’t load if some condition isn’t met. It seems rather unfortunate that the only mechanism provided to do this shouldn’t (in theory) be used to do it.

  3. Dan Maas says:

    Is this "loader deadlock" responsible for the phenomenon where Windows gets into a state where you can’t start any applications? (double-click and they just do nothing)

  4. Mike says:

    Shane: Can you provide examples? I’d be interested in knowing any loader problems that can’t be worked around.

    Dan: On Windows NT, the loader deadlock issue is local to the process. Each process has its own loader lock.

  5. Peter Torr says:

    Dan, you have probably run out of desktop heap. Happens with 30-ish IE windows open (or a similar number of other apps):

    http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q126/9/62.ASP&NoWebContent=1

  6. Larry Osterman says:

    Dan: Yes, the loader lock IS responsible for some of those situations. Especially on Win9x.

    And even more to the point, it turns out that many of these situations are undebuggable – the problem is that the debuggers can’t break into the app because to break into the app, they need to create a thread, which means that the loader lock needs to be taken (to call the DllMain in each DLL to tell them that the thread was created), but that hangs because someone has the loader lock held and hasn’t released it.

    The most heinous example of this is DllMain functions that do:

    DllMain(<whatever)>)

    {

    if (ulReason == DLL_PROCESS_ATTACH)

    {

    CreateThread(ThreadToInitializeStuff);

    WaitForSingleObject(EventThatsSetWhenThreadRuns);

    }

    :

    :

    }

    This is a recipe for deadlocks, unfortunately there’s way too much code that does this.

    One nasty problem is that this SOMETIMES works – not always, and not on all versions of Windows.

  7. Shane King says:

    One scenario is where I’m loading the dll as a CLR profiler, which allows me to modify .NET code on the fly for tracing and debugging purposes.

    The profiler needs to have the chance to load on the startup of every .NET program, because you can’t control the environment variables that cause the CLR to choose to laod it for a dllhost based process. Therefore, you must modify the global environment variables to cause it to always load.

    The problem is, for most processes, I don’t want it to be loaded. So I have to deliberately cause a failure result from DllMain to cause it to not be loaded and held in memory by the CLR. This is being determined via the registry, which is apparently a no-no in DllMain.

    I’d love for there to be another way (particuarly since returning failure from DllMain causes nasty event log messages about not being able to load the profiler), but if you return failure from the initialization function of the profiler, it holds it in memory anyway (which is unnaceptable, as it makes replacing the dll with a new version needlessly difficult, as you’ll need to shut down every process in which the CLR is active. This is particuarly a pain when developing the component, as Visual Studio .NET contains managed code).

    Probably not the most common case in the world, but it’s one where the DllMain approach seems to work well. Since it’s basically used for debugging problems on production type machines where installing a real debugger is unacceptable, I don’t particuarly care if a future version of windows breaks it. It’s not like it’s going to hurt anyone.

  8. Pavel Lebedinsky says:

    it turns out that many of these situations are undebuggable – the problem is that the debuggers can’t break into the app because to break into the app, they need to create a thread…

    This was definitely a problem with VC6 (I don’t know if it was fixed in later versions). Fortunately, windbg/cdb can handle this just fine – they eventually time out and attach "non-invasively" (or you could use -pv option to force non-invasive attach to begin with).

  9. Doug says:

    OCXs create this problem. VB is quite willing to unload an OCX while it is still running. So, the DLL’s get unloaded, but the process is not in termination.

    It is a great way for a thread to wake up and land in code space that is no longer loaded.

    You can make a self reference to the dll so that it won’t unload, but that can create other problems.

    Personally, I think the bug is unloading of a DLL/OCX during runtime, but the VB development environment is quite willing to do it. If Microsoft does it, it must be correct…. grin.

  10. Mr. Chen, speaking of the MSDN page which Mr. Chen linked to, http://msdn.microsoft.com/library/en-us/dllproc/base/dllmain.asp

    Here’s another example where MSDN has recursion instead of useful information: "DllMain is a placeholder for the library-defined function name. You must specify the actual name you use when you build your DLL. For more information, see the documentation included with your development tools."

    I don’t understand "DllMain is a placeholder for the library-defined function name" because whenever I’ve seen DllMain the name has always been DllMain. Therefore I tried to obey "For more information, see the documentation included with your development tools."

    Of course I first saw those instructions in documentation that came on CD instead of seeing them on Microsoft’s web site. Guess what documentation is included with my development tools.

  11. Pavel Lebedinsky says:

    I don’t understand "DllMain is a placeholder for the library-defined function name" because whenever I’ve seen DllMain the name has always been DllMain.

    Typically, the real entry point is provided by CRT and is called something like DllMainCRTStartup. This function is the "real" "DllMain" as far as the loader is concerned. It performs CRT-specific initialization then calls your DllMain.

    This is by the way the reason why global C++ constructors have the same limitations as DllMain.

  12. Mr. Lebedinsky, thank you for your explanation of DllMainCRTStartup but I’m still confused by MSDN. I think the MSDN page on DllMain is intended to instruct a DLL programmer on how to write a DllMain in VC++. I don’t think that MSDN page is intended to instruct a CRT programmer on how to make a CRT support the writing of DLLs in VC++.

    For comparison, if MSDN has a page on coding a main() (I’m neglecting to check if it does), I don’t think it instructs CRT programmers on how to initialize the CRT and call the C program’s main().

    (Meanwhile the recursion in that MSDN page is still there and it’s about to overflow my stack.)

  13. Mike Dimmick says:

    When the DLL is initialised, the loader jumps to the address in the AddressOfEntryPoint member of the IMAGE_OPTIONAL_HEADER for the file. With MS tools, you can specify this with the linker’s /ENTRY option.

    If you don’t specify /ENTRY, the MS linker defaults to _DllMainCRTStartup if you’ve also specified /DLL. This function (as Raymond alludes to on the other thread) has an explicit reference to the function name DllMain. It’s always statically linked to your binary, even if you’re using the DLL version of the CRT (it would be a chicken/egg situation otherwise). The CRT has a special weak-reference to an internal do-nothing DllMain so that your DLL still links even if you don’t provide a DllMain of your own.

    MSDN does have a page on WinMain, although it doesn’t cover wWinMain. wWinMain is a fake provided by the CRT which uses GetCommandLine to retrieve the Unicode command line and pass it as the lpCmdLine parameter. main and wmain are also wrapped up by corresponding CRTStartup functions in the CRT, to present a WinMain-compatible entry point for the loader.

    Indeed, all of the [w][Win]{M|m}ainCRTStartup functions share the same code, implemented in crt0.c (which you can read if you’ve installed the CRT source code).

    If you have to specify your own entry point but still need the CRT (maybe you need to initialise something before the CRT starts up), call _cinit to initialise the CRT.

    A peculiarity of Windows CE is that it has no wWinMain; instead, WinMain’s lpCmdLine parameter is a LPWSTR. CE has no need for ANSI backward compatibility, of course. For CE, the CRT is part of the core system DLL, coredll.dll, which also includes all of the (supported) system entry points from the desktop’s kernel32.dll, advapi32.dll, user32.dll, gdi32.dll and some others. The xxxCRTStartup functions are implemented in corelibc.lib.

  14. 1/30/2004 7:09 AM Mike Dimmick:

    Thank you for the detailed explanation of DllMain and [w]WinMain. When I have a free hour or two I might indeed look at the source of crt0.c. But actually the part of your posting I found most important was this:

    > This function […] has an explicit

    > reference to the function name DllMain.

    So my function still really has to be named DllMain(), and the MSDN page about DllMain might benefit by deleting threee sentences.

    Now for one tangent.

    > CE has no need for ANSI backward

    > compatibility, of course.

    For one application I accomplished that happy result by writing "xFExFF" at the beginning of every text file. Then Pocket Word was able to display the Unicode contents of the text files. Otherwise I would have had to convert all strings from Unicode to Shift-JIS (ANSI code page 932) before writing them to text files.

  15. Raymond Chen says:

    The point is that from the Platform SDK’s point of view, the function name can be anything, and it’s up to the language runtime provider to decide what to call it.

    It so happens that the VC people decided that the name of the function is… DllMain! This is something that should be called out in the VC docs; the Platform SDK is compiler-agnostic. For example, if you write a DLL in assembly language (which I have done), then your DLL entry point can indeed be called anything you want, as long as you tell the linker.

    Arguably the VC people should have called the entry point function something like CRTDllMain. But then again, most people don’t understand the difference between the platform and the runtime. (That’s part of the problem we’re having here. The Platform SDK says the name can be anything, but the runtime says the name must be DllMain.)

  16. 2/1/2004 5:07 PM Raymond Chen:

    > Arguably the VC people should have called

    > the entry point function something like

    > CRTDllMain.

    _DllMainCRTStartup is something like CRTDllMain, no problem for me on this point.

    > most people don’t understand the difference

    > between the platform and the runtime.

    Bingo. If Microsoft’s development group in its internal operations finds it convenient to architect these separately, no problem. But why should customers have to distinguish them? From the point of view of ordinary programmers, a function call results in calling a function in some library which usually result in system calls somewhere down the line. There are reasons for ordinary programmers to discover that MFC is separate from the API, but why should we have to know or care which parts of the API are which?

    If Visual Studio could still be used for programming for other than Microsoft OSes then I could see your point. (Yeah I see comments in some header files about targeting Macs, but the Visual Studio IDE doesn’t provide that target.)

  17. Raymond Chen says:

    The distinction between the Platform and the runtime is critical for people programming in languages other than C/C++ or for people using a compiler different from Visual Studio. If the Platform SDK blurred the distinction, then people using, say, the Borland compiler, would complain furiously that "Microsoft is playing unfair, writing Platform documentation that works only with its own compiler", and they would have a valid point.

  18. 2/2/2004 7:23 AM Raymond Chen:

    > The distinction between the Platform and the

    > runtime is critical for people programming

    > in languages other than C/C++

    Since there’s no Platform SDK for VB, I can see that VB programmers have to be aware that they only have the Runtime. But the VB Runtime is already different from the VC++ Runtime. So I still don’t understand why the distinction is critical.

    > or for people using a compiler different

    > from Visual Studio.

    Do you mean that Borland includes its own C++ Runtime? OK, if the difference between Platform and Runtime is critical for people who use a Runtime different from Visual Studio, then I understand. To achieve this understanding, it is critical to observe the distinction between the Compiler and the Runtime :-)

    (From the point of view of the standard, it isn’t critical to observe such distinctions. An implementation includes a preprocessor and compiler and library and everything. There is no guarantee that you can mix a compiler and half of a library from one implementation with the other half of a library from another implementation and come up with anything usable. But the standard isn’t the most relevant thing here.)

  19. Raymond Chen says:

    People programming in VB or C# often will use a p/invoke-style mechanism to call OS functions, so they need to know whether a particular function is OS (provided with Windows) or whether it’s runtime (doesn’t come with Windows; comes with the corresponding language).

    For example, you can’t p/invoke std::wstring.

    And yes, different compiler vendors have different runtime libraries. From a Standards point of view, the world "beneath" the program is monolithic. This is a valid view for the Standard to take (since it’s not in the business of dictating how OSs interface with languages), but it’s not how the world actually works.

Comments are closed.