Answers to a customer’s questions about memory and DLLs

A customer had a few questions about memory and DLLs.

We have some questions that no one here can answer.

  1. When are DLLs actually loaded (mapped) into memory? Does it happen when the calling program starts, or does it happen when the DLL is actually called?
  2. Do DLLs have their own heap and code memory, or do they use those of the calling program?

Okay, let's see if we can take these questions apart.

First: When are DLLs actually loaded into memory?

Well, it depends on how you linked to them. If you didn't do anything special, then a module that links to another DLL will list the target DLL in its import table, and the target DLL will be loaded at the same time the module itself is loaded. If that module is the main module, then the target DLL will be loaded at process startup.

On the other hand, if you listed the DLL in your /DELAYLOAD option, then the DLL will be loaded the first time any code in your module calls a function in the target DLL.

Next question: Do DLLs have their own heap, or do they use those of the calling program?

It is up to the DLL whether it wants to create its own heap, or whether it wants to use an existing heap. In fact, a DLL doesn't need to be consistent in its decision. It could use its own heap for some things and an existing heap for other things.

If a DLL wants to interoperate with other DLLs, and it wants to be able to allocate and free memory across DLL boundaries, then the two DLLs need to agree on which heap the memory should be allocated from and freed to. One way of doing this is to choose an existing external heap and have both parties agree to use that. You see this, for example, with COM interfaces, which all agree to use the COM task allocator to allocate and free memory across COM object boundaries. Memory returned by a COM method must be allocated from the COM task allocator (usually by calling Co­Task­Mem­Alloc), and that memory must then be freed to the COM task allocator (usually by calling Co­Task­Mem­Free).¹

Another pattern that is often used is for a DLL to allocate memory, and then provide a custom free function that the other DLL must call in order to free the memory. The most commonly encountered example of this is the function Net­Api­Buffer­Free

Note that the above rules about agreeing on a heap to use apply only in the case where the memory is allocated by one DLL and freed by another. If the allocation and free happens both in the same DLL, then that DLL can use any policy it likes, as long as every allocation is matched with a compatible free. Internal objects could be allocated with malloc, as long as they are freed by free.

Note that the malloc/free pairing must be to the same instance of the C runtime library. If one DLL calls malloc and passes it to another DLL, and that other DLL calls free, then the two sides of the equation must be using the same C runtime library.

Next part of the question: Do DLLs have their own code memory, or do they use those of the calling program?

I'm not sure what this question is trying to ask. Certainly the code for a DLL comes from the DLL. What would be the point of a DLL that just used code from the calling program? The calling program wouldn't need the DLL at all. It already has the code!

So I'll assume that the question really is "Are DLLs loaded into the same address space as the calling program?" And the answer is yes, all the DLLs loaded by a process share the same address space.

There's another question the customer didn't ask, but which is closely related: Which C runtime does the DLL use?

The answer to this question is a bit circular. The DLL uses the C runtime that it chooses. When you create the DLL, you specify somewhere which C runtime you want to use, and how you want to use it. (You may not be aware that you even made this choice, because most project templates will make a default choice for you.)

One choice is to link the C runtime statically into your module. This means that the code in the C runtime is incorporated into your module. This becomes your private C runtime, and it will not be shared with anyone. This private C runtime initializes when your module is loaded.

Another choice is to link the C runtime dynamically into your module. This means that the code in the C runtime remains in a DLL which will be loaded either when your module is loaded (if linked statically) or when the first function in it is called (if linked with /DELAYLOAD).

In practice, nobody delay-loads the C runtime library.

If you choose this route, then the C runtime library will be shared with any other modules that use the same version of the same C runtime library. The C runtime DLL will be loaded when the first module that uses it is loaded.

If you use malloc or new, then you are calling the corresponding function in the C runtime library that your module has chosen. In the case where you statically linked the C runtime, this is definitely not the same instance of the C runtime that anybody else is using, so that memory can be freed only by your module. In the case where you dynamically linked the C runtime, then this might or might not be the same instance of the C runtime that another module is using. Only if the other module is indeed using the same version of the same C runtime library will it be able to free the memory with the corresponding free or delete function.

The last detail is something people often overlook. They will create two modules which do not share the same instance of the C runtime, either because one or both linked the C runtime statically, or because the two both linked to different versions of the C runtime. If you are in either of these cases, then you cannot share C runtime data structures between the two modules because the two modules are using different C runtimes. This means that you cannot pass a FILE* between modules, you cannot pass a std::string between modules, and you cannot even pass a file descriptor between modules, because even though file descriptors are integers, they are integers that are managed by the corresponding C runtime library.

This is why most coding guidelines specify that each module is responsible for allocating, managing, and freeing its own objects, rather than handing objects to other modules expecting the other module to be able to free them properly. The things you have to line up in order to get cross-module object memory management to work can be easily thrown out of alignment.²

Bonus chatter: Note that there is is a confusing overload of the word "static" when applied to DLL linkage. You can choose to link a library statically or dynamically, and if you choose dynamic linking, then your next choice is whether the target DLL is loaded statically or delayed (also called "dynamically"). Let me draw a table, because people like tables.

How was the library linked? How is the DLL loaded? When is the code loaded?
Statically N/A (there is no other DLL) When the module loads
Dynamically Statically When the module loads
Delayed/dynamic When the first function is called

¹ There are a few old interfaces that use HGLOBAL to share memory (such as STGMEDIUM), but those exist for historical reasons, not because they're a good idea.

² For example, you might decide to upgrade one module to a newer version of the compiler, in order to take advantage of a new feature, but choose to leave another module with an older version of the compiler because you haven't made any changes to it and don't want to take any risk that the new compiler will expose an issue. (Or because you're building a patch for your program, and you want to minimize the size of your patch by omitting files which didn't change.) Once you do that, you have a C runtime mismatch, and then scary things will happen if the two modules are not in agreement on how memory will be allocated and freed.

Comments (12)
  1. Joshua says:

    I once solved the cross-DLL memory alloc/free problem with an amusing solution. The policy was all such allocations are made using HeapAlloc(GetProcessHeap(), …) and are freed with HeapFree(GetProcessHeap(), …).

    There was a rumor that you could in fact call malloc in one C library and free the resulting memory block in a matching C library. The scary thing is I encountered two C libraries for which this was in fact true. One did malloc -> HeapAlloc(GetProcessHeap(), …) and the other had this weird artifact where free() located the heap by pointer arithmetic on the pointer being freed. You could easily depend on this behavior by accident and years later wonder why it doesn’t work anymore.

    1. kantos says:

      glib does that, particularly for files (g_open but no g_close), this has caused me innumerable problems because the GTK-WIN project doesn’t always keep up with the latest VC++. Fortunately there are other options for opening files with UTF-8 paths (boost.filesystem). But it is a very serious issue when dealing with libraries imported from *nix which assume a system wide allocator and syscalls in the C library.

    2. alegr1 says:

      In a debug build, this might not be a straight HeapAlloc anymore.

  2. richard says:

    > The DLL uses the C runtime that it chooses.

    there isn’t any choice if the dependent modules have to be abi compatible, no?

    [That was the DLL’s choice, to take a dependency on other DLLs that constrain its choice of C runtime. -Raymond]

  3. Cesar says:

    > The last detail is something people often overlook.

    Probably because on other systems, there is only one C runtime library.

    > and you cannot even pass a file descriptor between modules, because even though file descriptors are integers, they are integers that are managed by the corresponding C runtime library.

    This one is even worse: on Unix-based systems (nowadays, basically every system except Windows), file descriptors are managed by the kernel (they are more like handles), so you can pass them between modules even if you somehow are using more than one C runtime library in the same process. It’s easy to forget that, on Windows, they are not managed by the kernel.

    1. Joshua says:

      [Posst flickered in and out of existence as I tried to reply to it]

      1: The hacked descriptors in the windows C runtime library have essentially no use. The NET Core wisely avoids them and just uses handles outright. The Linux port of NET Core stuffs the descriptor in the handle variables because they are the true equivalent.

      2: My Linux system has I believe 5 C libraries (or 6 if libc5 is still installed but I don’t think it is). It is quite impossible to compile any program that crosslinks them however. What would happen if you tried is the linker would yelp as they are utterly incompatible no matter how many layers of indirection you went through. In effect they are different executable file formats.

      1. NTAuthority says:

        The latter, of course, merely because the dynamic linker on Linux is part of the C runtime library, and has its own executable parser.

      2. Beldantazar says:

        So what would happen on linux if someone replaced a .so file with a version compiled with a different libc?

        1. Joshua says:

          The loader would try to load the second libc and fail because the libc references symbols in another loader.

          You might succeed if you statically linked the .so file against the target libc’s .so file (this is a weird case with no analog in Windows; the full path to the .so file and its load address are specified in the binary), but if you went through all that work and figured out how to keep ASLR from hosing you you deserve what you get.

    2. Joker_vD says:

      Many Linux developers seem to be surprised that on Windows, the C runtime is not built into the system. I remember reading somewhat recently a blogpost that started basically with “Hey, we used jemalloc for out new hot library, and when we tried to build and use it on Windows, terrible things happened when free() was called”. Then it proceeded to discuss possible solutins — and it was clear from the text that it was a new and exciting problem for the author, which left a kind of strange impression on a Windows developer: after all, the problem (and all the discussed solutions) was already old when Delphi 3.0 was new.

  4. Jonathan Wilson says:

    I deal with this in one project I work on (a game engine) by having a common dll that provides a memory manager (and all the other bits link to it). In this case this is done not just because it ensures cross-boundary memory allocation works but it has other advantages (performance, memory leak detection, things like that)

  5. Martin Ba. _ says:

    ad footnote (2): “…decide to upgrade one module to a newer version of the compiler … Once you do that, you have a C runtime mismatch”

    Except when the new compiler is VC++2017 and the old one is VC++2015, because here MS kept binary compatibility. And of course within the update timeframe of one VS version, the runtime library also stays the same.

Comments are closed.

Skip to main content