What is DLL import binding?


Last time, we saw how hinting is used to speed up the resolving of imported functions. Today, we'll look at binding.

Recall that the module loader resolves imports by locating the function in the export table of the linked-to DLL and recording the results in the loaded module's table of imported function addresses so that code from the module can jump indirectly through the table and reach the target function.

One of the consequences of this basic idea is that the table of imported function addresses is written to at module load time. Writeable data in a module is stored in the form of copy-on-write pages. Copy-on-write pages are a form of computer optimism: "I'm going to assume that nobody writes to this page, so that I can share it among all copies of the DLL loaded into different processes" (assuming other conditions are met, not important to this discussion; don't make me bring back the nitpicker's corner). "In this way, I can conserve memory, leaving more memory available for other things." But once you write to the page, that assumption is proven false, and the memory manager needs to make a private copy of the page for your process. If two processes load your DLL, they each get their own copy of the memory once they write to it, and the opportunity to share the memory between the two DLLs is lost.

What is particularly sad is when the copy-on-write page is forced to be copied because two processes wrote to the pages, even if the processes wrote the same value. Since the two pages are now once again identical, they could in principle be shared again. (The memory manager doesn't do memcmps of every potentially-shared page each time you write to it, on the off chance that you happened to make two pages coincidentally identical. Once a copy-on-write page is written to, the memory manager makes the copy and says, "Oh well, it was good while it lasted.")

One of the cases where two processes both write to the page and write the same value is when they are resolving imports to the same DLL. In that case, the call to GetProcAddress will return the same value in both processes (assuming the target DLL is loaded at the same base address in both processes), and you are in the sad case where two processes dirty the page by writing the same value.

To make this sad case happy again, the module loader has an optimization to avoid writing to pages it doesn't have to: We pre-initialize the values in the table of imported function addresses to a prediction as to what the actual address of the function will be. Then we can have the module loader compare the return value of GetProcAddress against the predicted value, and if they are the same, it skips the write. In context diff format:

// error checking deleted since it's not relevant to the discussion

for (Index = 0; Index < NumberOfImportedFunctions; Index++) {
  FunctionPointer = GetProcAddress(hinst, ImportEntry[Index]);

- TableEntry[Index] = FunctionPointer;

+ if (TableEntry[Index] != FunctionPointer)
+   TableEntry[Index] = FunctionPointer;
}

But wait, we can optimize this even more. How about avoiding the entire loop? This saves us the trouble of having to call GetProcAddress in the first place.

There is an extra field in the import descriptor table entry called TimeDateStamp which records the timestamp of the DLL from which the precomputed function pointer values were obtained. Every DLL has a timestamp, recorded in the module header information. (The format of this timestamp is in seconds since January 1, 1970, commonly known as unix time format.) Before the module loader resolves imported functions, it compares the timestamp in the import descriptor table entry against the timestamp in the actual DLL that got loaded. If they match (and if the actual DLL was loaded at its preferred base address), then the module loader skips the loop entirely: All the precomputed values are correct.

That's the classical model for binding. There have been some changes since the original implementation, but they don't change the underlying principle: Precompute the answers and associate them with a key which lets you determine whether the information against which the values were precomputed matches the information that you actually have.

Binding therefore is a performance optimization to address both wall-clock running time (by reducing the amount of computation performed at module load time) and memory consumption (by reducing the number of copy-on-write pages actually written to).

Exercise: Why is the timestamp stored in the module header? Why not just use the actual file last-modified time?

Exercise: When you rebase a DLL, does it update the timestamp?

Comments (25)
  1. Random832 says:

    Because the actual file time may differ depending on whether the DLL has ever been stored on a FAT filesystem?

  2. Tom says:

    Random832 is correct: the creation/modification timestamps on the file can be changed by copying, or they can be inaccurate due to the filesystem on which the module is stored.

    With respect to rebasing, I would say it does modify the timestamp.  I believe the export records are stored in RVA format (relative to the load address), but the cached export address would need to be resolved (base+offset) in order for binding to work.  If rebasing didn’t update the timestamp, binding to the rebased DLL would result in undefined behavior.

    And even if I’m wrong about why, my answer still stands: I just rebased a DLL and it changed the timestamp.

    P.S.  I find it awesome how you threaten your readers  with the nitpicker’s corner.  :)

  3. David Walker says:

    Would the actual file write time may differ depending on your timezone versus the timezone of the server that’s holding the file?  

  4. Nathan_works says:

    The baseaddress is stored in the DLL itself, so if you re-base (static, not run-time), it would modify the file last modified time.

    Should it modify the internal timetamp ? Well, are the function pointers all relative to the base address, ya ? Thus the offsets shouldn’t change — they are the same, since only the base changed.. So maybe not..

  5. Someone You Know says:

    @Random832:

    For those of us who aren’t familiar with the ins and outs of filesystems, why does it matter if it’s been stored on a FAT32 filesystem?

  6. @Someone You Know

    Presumably because of a lower timestamp granularity?

  7. Paul says:

    If rebasing updates the timestamp in the PE header, how is Windbg going to find matching symbols for the DLL?

    Now that the timestamps are different it won’t be able to locate the original PDB generated by the compiler.

  8. Rutger says:

    You dont want to be dependant on the file last accessed date time for more reasons(compression that might optimize this away for instance)

    The header is updated when you rebase, actually we found that out accidentally this week as that header timestamp is also used to resolve modules stored in our symbol server, and it found nothing. Moving rebase upstream in our build process solved that…

    But this then means that rebasing a dll will cause the loop to check the entire export table.

  9. X says:

    timestamp is stored in the header because the filesystem timestamp can change when a file is copied.   Also FAT had (has?) a 2 second granularity.

    Rebaseing makes the binding invalid so it should update the timestamp.

    How does binding work on Vista and Win7 (with ASLR)?  Is it disabled?

  10. mikeb says:

    Tiemstamp is stored in the header for the reasons mentioned by several people (that the file timestamp can change independently of the file contents), but a more important principal (I think) is that the timestamp stored int he header need not be a timestamp – as Raymond said, it’s just a key to indicate that the DLL being linked to at runtime is the same (at least as far as the imports are  concerned) as when the importing module was linked.

    Even though the field is called ‘TimeDateStamp’ it could just as easily be any reasonable hash or key that might uniquely identify a particular build of the DLL – at least for the purposes of the optimization discussed in the article – there may be other uses for the field that require it to be a timestamp.

    [Right, the TimeDateStamp field should have been called UniqueID, but the person who named it presumably chose that name to suggest “A good choice for the unique ID is the current time and date.” -Raymond]
  11. Mark Wooding says:

    Am I missing something important about baseball or is the `recording the results…’ link to the wrong page?

    [Oops, meant to link to the other article from the same day. Fixed, thanks. -Raymond]
  12. yuhong2 says:

    ” (The format of this timestamp is in seconds since January 1, 1970, commonly known as unix time format.)”

    Is it 32-bit or 64-bit? In case you don’t know why I ask this, read about the year 2038 problem.

    [If you read the rest of the article, you’ll see why this isn’t a problem. -Raymond]
  13. mikeb says:

    > Am I missing something important about baseball… <<

    I think what you’re missing is that, just as  baseball is not merely a sporting event, resolving the imports from a DLL is a social event, not merely a technical event. (jk)

  14. Nawak says:

    > the person who named it presumably chose that name to suggest "A good choice for the unique ID is the current time and date." -Raymond

    Raymond, even before your time machine is completed, somebody’s got a compatibility bug for you.

    Beware when you finally back-fix all those buggy DLLs!

  15. Nawak says:

    > Is it 32-bit or 64-bit? In case you don’t know why I ask this, read about the year 2038 problem.

    I guess the lucky maintainers of 68 year-old DLLs will manage to press ‘recompile’ once more when they realize they are reusing a "unique ID"

  16. Joshua says:

    binding doesn’t work on Vista when binding to system dlls due to address space randomization.

    (where is kernel32 today hmmm?)

  17. Sash says:

    “assuming the target DLL is loaded at the same base address in both processes”

    Isn’t this assumption automatically broken on OSs that use address space layout randomization?  Is it implicit then that DLL binding optimizations are moot in this case?

    [Yes, if the target DLL is ASLR’d, then binding becomes ineffective. (Though the pages could still be shared since ASLR keeps the base address the same across all processes. Dunno if the loader takes advantage of this today.) -Raymond]
  18. Nick Lamb says:

    The memcmp() trick was worth mentioning because people actually do this, or rather, they do something equivalent but a bit smarter.

    Suppose you have a big system running a dozen VMs. And although the VMs aren’t strictly identical, for the usual support reasons they’re as similar as they can be. All six run the same version of the OS, and if several run the same application, that’s probably the same version too. So “hardware” page 0x45e0000 in VM #1 may be identical to page 0x4390000 in VM #6

    In this scenario (which is common and becoming more so) you get a huge saving in physical RAM from having a mechanism in the hypervisor (or in the OS supporting the hypervisor, in this case typically a Linux distribution) which can coalesce these pages. The strategy is to track pages which change infrequently and to keep hashes of their contents. If two pages have the same hash, there’s an excellent chance they’re identical, and you should memcmp() them and coalesce any that match.

    Of course once someone had this working for an extreme scenario where it’s the only option (VMs) people began experimenting to see how effective it is on an ordinary desktop OS. Raymond describes one way that two applications can end up with identical pages that can’t be caught by traditional COW, but there are countless others. The RAM saving is modest but significant.

    One day everybody’s ordinary PC may do this. Or, with RAM getting ever cheaper, perhaps only big VM farms will care.

  19. Demid says:

    I don’t get it: why does timestamp change when dll is rebased?

    Both values are written to the dll and loader checks preferred base address anyway so even if timestamp is the same the preferred base address isn’t and loader forced to do the loop.

    Did I miss something?

  20. Anonymous Coward says:

    I thought this is relevant:

    ‘BIND.EXE is the most well-known way to bind an executable. However, it optimizes your executables based upon your system DLLs. If you distribute your program to users, they probably will have different system DLLs, so you’ll want to bind your executables on their system. The Windows Installer has the BindImage action, which looks pain-free to use (although I must confess I’ve never written an installation script). Alternatively, you can use the API BindImageEx that’s part of IMAGEHLP.DLL.’

    • Optimizing DLL Load Time Performance by Matt Pietrek

    http://msdn.microsoft.com/en-us/magazine/bb985014.aspx

  21. peterchen says:

    > I guess the lucky maintainers of 68 year-old DLLs will manage to press ‘recompile’ once more when they realize they are reusing a "unique ID"

    Because Meiantainers of 68 year old DLL’s

    (a) are aware of the problem

    (b) check all possible permutations of binary version combinations on every release

    (c) Have build scripts automatically check DLL Binding IDs against a database of previously released DLLs

    (d) all of the above

    (I guess the reason is it may be any ID, not necesarily a timestamp)

    @Demid:

    Because that attitude changes the contract from "skip the loop when id changed" gradually to ".. or any of another dozen conditions are met". Some problems can be solved with more code, others with less contract.

  22. Gabe says:

    It seems like it would be safe to assume that ASLR is now going to be the norm, so binding would be pointless for the most part.

  23. Médinoc says:

    I don’t understand how the program obtains that timestamp: Is it stored in the import library when the DLL is built?

  24. Neil says:

    Paul, won’t rebasing the DLL mean that the PDB has all the wrong addresses in it?

  25. Leo Davidson says:

    I don’t know if it changes the timestamp but rebasing does not break PDBs.

    At least, we have a rebase step in our release build process that happens after the PDBs are generated and the PDBs still seem to work.

    I expect that either:

    1) Each PDB stores offsets relative to the module’s base address

    Or:

    2) The PDB stores absolute addresses as well as the original base address (so you can calculate the offset).

    It must do something like that or PDB symbols would be useless with processes which had to move a DLL at load time.

    (There’s no guarantee that a DLL’s prefered address range will be free. Rebasing can effectively happen at runtime as well as at build time.)

Comments are closed.