What is the effect of the /LARGEADDRESSAWARE switch on a DLL?


Nothing. Large-address-awareness is a process property, which comes from the EXE.

Comments (27)
  1. Anonymous says:

    And that was a question from someone let's say a year ago?

  2. Anonymous says:

    With high certainty, that is is the case.

  3. Anonymous says:

    I don't see Raymond having any obligation to reply to any question ever, let alone in someone else's timeframe.  At least he is frank and honest, a rare quality on the internet.

  4. Programmerman says:

    Is this Raymond's shortest blog post? I like the brevity, since the topic didn't need a long description or story.

    [Inspiration. -Raymond]
  5. Anonymous says:

    Do all statically linked DLL have to have this flag, for CreateProcess to honor the EXE's flag?

    [What part of "Nothing" didn't you understand? -Raymond]
  6. Anonymous says:

    …and as such can be applied to any executable, be it compiled in .NET or whatever..

  7. MItaly says:

    Uh, thank you, it was a question that bugged me some time ago, but I was too lazy to do some research about it. :)

  8. MItaly says:

    [I would think the onus would be on the plug-in author to adhere to the rules for writing plug-ins. -Raymond]

    Ok, but what about the usual corner case of legacy shell extensions?

    IIRC explorer.exe, as all the Windows components and as other processes that may use the common dialogs, is large-address-aware, but may load non-large-address-aware legacy shell extensions.

    Since the loader does not check, do the shell take care of this in some way?

    [Nope. Your badly-written shell extension that broke the rules is broken. (Why enforce this rule and none of the others?) -Raymond]
  9. MItaly says:

    I was just thinking that explorer (and the shell in general), knowing that the loader does nothing to avoid such scenario, would auto-protect themselves a little bit (e.g. checking on their own if the dlls they are loading have the largeaddressaware bit set), since they often have to deal with broken shell extensions.

    If "your badly-written shell extension that broke the rules is broken" and, as far as I understand, there's no quick compatibility shim that can avoid troubles, why let it potentially make damage in explorer and in other innocent applications?

    [The shell didn't load the DLL directly; it just called CoCreateInstance(). It doesn't know what DLL to check even if it wanted to. -Raymond]
  10. MItaly says:

    Uh, yes, I didn't think about it. I suppose that implementing such kind of protection not in the loader would require major modifications in COM, which is definitely a no-do.

  11. MItaly says:

    The chosen alternative is "come on, LARGEADDRESSAWARE for anybody, hooray, even if your dlls can't handle it; you may experience some big boom but – hey – it's part of the game!"; I don't think it's so much better.

    By the way, if the other alternative was chosen, the ones who would be in a situation like the one you said and who liked to risk (as they can do, maybe without even knowing, now), could just flip the largeaddressaware bit in all their dlls with EDITBIN, or, to address the problems of those who can't even flip a bit to licensed dlls, an additional flag could be added in the exe headers, to tell to the loader "go on, load even non-largeaddressaware dlls for me, I like adventure!". In any case, the risky behavior should be an opt-in, not the default.

    However, I understand that it's an almost lose-lose situation ("no matter what you do, someone will still call you a moron"), and speculating on past choices isn't all that useful.

  12. GregM says:

    Matteo, the risky behavior is already opt-in.  If you, AND ALL THE DLL YOU LOAD are not large address aware, don't opt in to large addresses by setting this switch.

    That doesn't help with buggy shell extensions, but then, there's really nothing you can do to guard against buggy shell extensions short of not loading them.

    The chosen alternative is really "LARGEADDRESSAWARE for anybody who says that they can handle it.  If your dlls can't handle it; you may experience some big boom but – hey – that's what you get for saying that you can!"

  13. MItaly says:

    By the way, this means that any dll that has any chance to be loaded by a large-address-aware exe must be prepared to deal with large addresses, correct?

    So this means that, e.g., any shell extension (which may be loaded in *any* process) must be large-address-aware – if not in the linker flag, at least in the facts.

    This seems to be the recipe for a disaster; is there any particular motivation for not choosing a safer system, like requiring that all the dlls referenced from a large-address-aware process be large-address-aware, maybe providing some magic override for those particularly fond of shooting in their own foot?

    [I would think the onus would be on the plug-in author to adhere to the rules for writing plug-ins. -Raymond]
  14. MItaly says:

    Ok, last question for today, I promise. :)

    Now the games are closed, but had you had your time machine working, given enough time, a 5% raise etc, would you go back in time and change the loader to make it work in a more cautious fashion (large-address-aware exe can load only large-address-aware dlls) to avoid these potential problems*, you'd think it's better to keep it as it is now?

    * after fixing the WinHelp animation first, obviously :)

    [The alternative is telling people "In order to make your program large address aware, you have to modify every single DLL your program uses, including the ones you don't have permission to modify since you licensed them from another company. -Raymond]
  15. MItaly says:

    Sorry, forgot an "or"

    you'd think it's better to keep it as it is now => *or* you'd think it's better to keep it as it is now

  16. Anonymous says:

    LoadLibrary could of course fail if the LARGEADDRESSAWARE-flag was not set on a dll and the exe is started as a LARGEADDRESSAWARE-process. That whould be more compatible.

    Windows could even (with a little work) start the process as a non-LARGEADDRESSAWARE process if not all statically linked dlls are LARGEADDRESSAWARE. That wouldn't prevent late binded dll of breaking the app ofcourse.

    And why does this flag even exist on dlls? Why do VS support setting it?

  17. Anonymous says:

    @GregM:

    LARGEADDRESSAWARE is not a tri-state (true/false/null) "flag", it's a bit flag which always exists.

  18. Anonymous says:

    Matteo, 640k: Just a thought: Although the loader doesn't use the DLLs' LARGEADDRESSAWARE flags to restrict the process to non-large addresses, the EXE's developer could use them (if he can read them) to consider whether the EXE should be marked LARGEADDRESSAWARE, in addition to documentation and testing.

  19. Anonymous says:

    Perhaps verclsid.exe should have been made large address aware. Of course, that only helps developers using /3GB.

  20. Anonymous says:

    Matteo:

    Jules suspected that shrinking the address space of a process to 2GB would be infeasible; I suspect that extending the address space from 2GB would be difficult too, although perhaps less. Wouldn't each and every DLL in the process have to be "aware" not only of large addresses but of the possibility and/or the execution of this dynamic shrinkage or extension? To me, building support for this into the OS sounds like too much work for a need that might be very rare.

    BTW, yes, my thought above was just aimed at the scenario you too hinted at, in which the author of an EXE that would benefit from large addresses knows in advance which DLLs will be used in in the process – after I read between the lines a question like "what is the use of whether a DLL is marked LARGEADDRESSAWARE?"

  21. MItaly says:

    @Horst:

    Maybe I explained my idea badly; I agree that dynamic shrinking/extension would be infeasible, and I never said it was the solution. As already said, IMO the right thing to do would be to check the LAA flag of all the static dependencies, and start the process as LAA or not-LAA depending on this – after all an executable should be able to work on a system without a 4 GiB address space without problems.

    The real problem are the delayload/manually loaded dependencies: after the process has been created, we all agree that it's infeasible to change it's address space extension. What I proposed as possible is a loader flag, that, if disabled (which would be the default if it were to be implemented today e.g. for Windows 8) tells to LoadLibrary to ignore all this stuff and just load the dll as it does now, if enabled would check if the dll and its dependencies is LAA before loading it in a LAA process; in case of such requirement is not fulfilled by the dll to be loaded, LoadLibrary would simply fail. A registry setting may be provided to enable by default such flag systemwide.

    Such change would not impact existing applications, because by default everything would be as it is now, but would let LAA applications choose not to risk with non-LAA dlls.

    On the other hand, IMO this flag should have been on by default at the moment LAA was introduced: this would have forced developers of LAA applications to be potentially aware of the problem, but would still let them say "who cares, just load my damn dll". A bit like the CheckForIllegalCrossThreadCalls property in Windows Forms: we know you're doing something potentially nasty, by default we warn you but if you like risk you can shut up the warning.

  22. Anonymous says:

    CheckForIllegalCrossThreadCalls. Yeah, I had to turn that off after it yelped at provably safe code (accessing variables that I added in my derived class from another thread).

    As for VB .NET, I wish that there was an option to set the EXE to LARGEADDRESSAWARE instead of me calling editbin as a post build action (there might be now but there's not in VS 2005).

  23. Anonymous says:

    Matteo: it really *can't* work any other way.  If you attempt to honour it for DLLs, what do you do about DLLs that are loaded after the process has allocated memory (which almost all shell extensions are likely to be: AIUI they won't get loaded until the first time the process displays a shell dialog)?  Remap the process's address space after the fact to prevent it exceeding 2GB? Sounds technically infeasible to me.

  24. MItaly says:

    @GregM:

    my problem isn't with the dlls loaded directly by the exe (although a check in the loader could make the whole thing safer), but the ones loaded indirectly, often without the original application even knowing it, e.g. the shell extensions I cited. The fact that you don't know what shell extensions may be loaded and if they support large addresses means that, to be safe, you mustn't use e.g. common dialogs *at all* in your application, since they may crash your application or, even worse, start to write stuff randomly in your address space (Heisenbugs FTW). Or, you could split your application in two processes, one large-address-aware that does the "heavy lifting" and one just to do UI and interfacing with can't-check-if-address-aware dlls. A huge mess.

    What's tragic is that this potential problem do not affect just your executable, in which you can choose to avoid the problem by not loading anything unknown/not using LARGEADDRESSAWARE, it affects potentially *always* explorer.exe, which loads uncheckable (as far as LARGEADDRESSAWARE is concerned) stuff by default.

    @Horst Kiehl:

    this cannot be done if the dll is loaded implicitly by COM, as Raymond already said. Such check can be done simply and efficiently only in the loader.

    @640k:

    [ Windows could even (with a little work) start the process as a non-LARGEADDRESSAWARE process if not all statically linked dlls are LARGEADDRESSAWARE. ]

    This in my opinion would be a nice solution, and it could be implemented even now that the games are closed. Even better, a new API could be made available to check if the address space is "large" or not, to let applications who really *need* a large address space deal with being loaded in a "normal" address space and let the user know.

    [ That wouldn't prevent late binded dll of breaking the app ofcourse. ]

    Some magic flag (like the one to change the loader search order) may be created, to let the application specify that it wants the "secure" behavior from LoadLibrary (and maybe even a registry setting to let the user force the secure behavior systemwide). This would be security as opt-in, but would be better than nothing.

    However, since we all know that features start with -100 points, I don't think that we'll ever see such thing. The problem do affect only 32 bit processes (for x86_64 processes LARGEADDRESSAWARE means nothing, so e.g. Windows Explorer on a 64 bit version of Windows is not affected anymore), and everything should be moving towards 64 bit. I don't think that all this work will be done to fix this kind of potential problems (that regards only a small fraction of legacy, but recent, 32 bit applications). Moreover, since these applications are large-address-aware, they are supposed to treat pointers carefully, so they shouldn't have big problems to switch to 64 bit.

    @Jules:

    [ If you attempt to honour it for DLLs, what do you do about DLLs that are loaded after the process has allocated memory ]

    LoadLibrary should fail if the "secure LAA" loader flag I described is set, or should go on (as it does now) if it's unset. Being the games closed, security as opt-in is the only thing that can be done.

  25. GregM says:

    "LARGEADDRESSAWARE is not a tri-state (true/false/null) "flag", it's a bit flag which always exists."

    Where did I say that it was a tri-state?  It's off by default, so don't set it unless you mean it.

    "my problem isn't with the dlls loaded directly by the exe (although a check in the loader could make the whole thing safer), but the ones loaded indirectly, often without the original application even knowing it, e.g. the shell extensions I cited. The fact that you don't know what shell extensions may be loaded and if they support large addresses means that, to be safe, you mustn't use e.g. common dialogs *at all* in your application, since they may crash your application or, even worse, start to write stuff randomly in your address space (Heisenbugs FTW)."

    The same thing can happen even if you don't set LARGEADDRESSAWARE.  That's why I said

    "That doesn't help with buggy shell extensions, but then, there's really nothing you can do to guard against buggy shell extensions short of not loading them."

  26. Anonymous says:

    I believe unless you're very sure, you shouldn't switch on LARGEADDRESSAWARE (so this flag is off by default).

    If your application needs to call DLL functions that needs a much larger address space, in these days you should just target x64 instead.

    Btw, just be curious, if you really have to code the application as 32-bit one, need a larger address spae, but also sure some particular version of DLL isn't handling large address properly, is it possible to instruct the compiler/linker always pass shared pointers to that particular DLL (or function) in < 2GB address space? If so, problem solved. (As application writers marking their application as LARGEADDRESSAWARE should take the responsibility of ensuring the codes work on large address)

    [Are you suggesting that the compiler/linker marshall all pointers so they point below the 2GB boundary? This is harder than it looks. (Imagine marshalling a linked list.) -Raymond]
  27. Anonymous says:

    There's two cases I suppose:

    1) A call that caller allocates memory. If it's possible to tell compiler to generate codes allocate that buffer from the < 2GB buffer, that'd be enough.

    2) A call where the DLL allocates memory (even a possibly partial case like the linked list you suggested). Is it possible to "frame" the loaded DLL, so that whenever it tries to allocate memory, it's always from the < 2GB address space?

    [The default behavior of VirtualAlloc is bottom-up, so memory allocations already tend to stay at low addresses as long as possible. The hard part of "framing" is knowing when you are in the frame and when you're not. -Raymond]

Comments are closed.