In Windows, the directory is the application bundle


Aaargh! wonders why Windows doesn't just steal the concept of bundles from Apple and package up each application with all its libraries and everything else it needs into a single directory.

This is such a great idea, it's been around since the early 1990's. It's just that Windows didn't give it a cute named like bundle. It just gave it the boring name directory. In other words, it's a victim of bad marketing. Maybe we should have given it a cute name like... elfgrass.

The first directory searched by the LoadLibrary function is the directory containing the application. If you put all your supporting libraries in the same directory as your EXE, and if you access non-library data via paths relative to your application directory, then then you have successfully packaged up your application with all its libraries and everything else it needs into a single directory. Congratulations, you pre-invented the bundle.

Indeed, Microsoft's guidance for over a decade has been to discourage copying files into the System32 directory.

If this facility has been around for nearly twenty years, why do people still copy files into the System32 directory?

Probably inertia. "We've always installed that way, and we're not changing it because we don't know what would stop working."

There may be some people concerned about disk space. "We have a bunch of applications which share some common libraries, and by putting the common libraries in a shared location (instead of in the application directory), we avoid wasting the user's disk space." Which raises the troublesome question: Is disk space still an issue nowadays?

Some people would say that no, disk space is no longer an issue. Hard drives are hundreds of gigabytes in size. Increasing the size of each application by a few dozen megabytes is just noise.

Other people would say that yes, disk space is still an issue. (Note how many people got excited when Snow Leopard took up less disk space.) Solid-state hard drives are still limited in size, and even for people with large hard drives, you see them freaking out about the disk space used by things like Windows Update, volume snapshot services, and System Restore. (Nevermind that at least for volume snapshots and System Restore, the disk space is automatically released when disk space starts to run low. It's like getting upset that the disk cache uses so much of your memory even though the computer is not under any memory pressure.)

Bonus reading: Disk space and Windows 7.

Comments (70)
  1. Adam Rosenfield says:

    One thing that Mac OS X does with bundles is that the Finder presents them as opaque files instead of the directories that they really are.  If you double-click Application.app in the Finder, it launches the application (instead of forcing you to browse to Application.app/Resources/MacOS/application and double-click that); similarly, double-clicking a bundle like Project.xcodeproj opens up the project in Xcode.

    I don't really like the behavior — I like it better when the interface is transparent and shows me things for what they really are.  At least I can always open up the terminal to look through the bundle directories.

  2. David Maher says:

    Adam – You could also right click on the item in Finder and choose "Show contents".

  3. SimonRev says:

    Raymond, I would like to respectfully disagree with your assessment.  In fact over the past ten years or so it has become MORE difficult to put everything your app needs in a single directory, not less.

    You are absolutely correct that LoadLibrary etc makes it easy to dump all your DLLs into your application directory.  But there are a fair number of other bits that need to get scattered in various places around the system, such as registry settings (e.g. file associations) and shortcuts. Disclaimer:  I have no idea how this is handled on the Mac, so I don't know if the situation is better or worse there.

    For me, in particular, the worst offender is that if my app needs to write out files for any reason, it cannot really write them to the application directory (which is presumably in Program Files), I have to dump them off in one of the app data folders.  

    Now, I have followed the evolution of windows for quite a while and I am not sure I would have done things much differently than Microsoft did.  But as an app developer, I would really have loved the ability to dump absolutely everything in my application folder, so that an uninstall would consist of merely nuking that directory.  I suspect from a practical standpoint there would still need to be some sort of registration with Windows saying: "Hey there is an app in directory zzz" and a corresponding unregister. but at least aside from that registration I would not need to scatter my files and data over half the system.

  4. R. Bemrose says:

    "For me, in particular, the worst offender is that if my app needs to write out files for any reason, it cannot really write them to the application directory (which is presumably in Program Files), I have to dump them off in one of the app data folders."

    And this exact same "problem" has existed on UNIX (and clones) for the last 40 years, and OSX for the last 11.  Essentially, an application isn't supposed to be allowed to modify its own files during everyday use.  Administrator/root access is required to make changes to the app itself or any data that applies to all users.

    The opposite end of this is the Portable Apps trend.  In that one, the application is essentially a single-user application, carried by the user from computer to computer.  This type of application is expected to have its configuration stored with the application, because it's never intended to have multiple users.

  5. Av says:

    A huge problem is registered DLL's and controls and such, like Active-X COM DLLs and OCX's.  If Application A uses "Widget X" and places the control in it's own application directory and registers it, it will work fine.

    Now along comes Application B, which also uses Widget X.  It also registers it.  Now Application A is using Application B's copy of Widget X.  Now if you uninstall Application B, it will remove Widget X (or it will fail since Widget X is still in use).  But assume it succeeds — Application A will now fail, since it can no longer find Widget X, since it isn't registered anymore.

    That is why I copy the common files to System32, that way all apps that need them can find them and never uninstall them.  It's messy but in the long run it is the cleanest.

    [Applications A and B should be using registration-free COM, so that each one gets its own version of Widget X. The problem you describe is the result of using a global solution to a local problem. -Raymond]
  6. Mike Caron says:

    SimonRev, that's how it was in XP (assuming Administrator, which was nigh universal) and earlier.

    The problem with this is with roaming profiles. If all my settings are in Program Files (which is tied to a given PC), then when I log in on another PC, my settings are gone!

    So, I reconfigure everything, and in the process discover a new option that I didn't know existed, so I turn it on.

    Once I'm done working on this other PC, I go back to my desk and log in here again, and my awesome new feature is turned off.

    "Stupid Windows, forgetting my settings even though my administrator set it up so that everything should go over the network!"

    Whether or not this is a valid argument is up for debate, but that's one important rationale.

    (Incidentally, reading the comments on the Bonus link made my brain hurt.)

  7. Mordachai says:

    In a similar vein, I've never understood Microsoft, and general Windows developers, love of DLLs.  Just link the whole kit & caboodle into your exe.  It has several benefits: single file, no DLL hell, smaller footprint, faster load times, less "things to go wrong".  DLLs made sense in Windows 3.11, and perhaps as late as Windows 98.  But by the time we're talking fully 32 bit OSes with an NT core, DLLs were more lose than win.  The reasons against DLLs are legion, the advantages are only in the minds of exactly the sort of people who worry about imaginary problems of being able to save a few bytes of hard-drive (by wasting even more because DLLs are more likely to have a larger overall footprint, so even that concern is generally bogus).

    However, it would be nice if Windows supported the ability to mark a folder as "treat this as an opaque unit" just as application bundles do (and of course offer a context menu to be able to see the contents of the folder).

    Similarly, as SimonRev points out, Windows has made it very difficult to have a strong "just drop this here and your software will work" mentality: the Registry was a logical idea at the time, but has ended up being a nightmare for application developers.  Far better would have been to make the registry a Windows-OS only thing, and force developers to use their own mini-registries in their own app data folders, where there is no chance of leaving garbage in the OS (bloating it) with every install and eventual uninstall-that-doesn't-really-get-everything.

    Yes, yes, this requires a time-machine.  Can't go back.  But x64 or Vista would have been really good "transition-points" to make a break with the registry.  It could still be done in a future version of the OS, all it takes is some backwards compatibility shim and the will to admit that the registry is more headache than value.

    And similarly, registering file types (and other similar install-registration record-keeping tasks) should have dedicated APIs rather than having to fiddle with the registry at the system level (potentially breaking things for the end-user, and giving no clean way to universally institute a smart policy for "what to revert an association to when a given package is uninstalled".

  8. Joe says:

    Did someone mention SxS? Why yes, yes I did.

    And DLLs provide a great advantage to applications or systems which require more than one executable. What's inexplicable is Microsoft's obsession with COM, especially when a straight API would make far more sense and be much less cumbersome.

  9. Jonathan says:

    Of course, if you bring your own copies of a DLL, and then that DLL has a security update, how would you track down all the copies of it scattered around the disk? That's exactly what happened with GDIPlus, and Microsoft Update had to write a special hacky "update" that scans the drive for it. And if it's statically linked, you're even more screwed.

    I understand package management in Linux has a better solution: Dependencies are explicitly listed, the package manager can download (and update) them if it needs to, and it keeps track of which app depends on which package so uninstall works correctly (this is second-hand info, so I'm not sure if it's accurate).

  10. Leo Davidson says:

    @Steve Wolf:

    There are situations where you have no choice but to use a DLL. e.g. If you to install a window hook or (certain types of) shell extensions as part of your program.

    There are other situations where a DLL makes things much easier. e.g. If you want to use UAC to elevate certain calls/objects. (You could also do it by starting another copy of your exe elevated, but then all you've really done there is take your DLL and put it inside your EXE, and if you need the two instances to communicate you've got to re-invent a lot of IPC/permissions stuff that was already done for you.)

    There are other situations where it just makes sense from a development point of view. The DLL may be written by a completely different team (or even company) on a different schedule without access to your build-chain. You might want to be able to replace it with other versions, or even remove it entirely at a later date.

    Plugins that users can write/install themselves fall into most of the categories above.

    BTW, there are APIs for registering filetypes. I think they were added in Vista.

  11. B. Beck says:

    @Steve Wolf:

    DLLs aren't just about saving space on the hard drive – it's also about security (any time a DLL is patched, any programs that were statically linked against it would have to be patched/updated separately (and we all know how much users love updates), and the memory saving applies to RAM as well.

  12. Alexey says:

    Application size is still very much an issue for application that are downloaded, not sold in boxes. I believe, this is the vast majority of applications. Were .NET apps required to link the whole BCL into the exe, none would have used it.

  13. Absotively says:

    @Jonathan: Your description of Linux package management is basically correct.  It works very smoothly, so long as everything you want to install is available through your package manager.  For Linux, most of the available software is open source, so the distributions can repackage and redistribute it and all is well.  I think that doing the same with commercial software would be considerably trickier.

  14. Fidel says:

    "so long as everything you want to install is available through your package manager"

    Ah yes. If it is not, you may be screwed. Because App A needs library L in version 13.3.7, but only version 13.3.6 is available in the package manager (bonus points if it doesn't work with 13.3.8 either). Hilarity ensues.

  15. Peter da Silva says:

    The Application Bundle concept that Apple uses was invented by NeXT in the late '80s and released in NextStep in 1989. It is far more than "putting resources in the same directory as the application", it's a specific layout of components that exposes information via specific property list files in the bundle contents. It provides a template for helper application registration, file type registration, preferences, and shared libraries and other frameworks.

  16. Evan says:

    "The first directory searched by the LoadLibrary function is the directory containing the application."

    The inability to do something like this on *nix is a huge PITA and has been a continual source of annoyance. (You could do it if you manually dlopen the .so then hook up all the symbols, but if I wanted to write a linker I'd write a linker.)

    @SimonRev: "But there are a fair number of other bits that need to get scattered in various places around the system, such as registry settings (e.g. file associations) and shortcuts."

    If you really wanted to write a "zero install" app, most of those things you could do the first time the program starts.

    @Alexey: "Application size is still very much an issue for application that are downloaded, not sold in boxes."

    Eh, it's an issue, but I don't think it's "very much" of an issue a lot of the time. (It would be for sort of "impulse downloads.") I mean I have just about the worst DSL or cable connection you can get (I'm cheap), and I don't really think twice about getting a 5 or 10 GB game off Steam. The number of times that I've failed to download something because it's too big is close to zero. (The main thing I do do to save bandwidth is to turn the quality down on YouTube.)

    @SimonRev: "as someone who has never used roaming, my inclination would have been to save per-user settings somewhere like <app folder>/<username>/usersettings.xml.  I suspect that might not be roaming friendly though."

    There is a roaming folder like that. For me, it's at c:UsersEvanAppDataRoaming, but I'm not sure the correct way to find that folder.

    @Absotively: "@Your description of Linux package management is basically correct.  It works very smoothly, so long as everything you want to install is available through your package manager."

    And as long as you're root. For whatever reason, virtually no package managers allow use as non-root despite the fact that it's entirely possible to install almost anything as a private install if you do './configure –prefix=…'.

  17. Evan says:

    Oh man, I lied a bit: "The inability to do something like this on *nix is a huge PITA and has been a continual source of annoyance."

    So apparently you CAN do this by using an RPATH relative to ${ORIGIN}. My manpages are too old to document this, but supposedly it's in newer versions of them, and it works with my ld.so anyway.

    That solves half the problem… now if only it would be come standard practice to use them.

  18. Kemp says:

    @Evan: Being root temporarily isn't a problem. Using sudo via the command line, or letting the system ask you for your password to elevate a GUI app. I think the general consideration is that in most end-user scenarios, a program is installed for a machine, not for a user. Additionally, in a work environment you don't want your users installing random things whenever they feel like it, and in a home environment you know your password anyway. Imagine how much people would freak out about disk space on a machine with multiple users who all had to install their own copies of each program… (yes, symlinks and so on, but that would start to become a very complex solution to a simple issue – type your password).

  19. MWF says:

    @Steve Wolf

    To say that we would be better off without DLLs is just plain crazy. DLLs provide a necessary ability to create modular, extensible applications. Dynamic/shared libraries are also very useful to streamline development of large projects whose team may be geographically separated.

    Without dynamic libraries, you can't efficiently develop portions of a larger solution in parallel, since you have to re-link to the new static libraries whenever there is an update available. With DLLs, you would only need to take action if the interface changed. (And depending on some specifics, you may only need to take action if there were a change to an interface method you were already using, not needing any action if the interface was only expanded with new functionality.)

    Without dynamic libraries, sharing code between multiple projects becomes more of a pain – again, you have to re-link your entire application to the static libraries instead of just dropping the new version of the DLL in.

    Without dynamic libraries, creating modular applications is nigh-impossible. Any plugin functionality you would like to have must be developed from scratch, using whatever constructs you decide on – instead of just allowing a dynamic library to load with a specific interface requirement. And keep in mind that plugins can mean more than just optional extensions to an application – they might feature essential functionality as well. For example, you might have an application that interfaces with some specialized piece of hardware. Say that a competitor hardware vendor creates a new device that serves the same purpose, but interfaces differently. Without dynamic libraries, you would have to update your entire application if you wanted to support both. If you had instead architected your application to load a dynamic library for the implementation of the necessary interface, you could just deploy an additional DLL that supports the new device. (In addition to the existing DLL to support the original device that shipped with your application in the first place.)

    The only "problems" with DLLs are the people who use them (or try to use them) without understanding what they are doing.

  20. Colin says:

    @Kemp: so instead of users being able to install packages to their account from a repository the admin trusts, the user is forced to download the source to the package and build and install it locally (assuming their sysadmin gives them access to the system compiler), where it's completely invisible to the package management system.

    "cutting off your nose to spite your face" about covers it.

  21. Michael Grier [MSFT] says:

    @James Bray:

    You should read the linked article on Windows 7 disk space usage.  The WinSXS directory really doesn't use significantly more space than was present in prior versions (XP, Windows 2000), it's just that it's all in one place so you can see it.  It used to be that the same files were kept in a myriad of different places, never shared.  Starting with Vista, files superceded by newer updates, links to the current files, copies of files applicable only when you move to the LDR branch (QFEs) and staged updates for future installation are all present in one place.

    It's true that there are a ton of new single file directories present and each one uses an MFT entry but really all this stuff was there before.

    The use of hard links is also problematic for older tools which had never grown up to deal with them before.

    There are higher level policy decisions which somewhat exacerbate the issue (e.g. nobody wanted to do the new work to connect to WU and download different files when installing or uninstalling an LDR) but actually this was the policy / behavior since Windows 2000 in any case.

  22. voo says:

    @Steve Wolfe: Why would your exe that includes every dll be smaller and load faster? If at all it'd generally be slower because it always has to load the extra code which may not be necessary for a dll. Smaller also won't work everything else considered equal.

    But really all that stuff is unimportant (saving a few mb of RAM? ah well, not that important) with the fact that if you statically link your program with any kind of security relevant library you'll have to release a new version of your program each and every time a flaw is found – that's not only inconvenient for the programmer but also the users (not many users appreciate having to update a program every other week).

    @Evan: "There is a roaming folder like that. For me, it's at c:UsersEvanAppDataRoaming, but I'm not sure the correct way to find that folder."

    SHGet(Known)FolderPath to the rescue. A quick glance at msdn.microsoft.com/…/dd378457%28v=VS.85%29.aspx shows FOLDERID_RoamingAppData for Vista+ or CSIDL_APPDATA for the XP guys.

  23. Klimax says:

    SimonRev 20 Jun 2011 8:33 AM:

    "My files fell into the category of temporary files as well as files generated as part of normal operation of the hardware that my software was emulating (and had no purpose existing after an uninstall)."

    If tempoary then put them into Temp folder. There is API for creating temp filenames (from pattern) and getting path to temp directory.

    If generated during operation and to be persistent until uninstall then ProgramData (aka old AllUsers AFAIK) is place for them or if user specific then AppData inside User profile.

    Writing into Application directory during normal operation(by non-admin) should not be done at all as that is hole in security.

    APIs:

    msdn.microsoft.com/…/aa364991(VS.85).aspx //GetTempFileName Function

    msdn.microsoft.com/…/aa364992(VS.85).aspx //GetTempPath Function

  24. moi says:

    What about the 2008 CRT DLLs that won't load from your own library.  That was a huge mess that a lot of us are still stuck with.  If you want to use 3rd party DLLs, you can't unless you buy the source code and recompile everything.  If I use 2008 SP1 but one vendor used 2008 vanilla, things go south.

  25. Chris L says:

    As silva points out, the Mac app bundle contains all sorts of metadata indicating what file types and URL schemes the application can open, so you never need to manage this. Copying the app to your hard drive is enough to register it, deleting it will unregister it.

  26. Evan says:

    @Kemp: "Being root temporarily isn't a problem."

    It is if you don't have root privileges.

    "Additionally, in a work environment you don't want your users installing random things whenever they feel like it,"

    Then why can I just run configure & make? Why isn't it standard practice to mount everything but /usr noexec?

    "Imagine how much people would freak out about disk space on a machine with multiple users who all had to install their own copies of each program…"

    If only there was some way to keep track of how much space a user was user and limit it. Oh well, I guess that's a pipe dream.

  27. Absotively says:

    @Evan: Package managers generally install binaries, and of course using './configure –prefix=…' would require compiling everything on install.  Plus, most package managers are meant for system administration; it's not that they want to prevent users installing software, it's just that making it easy for users to install software is not why they exist.

    There also exists Zero Install (http://0install.net/), which is a system specifically designed for making it easier for users to install software.  So it's not that no one cares about this problem, it's just that the people making system administration tools don't care about it.

  28. Jory says:

    Adam – that's the correct level of abstraction though. For most people, an app "folder" is an app. They don't want to know how an app is put together. Here's what people want to do with apps: Install them, launch them, remove them from their systems. For those operations, the opaque bundle is the correct metaphor.

  29. Mordachai says:

    Things people forget about static-linking vs. DLLs:

    1. static-linking only includes the part of the library you actually use, not all of it, resulting in smaller disk & memory footprint.
    2. DLLs are an additional security liability: hack the DLL (or replace it)

    3. "Instead of releasing a whole new app, you can just drop in the replacement DLL" – what hogwash!  How is that one whit different than releasing a patch for your application?  In what way does that require any less testing, security issues, transport & install issues?  This is just vacuous.

    4. Because my app is correct, I can patch it, and only it.  Because your app depends on some specific version of some specific DLL, Windows has to track every version of every DLL ever written in order to pair them up correctly.  You're wasting vast chunks of the user's hard drive to SxS while I waste zero.

    5. Many different exe's use the same DLL so there is a memory footprint (and hard disk) savings.  Bogus!  Every exe typically uses a very specific version of a DLL or it crashes (or otherwise behaves in ways not conductive to the user's happy experience).  Along with the fact that the DLL is installed 10 different places consumes 10x as much space.  And the fact that SxS keeps track of them all for all versions uses up yet more space.

    To those who worry that I'm eschewing DLLs en masse, I am not suggesting that they have no place (e.g. the OS kernel is a fabulous place for them, as is localized resource-only DLLs).  I'm sure there are other good examples of when to use DLLs, but most people resort to the above 5 bassackward justifications (which are all verifiable falsehoods).

    I'm not suggesting that dynamic linking is specific to Windows.  However, Microsoft & most Windows developers over-reliance on DLLs is to what I'm referring (their near religious affinity for them despite the excruciatingly well documented DLL-Hell of which most of us who've done any business in developing and supporting Windows software are well aware is bordering on insane).

  30. Evan says:

    @Steve Wolf: "It's just there to counter the idea that static is less secure than dynamic, which is goofy."

    Fair enough.

  31. grumpy says:

    huh, bundles in Windows? I guess that's why installers and uninstallers frequently take a good 20 minutes to scatter as many bits and pieces over as many difference places as possible, yes?

    Or perhaps it's just that Apple is wrong, and wasting the user's time and diskspace, and turning defective installers and uninstallers into rocket science is a better solution? There must be a reason why Microsoft does so much to encourage that model for Windows applicatinos.

  32. Fidel says:

    @SimonRev:

    But the stuff the application writes in the user-specific directories is *not* part of the app itself and should not be automatically removed. As for the registry keys, yeah, that's definitely a problem. A somewhat cooler solution would have been to add an application-specific hive in its directory that automatically gets unloaded when the directory is removed.

  33. Haha says:

    Wow that's great.  I can move Office from /Program Files to /Apps and it will just work!   Just like OSX!   Yay Windows!

    /s

  34. SimonRev says:

    Ironically I think this discussion has drilled into me more than perhaps anything else how diverse the use cases of Windows are.

    In my app, I am specifically thinking of files that are generated by the app as part of its execution that are neither documents/typical output as Fidel immediately thought (which go through a normal save-as process and default do My Documents), nor are they application settings as Steven thought (which I save to the HKCU/Software/…)**. To me, the fact that everyone jumped to a different conclusion speaks volumes to how difficult a one-size-fits-all solution would be.   My files fell into the category of temporary files as well as files generated as part of normal operation of the hardware that my software was emulating (and had no purpose existing after an uninstall).

    ** However, as someone who has never used roaming, my inclination would have been to save per-user settings somewhere like <app folder>/<username>/usersettings.xml.  I suspect that might not be roaming friendly though.

  35. Ben says:

    @Steve Wolf: "5. Many different exe's use the same DLL so there is a memory footprint (and hard disk) savings.  Bogus!…"

    I think you're misunderstanding people's argument here. While you may be correct when considering disparate apps (Which may get out of sync and each require their own dll version), this is a good reason to use dlls if you are making something that could be considered a "suite", like Microsoft Office, where all of your binaries are shipped at once. You'll get a memory and disk footprint saving from consolidating into a dll all of your UI controls, for example, or file formats.

    Or, the logic of your program can live in a dll, while your GUI and command line versions can be different executables(Again, considering a program in which both are installed a s a single package). An extra benefit of this is that you can't half apply a patch anymore. (Say you find a bug in your logic, and issue a patch. If you are only patching one dll, instead of 2 executables, you are guaranteed that for a given machine, the logic won't be different between the different interfaces.)

    Along those lines, it could be argued that dlls with well defined interfaces can be used to provide an extra layer of encapsulation.

  36. James Bray says:

    Regarding the disk space usage; I'd say WinSxS (side-by-side DLL storage) uses *by far* the most disk space on Vista and Windows 7.

    I understand the problem it solves, but its at a huge expense of disk space (at least for us SSD folks)

    James

  37. Joshua says:

    True story: I've had to write "patches" that copied the whole install directory and changed one file so the patch was different between two instances so that the logic change required the least amount of testing.

  38. Reg-free COM says:

    [Applications A and B should be using registration-free COM, so that each one gets its own version of Widget X. The problem you describe is the result of using a global solution to a local problem. -Raymond]

    And if application A and B are Sidebar (Desktop) Gadgets?

    Glad to hear it works SO well there.

    [Sorry, didn't realize that Av was talking about gadgets. Sadly, gadgets don't support manifests. -Raymond]
  39. dave says:

    @Steve Wolfe

    >In a similar vein, I've never understood Microsoft, and general Windows developers, love of DLLs.

    This implicitly indicates that shared libraries are some sort of Windows thing.  And yet shared executable code modules have existed in practically every general-purpose operating system I've used since 1975 (and they weren't new then).

    There *is* a discussion to be made about modularization tradeoffs, to be sure, but let's not pretend it's somehow specific to Windows.

    FWIW, app isolation ("if someone changes the code, I've still got the old stuff") is a blessing if you don't want the change and a curse if you do want that change.

  40. 640k says:

    @Raymond: Increasing the size of each application by a few dozen megabytes is just noise.

    Yes, a few dozen megabytes is just noise, WinSXS on the other hand is insane.

    @R. Bemrose: And this exact same "problem" has existed on UNIX (and clones) for the last 40 years.

    Those who don't understand UNIX are condemned to reinvent it, poorly.

    @Joe: What's inexplicable is Microsoft's obsession with COM, especially when a straight API would make far more sense and be much less cumbersome.

    Yes, yes, a thousand times yes.

    @James Bray: Regarding the disk space usage; I'd say WinSxS (side-by-side DLL storage) uses *by far* the most disk space on Vista and Windows 7.

    No, pagefile + hiberfile does.

    @Evan: If you really wanted to write a "zero install" app, most of those things you could do the first time the program starts.

    Easy to forget that apps must be able to run as non-admin, isn't it? I know that file associations can be a user setting, but in a large enterprise every user should not have to setup every app, it should be done by admins. And there's lots of other system settings which requires admin privileges to set up.

    @Michael Grier: The WinSXS directory really doesn't use significantly more space than was present in prior versions (XP, Windows 2000),

    This is simply not true. WinSXS keeps every dll ever installed, both 32-bit and 64-bit. Good luck finding gigs of 64-bit dlls in Windows 2000.

    [Meaningless comparison because there was no 64-bit version of Windows 2000. (And the 64-bit version of Windows XP was never in wide distribution. I'm not sure if it's even serviced.) I think Michael's point is that the disk space usage hasn't changed; the files just moved from the various scattered locations they used to be (so you never noticed them before) to a single place where you can complain about them en masse. -Raymond]
  41. Evan says:

    @Absotively: "Package managers generally install binaries, and of course using './configure –prefix=…' would require compiling everything on install."

    That shouldn't make a difference if the *nix method was sane. (But then we come back to the fact that ${ORIGIN} was apparently long-undocumented, largely unknown, and almost entirely unused.) But even still, last I checked even something like Gentoo's Portage requires root.

    > "There also exists Zero Install (http://0install.net/), which is a system specifically designed for making it easier for users to install software.  So it's not that no one cares about this problem, it's just that the people making system administration tools don't care about it."

    That *is* basically what I'm lamenting. :-)

    @640k: "Those who don't understand UNIX are condemned to reinvent it, poorly."

    My corollary to that is "Those who understand Unix are condemned to invent it, well." (Read that in a somewhat disparaging tone. Unix does a lot of things well, but a lot of things… let's say "not so well", especially given that it's no longer the 1970s.)

    > "Easy to forget that apps must be able to run as non-admin, isn't it? I know that file associations can be a user setting, but in a large enterprise every user should not have to setup every app, it should be done by admins. And there's lots of other system settings which requires admin privileges to set up."

    So we have settings in three categories. (1) are those which need admin rights, in which case you're right. But, say, OS X .app bundles don't support such setup either, and for a variety of factors it's reasonable that if something needs admin rights something like a simple bundle won't work. (2) Settings which are the same from user-to-user, but do not need admin-privileges. Here the first-time startup code can just display a progress bar for a couple seconds. The user is hardly inconvenienced at all. (3) Settings which differ from user-to-user. Here you need to display a dialog to the user… but that's sort of fundamentally true anyway. Otherwise how are those settings going to differ?

    The main objection you can take to the Windows way perhaps is the quantity of settings that require admin rights. I can't speak to that too much.

    @Steve Wolf: "DLLs are an additional security liability: hack the DLL (or replace it) … How is that one whit different than releasing a patch for your application?"

    I like how you juxtapose these two things. What are you going to hack about the DLL that you couldn't hack about the original program? How are you going to be able to replace the DLL and not the original program?

    With a reasonable setup, I know of basically one additional vulnerability that DLLs introduce, and that's import table patching. And if your program is to the point where an attacker is patching your import table… you don't have much hope. I might be missing something, but I don't view your (2) as being very convincing.

  42. Mordachai says:

    @Evan – Mainly I'm countering the notion that having the code statically linked somehow makes it less secure than the DLL, which is silly.

    The additional vulnerability that DLLs introduce is that your software, unless it fully specifies the path, relies upon the OS loader's search algorithm, which can be taken advantage of to load a bogus DLL instead of the real one.  It is *potentially* more vulnerable, because the behavior is more complex, there are more points of attack (every folder in the loader's search algorithm).

    However, I could care less about this angle of argument.  It's just there to counter the idea that static is less secure than dynamic, which is goofy.

  43. Gabe says:

    If app bundles are so great, why are there still apps with installers for OS X? What do they do that's so special that the all-powerful app bundle system can't handle it?

  44. Joshua says:

    @Myria: Microsoft listened and fixed it for VS2010. Sadly, I can't find a MSDN-documented solution for VS 2005 although there are a couple of third-party methods.

  45. Will says:

    @Evan: "There is a roaming folder like that. For me, it's at c:UsersEvanAppDataRoaming, but I'm not sure the correct way to find that folder."

    %appdata%

  46. Myria says:

    @Joshua: Sure, except that 2010 is four times the price of 2008…

  47. zedware says:

    If most of the system files are read-only

    then

      why not let the Windows OS de-dup it in background to save disk space?

    fi

    Anyone know if Windows Desktop support this feature?

    [Even better: The de-dup'ing was performed at setup time! The files in WinSxS are hard linked to other copies in the system where applicable. -Raymond]
  48. Dave G says:

    The difference between a directory in Program Files and a bundle is **organisation**. Bundles have an organised directory structure that the shell/Finder (or I suppose anything else) can utilise. For example, because a bundle contains localised resources, Finder is able to show localised information, including a localised application name in the Finder window if available. The organisation of a bundle makes it better than a directory in Program Files.

  49. Worf says:

    @Gabe: because there are still instances where bundling doesn't work. E.g., utilities that hook into the system, device drivers, etc. And there are componentized installs where a common install media may install more than you're licensed for, so they'll just create the app with the parts you bought, rather than a big giant package.

    But Apple has encouraged bundling – iOS apps must be self-contained save for the system library – they run as a non-root user in a sandbox where they can only see themselves and a few system-created directories one level up, but not other apps and their data, or user global data like contacts and content (they can use APIs to get at them though).

    The Mac App Store has similar requirements – apps must be standalone. They download and are ready to use. Which is one reason a bunch of app types are banned (e.g. drivers)

  50. Dave G says:

    @Gabe: Bundles might not eradicate the need for an installer, but it greatly reduces the need. Many of the apps that use installers probably don't even need them [citation needed]. Also, it's not just about installation and uninstallation; the consistent organised structure makes bundles far more utilisable than a plain directory with an EXE and some DLLs in it.

  51. Simon Buchan says:

    Local deployment is possible with MSVCRT 70-80, although it can be complicated to set up: blog.kalmbach-software.de/…/howto-deploy-vc2008-apps-without-installing-vcredist_x86exe

    I wonder why MSVCRT's default manifest setup didn't allow satisfying a load request with a security patched version – as @Myria noted, the existing behaviour seems more likely to break applications (not to mention increases the amount of code with known security exploits on people's machines!).

  52. Troll says:

    In XP, back up of files superceded by newer updates could be skipped so only current files remain on the system, in Vista/7, it cannot. In XP, Update.exe could be told to install the QFE branch using the /b /SPxqfe switch, in Vista, there is no such switch. In XP, there was no concept of staging, sure the user had to insert the XP CD when adding/removing components and possibly reinstall a few updates but this saved space. In Vista, the decision is not left to the user, they are all compulsorily staged so even components that aren't "on" and their serviced copies of copies of copies take space on the system. Office 2007/2010 also gives no option like Office 2003's Local Installation Source did whether or not to stage all MSI and CAB setup files on the system. Windows Installer also by default caches files for all apps modified by delta updates which is why when installing for example, SP1 for Visual Studio your %windir%InstallerPatchCache folder gets completely bloated up even if the users prefers it not to be cached and rather be prompted for the original install media.

  53. Troll says:

    In XP, back up of files superceded by newer updates could be skipped so only current files remain on the system, in Vista/7, it cannot. In XP, Update.exe could be told to install the QFE branch using the /b /SPxqfe switch, in Vista, there is no such switch. In XP, there was no concept of staging, sure the user had to insert the XP CD when adding/removing components and possibly reinstall a few updates but this saved space. In Vista, the decision is not left to the user, they are all compulsorily staged so even components that aren't "on" and their serviced copies of copies of copies take space on the system. Office 2007/2010 also gives no option like Office 2003's Local Installation Source did whether or not to stage all MSI and CAB setup files on the system. Windows Installer also by default caches files for all apps modified by delta updates which is why when installing for example, SP1 for Visual Studio your %windir%InstallerPatchCache folder gets completely bloated up even if the users prefers it not to be cached and rather be prompted for the original install media. Some users would like it if there are no "staging" done for system files or drivers (driverstore) which is another bloated store.

  54. anon says:

    well.well.well. welcome to Visual Studio 2010 where you will learn that deployment means simply dumping msvcr100.dll into your system32 directory. Back to the old days, huh? makes servicing easy, huh?

  55. Myria says:

    @moi: It gets even worse with security updates.  Twice this year, Microsoft has released new versions of msvcr90.dll.  Let's say you have a third-party DLL to which you do not have the source code.  This third-party DLL is dynamically linked against msvcr90.dll, as is your application.

    When a second Tuesday rolls around, your company's security team pushes out the new security updates to your company's computers.  You then make a build of your application and send it to customers.

    The new version doesn't run on your customers' computers!  They have to install *two* versions of the redistributable packages in order to fix it.  Worse still, your application ends up loading two copies of msvcr90.dll, one of the old version and one of the new version.  If you pass CRT objects like FILE *'s between your application and the third-party DLL, your application will probably crash, because the other msvcr90.dll won't recognize the object.  And this all broke because of a security update on a developer's machine.

  56. Random User 288534 says:

    So, I think we have determined the "bundle" is not a suitable form for every kind of software. I begin to wonder how difficult it would actually be to create the pieces needed (shell extensions, etc.) to enable "bundle support" in the shell for most or all types of software for which a bundle makes sense.

  57. Random User 288534 says:

    And just to clarify, I am referring to "bundle support" in the sense most of the commenters are using, since so many seem to disagree with Raymond's interpretation.

  58. Troll says:

    Wasn't the ability to load DLLs locally (not to be confused with WinSxS and Reg-free COM) introduced in Windows 2000 or something? I recall it was called Fusion or something.

    "Is disk space still an issue nowadays?" Disk space oh don't get my started on that please. Seems like a typical shameless Microsoft stance since the days of Vista. All products are engineered without disk space in mind (compulsory caching of Office/Windows Live Setup files, all language files, entire components in case of Windows Vista/7's disastrous Component Servicing Store). In case anyone dunno, ****WinSxS is not just used for side-by-side DLLs in Vista/7 like XP****. Beginning with Vista, it is also used to store the OS itself, the CBS store and copies of copies of backups of backups of files. Steven Sinofsky even blatantly lies in that blog post Raymond linked to saying the size of WinSxS is close to 400 MB. Probably he meant 4 GB which is excluding hard links. Yeah I know SOME of the files are hard linked, not ALL. The servicing mechanism in the abomination called Vista took away the /nobackup switch from users. You can measure the true size of WinSxS exluding hard links using a utility called cttruesize (ctts.exe). WinSxS is enough reason to stay away from the bloated by design Windows 7 and Vista systems. The Servicing stack backs up files before they are serviced by updates and then never deletes or what MS calls scavenging. Only service pack files can be scavenged and although Microsoft claims that adding and removing Windows components will scavenge backed up files, I see my free disk space only decreasing as I remove and add back the same component again. Their ridiculous recommendation is to install fewer updates to keep the size of WinSxS under control. Of course, one can't deny installing security updates so the cost of fixing bugs by installing hotfixes comes at the price of enormous amounts of disk space. The servicing mechanism was redesigned with the aim of more reliable servicing, offline servicing and fast first install, but what turned out to be is more of a downgrade to XP's update.exe method. The servicing stack causes all sorts of other issues as well like slow boot (Please wait while Windows configures updates crap which we didn't have in XP), heavy I/O upon logon and logoff when updates are installed and systems being unable to boot because of failed updates (reliable really?). Not to forget the inability it introduced to do a true slipstream of service packs and hotfixes. Windows 7 and SSDs are a huge problem because of WinSxS and sadly I don't see the servicing mechanism being reworked completely in Windows 8.

  59. Troll says:

    "WinSXS directory really doesn't use significantly more space than was present in prior versions (XP, Windows 2000), it's just that it's all in one place so you can see it. It used to be that the same files were kept in a myriad of different places, never shared. Starting with Vista, files superceded by newer updates, links to the current files, copies of files applicable only when you move to the LDR branch (QFEs) and staged updates for future installation are all present in one place. It's true that there are a ton of new single file directories present and each one uses an MFT entry but really all this stuff was there before. The use of hard links is also problematic for older tools which had never grown up to deal with them before. There are higher level policy decisions which somewhat exacerbate the issue (e.g. nobody wanted to do the new work to connect to WU and download different files when installing or uninstalling an LDR) but actually this was the policy / behavior since Windows 2000 in any case."

    That's so not true. Looks like people believe what they read if its published by Microsoft without actually verifying how much space WinSxS actually takes. Sure XP stored them scattered in %windir%dllcache, $Uninstall_KBxxx folders and $hf_mig$. But the WinSxS directory takes far far more space in Vista/7 than these folders in XP/2000 combined because in XP/2000, you could specify the /nobackup switch when installing updates so previous versions of files are never backed up. This ability/feature is completely absent in Vista/7 so copies of copies of copies of files accumulate. You can't delete files like you could delete $Uninstall_KBxxxx$ folders in XP/2000. Deleting those was *harmless* in XP/2000 IF you have no intention to uninstall the hotfixes, the only side effect was they became permanent. You also cannot delete the equivalent of %windir%$hf_mig$ files in XP (now these shouldn't be deleted) because when doing SFC /scannow, the latest correct version can be obtained from $hf_mig$) but these could be compressed in XP using NTFS compression without sacrificing performance of the OS. In contrast, in Vista/7, compressing WinSxS leads to direct reduction in performance of the OS. Just give users back the damn equivalent to the /nobackup switch for the new servicing mechanism so updates will become permanent and can't be removed but tons of space is saved. Secondly, all Microsoft tools and especially the Explorer shell needs to be updated to correctly calculate and report the ACTUAL space used instead of just saying "hey those are just hard links, it's not our problem Explorer miscalculates them". Vista/7 give no choice to the user whether or not to reserve space for files from various brances (QFE, LDR, GDR) backed up during servicing. That is the main problem. It's there in Features removed from Vista article on WP: "Windows Vista uses Package Manager (Pkgmgr.exe) and Windows Update Standalone Installer (Wusa.exe) to install software updates and hotfixes. However, these do not support the various command-line switches like Windows XP's Package Installer (Update.exe) did. Much of the functionality from Update.exe is missing. For example, there is no way to skip backing up uninstall information for hotfixes using the /nobackup or /n switch. Windows Vista backs up files before installing hotfixes to the %Windir%WinSxS folder. Since the backing up of files cannot be skipped, this folder's disk usage can increase considerably over time."

  60. lolski says:

    yeah, and because directories == bundles there's no need for a registry or a installer on windows. … oh wait

  61. Mordachai says:

    Microsoft has an incredibly difficult job, in that everyone has different designs on how the system should work, and pointedly, to what purpose they're using Windows.  Hence, different needs pulling Microsoft's design & development teams in many directions at once.

    That said, I think it's fair, giving that I've been a professional programmer for DOS/Windows since DOS 2 or 3, that we can say in all fairness that Microsoft has never been very good at organization.  Creating standards on the filesystem has, it seems to me, always been an afterthought.  

    Bundles are a very conscientious forward-thinking approach.  They're imperfect, and don't cover every usage scenario, but they cover the 80/20 rule, and for the remaining 20% of cases, a custom installer can be written which means that for 80% of the software out there, the user's experience is much more standardized and under their control.

    It's a difference in philosophy: Apple over-designs and perhaps over-constrains their developers.  Microsoft errors on the side of giving very little help by way of standardizing things and leaves it wide open to developers to invent 100 different solutions to the same problem (everyone writes their own installer / uninstaller, e.g.).

    Perhaps there is a better balance, that looks towards the 80/20 rule: make it easy for the common cases to "get it right" and "be consistent", while allowing for the uncommon to have an escape hatch that allows for completely custom scenarios.

    In conclusion: Bundles (as the full-meaning from NeXT and now OSX) would be a welcome addition to Windows.

  62. Skyborne says:

    Re Linux package management: it depends on your package manager.  I quit using Gentoo when I accidentally broke coreutils (ls, cp, mv, install, etc.) by removing something they actually depended on.  Because in the 5 years I was running Gentoo, they never got around to implementing reverse-dependencies, which would have told me, "n00b! ur b0x0rz r 70457 if u d0 th47."

    Regarding security, there are some problems with "just working" without root permissions.  Aside from losing some abilities (e.g. on Linux, exposing your gconf schema properly and having a canonical name on DBus), it is also the ideal situation for malware.  If there is a hole that allows for running something automatically, then you just lost the game.  I assume this is why Windows keeps track of whether files came from the Internet and gives you a chance to reconsider running them.

    Also, I wish people would write their installable web apps to support secure configurations instead of requiring code directories to be writable by the webserver, complete with the "You can do this by issuing CHMOD 777 in your FTP client" directions. >X-(

  63. In vein with getting stubborn vendors to not put DLLs in system32, or worse c:<theirProduct>, it's also hard to get them to put non-program files into user profiles appdata and program data.   Even when they start putting some data into user/appdata program files are still littered with hundreds of non-program related files.  This problem also exists with ini files and the registry.

  64. Medinoc says:

    I think the main problem with uninstallation nowadays is that there is no convenient way for other users to choose if they want to remove their settings or not. Especially as applications hardly document which registry keys they use (and lets not talk about sharewares, which *purposely* leave something behind when you uninstall them).

  65. Daniel Colascione says:

    @Evan You can use LD_LIBRARY_PATH=/opt/yourapp to get equivalent

    behavior, but you're better off using the package manager if it's

    available. rpath is evil because it doesn't allow the search path to

    be modified at runtime; LD_LIBRARY_PATH is safer and more flexible.

    Also, Windows also has a very dangerous policy of loading DLLs by

    default from %PATH% and from the current directory in addition to

    the executable image directory. You can use SetDllDirectory("") to at

    least prevent the current directory being searched; every program

    written or modified today should have a call to SetDllDirectory("") at

    the beginning of main or winmain.

    @SteveWolf

    What do you mean by "hack the DLL"? If you can modify or replace a DLL

    used by an image, you can either modify the image directly, or you are

    the user running the program in the first place. The attacker is

    already on the "other side of the airtight hatchway", as Raymond likes

    to say. Granted, DLL-planting attacks do exist (see my reply to Evan

    above) — but this issue is orthogonal to the security implications

    of dynamic loading itself.

  66. 640k says:

    @Raymond: [Meaningless comparison because there was no 64-bit version of Windows 2000. (And the 64-bit version of Windows XP was never in wide distribution. I'm not sure if it's even serviced.) I think Michael's point is that the disk space usage hasn't changed; the files just moved from the various scattered locations they used to be (so you never noticed them before) to a single place where you can complain about them en masse. -Raymond]

    xp64 was a commercial available product, w2k64 was available on msdn. You know this.

  67. yuhong2 says:

    FYI, XP IA64 no longer gets security updates.

    "(e.g. nobody wanted to do the new work to connect to WU and download different files when installing or uninstalling an LDR)"

    Remember the MS04-xxx IE security updates where the GDR and QFE versions of updates were in different files for everything except Server 2003 and XP SP@?

  68. aaawww says:

    "Increasing the size of each application by a few dozen megabytes is just noise"

    I've invested in 30G raptors, and those are always full just by windows looking at them.

    and yes, it's more a problem of my pc configuration than a windows problem, I'm not blaming windows here.

  69. Engywuck says:

    *my* "ideal"[1] solution would be to have each program in just three files:

    – myawesomeprogramname.prog which would be a zipped version of all the files needed to run the program plus some support structure for exporting dependencies, file associations provided, available sub-exes, icons etc. So essentially something like an extended .jar but non-java :) or packaged programs for streamed apps (like in XenApp or so). This file could/should be read-only for everyone except the Installer service and maybe "Administrator"

    – myawesomeprogramname.sysconf containing all system-wide settings in a zipped file, editable via some configuration program or otherwise. Can contain a hive structure which is added to registry under HKLMSoftwareMyAwesomeProgramName (not in the previous file because it's changeable, but that one could contain a static pre-installation version). Writable only for Administrators, can be moved between systems for easy deployment of always-the-same setting in absence of trusted AD relationships (different forests, test machines, …)

    – myawesomeprogramename.userconf containing all user configurations, located in the users (roaming) app folder. Can contain some (copy of a) hive structure which is mounted in HKCUSoftwareMyAwesomeProgramName . Obviously writable for the user only :D

    The latter two may be created upon installation/first run of the program. All neatly packaged, which is what we all want, right :D

    For shared librarys I could perhaps be convinced to accept some form of .sharedlibs file :)

    [1] i.e. correct for most use cases except drivers etc

  70. Evan says:

    @Daniel Colascione: "You can use LD_LIBRARY_PATH=/opt/yourapp to get equivalent behavior"

    Believe me, I'm well aware of LD_LIBRARY_PATH. I have several scripts sitting around whose purpose is to basically set LD_LIBRARY_PATH so I can use some program. That "solution" sucks. (I think I can replace at least many of those with rpath and ${ORIGIN} now that I know about that.)

    "but you're better off using the package manager if it's available"

    Which it basically isn't in my situations, for a number of reasons.

    "Also, Windows also has a very dangerous policy of loading DLLs by default from %PATH% and from the current directory in addition to the executable image directory."

    Yes, this is a dangerous policy.

Comments are closed.