Woe unto PROGMAN.INI


Sad but true: Once you document a file format, it becomes a de facto API.

The Windows 95 team learned this the hard way when they set out to replace Program Manager with Explorer. Not only were the settings in the PROGMAN.INI file documented, so too was the binary file format of *.GRP files. The binary file format was included for diagnostic purposes: If you have a corrupted GRP file, you can use the binary file format documentation to try to recover what you can out of it.

But many people treated this documentation not as a FYI, but as a backdoor API. Instead of using the formal DDE interface for creating program groups and icons, they just directly edited the PROGMAN.INI file and the applicable GRP files to get the icons and groups they wanted.

Oh wait, and then you need to reboot in order for the backdoor API to take effect, because all you did was modify the on-disk files, not the in-memory copy held by PROGMAN.EXE.

Of course, when Windows 95 replaced Program Manager with Explorer, these programs found themselves modifying the data files of a program that no longer was running. Special code had to be added to Explorer to read settings from PROGMAN.INI and even detect that a new GRP file was added and convert it into shortcuts on the Start menu.

I wouldn't be surprised if that code is still lying around, just in case somebody pulls out an old application from 1994 and installs it.

Comments (17)
  1. John says:

    I am dealing with a similar situation.  We use a 3rd-party component consisting of a user-mode library and a kernel-mode driver.  The library deploys the driver not via the SCM interface, but by writing the necessary registry entries and calling NtLoadDriver.  The problem is that the in-memory service table doesn’t get updated and you can’t manage the service until you reboot.  Idiots.

  2. dave says:

    Oh wait, and then you need to reboot

    in order for the backdoor API to take effect,

    Clearly, progman should re-write the files from its in-memory version during its shutdown sequence.

    ;-)

  3. The library deploys the driver not via the SCM interface,

    but by writing the necessary registry entries and calling NtLoadDriver.

    That’s clearly a case of having to work thrice because you didn’t read the documentation and reinvented the wheel. SCM has a simple API that can make most things with just a handful of calls: it took me just one afternoon to code in Visual Basic the equivalent to the Services MMC snap-in, including a couple of classes that expose the registered services in nice objects :-) . I wonder how much time did those guys spend in coding and testing the code against the different target platforms…

  4. someone else says:

    We *really* need a way to deliver electric shocks through the keyboard …

  5. Scott says:

    I’m afraid even undocumented files become unofficial APIs. Just look at the people who reverse engineered bits of the excel and word file formats before there was official documentation. Caused all sorts of odd issues for the office team.

    At a certain point you either have to embrace their hackery or break backward compatibility.

  6. Cheong says:

    I’d say how dare they killed the functionality of AUTOEXEC.BAT before killing the code for reading PROGMAN.INI / GRP files, considering the number of application using these "documented features". :P

  7. Worf says:

    Luckily, support for it doesn’t have to around much longer. 16-bit apps won’t work if you’re running in x64 Windows – x64 mode disables everything but 32-bit protected…

  8. Colin says:

    I wouldn’t be surprised if that code is still lying around, just in case somebody pulls out an old application from 1994 and installs it.

    See KB 119941.  GRPCONV.EXE is still shipped in Windows 7.

  9. Ian Ellison-Taylor says:

    GrpConv!! I remember it well :)

    There was one app that both wrote the binary files directly as well as used DDE. I could only guess that two different devs worked on the app and didn’t talk to each other.

  10. ender says:

    Luckily, support for it doesn’t have to around much longer. 16-bit apps won’t work if you’re running in x64 Windows – x64 mode disables everything but 32-bit protected…

    Don’t worry, I’m sure there are 32-bit programs that still use this backdoor.

  11. Anonymous Coward says:

    Maybe the team that documented the GRP/ini file formats should have provided a dire warning not to use it like that and a working sample of how group creation and modification should be done.

    [You forget that back in the time the documentation was written, it was assumed that programmers were smart. -Raymond]

    @James: So you’re saying that shoehorning driver loading onto the for this purpose ill-fitting registry API, with registry keys that are just as arbitrary as any new functions in a brand new API, is a good idea? I’m glad you’re not on the team that gave us SCM.

    And a little tip: it really is faster to use the proper API to do things, even if you have to look it up, than it is to look up the registry entries/file names/what have you, and come up with a half-working hack.

  12. Alexandre Grigoriev says:

    I suggest the ultimate approach to weeding the hacks. To use any new features of the OS, the executable should be marked as compiled for the OS version. Then, all compat hacks get also disabled.

    So there is the choice for the app programmer: Want new features – remove the hacks.

    [We’ll have to delete all the articles that say “It’s easy to add this feature to your program,” then. Because the implied zeroth step is “Fix all the existing places in your program that rely on old behavior. (Good luck finding them all. Come back here after a few months when you think you got them all. Oh wait, your program links to a DLL that hasn’t been upgraded? Then I guess you’ll never be able to use this feature.)” -Raymond]
  13. "Once you document a file format, it becomes a de facto API."

    Yes – moreover, fail to document it, it becomes a de facto undocumented API. As Scott points out, failing to document something won’t stop people using it – but worse than that, it’ll make it harder for them to use and more likely they use it wrongly, meaning more breakage and/or more compatibility headaches in future.

    If I’d been given a straight choice between creating a GRP file and editing an INI file or jumping through some DDE hoop, I’d go the file route every time and twice on Sundays. Antonio, I’d say the "simple" SCM API is still far more work than creating a few registry entries, because there’s no new API to learn at all there! Much better to have a single operation which makes services.exe re-read the registry data after a change (or have it use change notification, meaning no extra work at all) rather than introduce yet another new pile of functions and structures. The end result in both the PROGMAN case and Services is that there are now *two* ways of doing something which need to be supported forever, instead of just one!

    "someone else": Yes, and that feature should be activated every time someone thinks of introducing yet another new API instead of using an existing approach :-)

  14. @AC: I see nothing "ill-fitting": you’re adding a few pieces of data to a list of services, which is precisely the intended usage of the Registry and indeed precisely what service.exe will be doing for you in your own route. The only drawback of any kind is services.exe’s apparent inability to update its internal state to reflect updates, which is a trivial implementation flaw. Not naming names, this also seems to be the route taken (and explained and boasted about as a feature!) by at least one popular MS tool  Then there’s the .INF method, which is essentially a direct mapping onto the Services registry keys – and AFAICS, these are very much an officially approved way of installing, not to mention the way most of MS’s own drivers get installed…

    Your "tip" seems rather implausible when merely finding the right documentation for yet another single-purpose API will take longer than figuring out the handful of registry values to insert – and that’s before taking into account the fact the API may very well not be directly available in the environment you’re in (an MSI or installation script, for example) – meaning now you’ve got another hurdle, and need to resort to a custom action or process invocation, rather than using your installer’s built-in registry functionality. Moreover, a quick Google for blog posts from people struggling to jump through this very hoop in their installers seems to suggest you’re wrong about the API being such a wonderful option.

  15. Different Anonymous Coward says:

    [You forget that back in the time the documentation was written, it was assumed that programmers were smart. -Raymond]

    Shame Raymond you don’t have to work with the programmers I have too.  I swear the universities are just rubber stamping CS degrees.

  16. Dan says:

    Cheong: Considering Windows is no longer DOS-based, AUTOEXEC.BAT support was never really killed… NT was built from the ground up without it!

    Nowadays they are only used by NTVDM when running DOS or 16-bit apps AFAIK, I think there’s still a few lines you can stick in there to fix some compatibility issues or whatever.

  17. Me says:

    Regarding SCM — it is much slower and more cumbersome to use than the NtLoadDriver and that is the reason why people are avoiding it.

    Same reason must have been with DDE .vs. GRP/INI — file API is something everyone knows while DDE has learning curve.

    It is a shame that nowadays you have to invoke COM and suffer its slowness and bloat just to create a damn shortcut.

Comments are closed.

Skip to main content