Hidden compatibility constraints of redirecting program execution via a stub


One of the "obvious" solutions to the issue of how much work you're willing to do to save 68KB of disk space was to replace one of the copies with a stub that launches the other copy.

If you try this obvious solution, you may run into some compatibility issues.

First of all, there are programs which launch Notepad and then wait on the process handle so they can wait until the user closes Notepad. Your stub program cannot just do a Create­Process on the target, because programs which perform a wait will find the wait satisfied when your stub program exits.

Okay, so your stub program has to wait for the real copy of Notepad to exit before it can exit itself.

Once you fix that, you'll find another problem: Programs call Get­Exit­Code­Process to see how Notepad exited. Your stub program therefore cannot just perform an Exit­Process; it has to do a Get­Exit­Code­Process on the real Notepad and pass that exit code to your own Exit­Process.

Once you fix that, you'll find another problem: There are programs which execute a process and then look for windows owned by that process. (Yes, there can be more than one, but Notepad is a simple program that creates only one top-level unowned window.) Those programs will get the process ID of your stub program and be unable to find the Notepad window (since it belongs to the real Notepad program, which has a different process ID). I'm not sure how to fix that one.

Yes, you can write a stub that launches another program, but that solves the "save disk space" problem by introducing other problems.

Remember, even though people are supposed to stick to documented behavior (since that is all that is contractual), in practice any implementation detail becomes a compatibility constraint.

Comments (20)
  1. Henke37 says:

    Was this before or after it was possible to load an executable as a dll?

  2. Dan Bugglin says:

    You could always do that, but the executable has to expect to be loaded as a DLL, and have the proper exports etc.

  3. Skyborne says:

    I note that the suggestion was from "smug linux user", who lives in a world where you call exec() and it replaces your process without exiting, which avoids many problems.  AFAICT, the only one left is that if the real notepad is missing, the stub has to exit itself with some sensible error code, which may not exist.

    Providing an exec-like call on Windows would also be a horrible "solution" because that breaks the semantics of process handles as a process's identity.

  4. dave says:

    Providing an exec-like call on Windows would also be a horrible "solution" because that

    breaks the semantics of process handles as a process's identity

    I don't see how.  If the process does a 'exec-like' call, then it's still the same process and therefore the existing process handle still refers to the same process.  If it were a different process, then we could scarcely refer to the hypothetical operation as an 'exec-like' call.

    I think you're confusing 'process' and 'program'.  (The same confusion that thinks 'notepad.exe' can be considered as the name of a process).

  5. kinokijuf says:

    @Skyborne: There's no such thing as nonexistent notepad.

  6. Alex Grigoriev says:

    Compared to the hoops that Microsoft had to jump through to make legacy 16-bit InstallShield executables to work on 64 Windows, this Notepad issue is a piece of cake. I wonder if there is a good writeup for that.

  7. Alex Grigoriev says:

    @The mazzter:

    No, an EXE can't be loaded as an executable DLL. It can be loaded to access its resources, but its relocations will be ignored, and imports not resolved. In other words, it's a cold piece of data.

  8. Joshua says:

    In this particular case, hardlink!

    But yeah, the general case is messy.

  9. Anonymous Coward says:

    I ran into exactly this problem once. I ended up giving the stub a weird base address and loading the executable as a DLL, and then it turned out that one of the DLLs it needed was located somewhere else under a different name, so I had to do my own imports too. But I got it to work all the same.

  10. Cesar says:

    I just did the same thing: a stub program which launches another program. In my case, the very small stub program (it does not even link to the C library) runs from the Startup folder, checks if the "run automatically on startup" option for my program is set on the registry, and if it is set does a CreateProcessW() for the main program (passing it the STARTUPINFOW it received). It does not wait for the main program to exit.

    I did it that way to be polite and use the minimum possible amount of resources if the "run automatically on startup" option is not set. I do not think the code in Explorer which runs the Startup menu entries waits for the launched program to exit or has any dependency on its return code (the main program is a system tray icon, so there is no main window). And yes, if this were Unix, I would have used exec() (I am a Linux user too – perhaps the line of thinking which leads to these stub executables comes from being used to having exec() available?).

    I am now wondering if I should modify my stub to wait for the main process (ending up using a little more resources if the "run automatically on startup" option is set).

  11. Stuart says:

    I actually had a problem with this with Visual Studio. Our company were using Visual Studio .NET 2003 and I'd written a tool which, for various reasons (actually, to hack around other limitations of what I was able to figure out how to do programmatically with the SourceSafe of the day) would open up visual studio to a particular solution with instructions to the user to perform a particular operation manually, then exit.

    My wrapper script would launch (with shell exec) the .sln file, then wait for the process to exit and then continue.

    That worked fine until VS2005 came out and .sln became associated with "Visual Studio Version Selector", a stub exe that looks in the sln file for the appropriate visual studio version and then launches it – and immediately exits. So now my program would attempt to continue immediately without giving the user time to do what they'd been asked to do…

    I was very glad that I was able to replace that hacky script with a pure .NET executable when Team Foundation Server came out!

  12. Copi says:

    copy notepad.exe from 32-bit windows and use it. It's only 70kb.

  13. Neil says:

    @Cesar: Or you could just make the "automatically run on startup" option in your program add or remove the link in the Startup folder. (Or forget about links and just write the HKCU…Run registry key.)

  14. Cesar says:

    @Neil: the problem is that it runs on startup by default, and the installer for the obvious reasons can only write to either the All Users or the HKLM key.

  15. Mike Dimmick says:

    @Alex Grigoriev: It's been a while, but as far as I recall it *is* possible to load an EXE with LoadLibrary{Ex}, if the relocations remain in the EXE. The default setting for the linker is to strip the relocations when building an .EXE though.

    Windows will not load any PE file – EXE or DLL – with relocations stripped if it can't load at its default address. The LOAD_LIBRARY_AS_DATAFILE flag is there to tell Windows that it's OK to ignore the absence of the relocation information, because you're only going to look at the image.

    It's possible that if the EXE has an atypical base address, it could still be loaded even if the relocations are stripped – if it doesn't clash with the initial image.

  16. Crescens2k says:

    @Copi:

    Then you have to deal with the hidden compatability constraints of just taking notepad from Windows. That may work for earlier versions, but if you forget to take the localised mui file from newer versions then it will fail. One compatability restraint being replaced by another. o/

  17. Cesar says:

    After thinking for a while, we decided to change the program to add and remove itself from the HKCU…Run key, launch it after setup (if requested by the user) with a parameter to add itself to the HKCU…Run key (for the current user only), and removed the stub.

    Now the "run automatically on startup" flag is the presence of the entry on the HKCU…Run key, the whole process became much simpler and cleaner, and it still runs on startup by default (for the current user only, and only if the user did not uncheck the checkbox in the last page of the installer). And now if "run automatically on startup" is unset, it uses even less resources than before.

  18. Yuhong Bao says:

    BTW, forcing a program to launch using a debugger via IFEO would have the same problems, as it works in a similar way.

  19. Dogfood says:

    Would a lnk file suffice? On this blog I've repeatedly been told lnk files are better than real (soft/hard) links. I think the reason was that lnk files can be emailed.

  20. Clearly, the neat answer would be for Notepad to be a DLL, with notepad.exe being a stub which just loads that and calls NotepadMain from the DLL. Of course, as a performance enhancement, the next version will be static-linked to that library instead of using it as a DLL…

    (That brought to mind a discussion a few years ago, which started with "this recursive function is putting too much on the stack, I'm changing it to allocate this structure on the heap instead", which made someone ask about performance impact, bringing the reply "instead of the heap, we can use alloca, that's faster"…)

Comments are closed.

Skip to main content