What does the CreateProcess function do if there is no space between the program name and the arguments?


In an old discussion of why the Create­Process function modifies its command line, commenter Random832 asked, "What if there is no space between the program name and arguments - like "cmd/?" - where does it put the null then?"

The Create­Process function requires a space between the program name and arguments. If you leave out the space, then the arguments are considered as part of the program name (and you'll almost certainly get ERROR_FILE_NOT_FOUND back).

It sounds like Random832 has confused Create­Process command line parsing with cmd.exe command line parsing. Clearly the two parsers are different; you can see this even without playing with spaces between the program name and the arguments:

C:\>C:\Program Files\Windows NT\Accessories\wordpad.exe
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.

If the command line had been parsed by Create­Process, this would have succeeded in running the Wordpad program, because, as I noted in the original article, the Create­Process function modifies its command line in order to find where the program name ends and the command lines begin, an example of which can be found in the Create­Process documentation. In this case, it would have plunked the null character into each of the spaces in the command line, finding that each one failed, until it finally tried treating the entire string as the program name, at which point it would have succeeded. The fact that it failed demonstrates that Create­Process didn't do the parsing.

The cmd.exe program permits the space between a program name and its arguments to be elided if the arguments begin with a character not permitted in file names. Once it figures out what you're running, and it determines that what you're running is a program, it call the Create­Process function with an explicit application and command line.

But you don't have to take my word for it. You can just see for yourself. (In fact, this is exactly what I did to investigate the issue in the first place.)

C:>ntsd -2 cmd.exe

Two windows will open, one for your debugger and one for cmd.exe. (You are welcome to replace ntsd with your favorite debugger. I chose ntsd because—at least until Windows XP—it came preinstalled, thereby avoiding multiplying the problem from one to two.)

In the debugger, set a breakpoint on kernel32!Create­ProcessW, then resume execution. In the cmd.exe window, type cmd/?. The breakpoint will fire, and you can look at the parameters:

Breakpoint 0 hit
eax=0046f600 ebx=00000000 ecx=004f8de0 edx=00000000 esi=00000000 edi=00000001
eip=757820ba esp=0046f544 ebp=0046f704 iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
kernel32!CreateProcessW:
757820ba 8bff            mov     edi,edi
0:000> dd esp l4
0046f544  4a5e3dd7 004f5420 004f8db0 00000000
0:000> du 004f5420
004f5420  "C:\Windows\system32\cmd.exe"
0:000> du 004f8db0
004f8db0  "cmd /?"

Observe that cmd.exe did its own manual path search to arrive at an executable of C:\Windows\system32\cmd.exe, and also that it secretly inserted a space between the cmd and the slash.

Comments (25)
  1. laonianren says:

    A corollary: create a text file called "C:Program" and you'll find that various (badly registered) things fail because CreateProcess's search never reaches the executable.

  2. henke37 says:

    Yeah, that is why windows has a warning built in against doing exactly that.

  3. LocalH says:

    I wasn't able to get Win7 to trigger such a warning. Admittedly I use Directory Opus as an Explorer replacement (for purposes of file management) but I explicitly did this in Explorer (both as a directory and as a file) and got no such warning.

    [The warning is raised at logon, not when you create the folder. (If you think about it you'll realize why.) -Raymond]
  4. LocalH says:

    I stand corrected. It makes perfect sense once I mull over it a bit. Thanks, Raymond.

  5. googly says:

    As an aside, CD doesn't require you to enclose a path containing spaces in double quotes (anymore? I seem to recall it didn't use to work).

  6. Adam Rosenfield says:

    That's the problem with the "do what I mean", not "do what I said" command line interpretation.  Yes, it's more user-friendly in general, but it breaks corner cases.

    Programming APIs like CreateProcess should be DWIS — if I ask you to run the program named C:Program with an argument of Filesfoobar.exe, then you darn well better do that.  Then, programmers wouldn't write buggy code in the first place, and there's no longer a need for "You have a file named C:Program" warning at logon.

    User-facing APIs (like Cmd.exe) can be more DWIM, but I'd argue that even a command line should be DWIS, like most *nix shells.

  7. MikeCaron says:

    Adam, in Windows 3.1 when spaces were obviously illegal, why would anyone bother quoting paths? I mean, you don't need to guard against something that's impossible.

    Suddenly, in Windows 95, spaces were legal. However, the old programs (or batch files, or…) don't know anything about this, so why would they add quotes to the paths that were obviously never going to need them?

    These days, it's obvious that paths CAN have spaces. However, it was not always the case.

  8. Adam Rosenfield says:

    @Mike: What's there to quote?  If spaces are illegal, why on earth are you trying to run a program under "C:Program Files"?

  9. Matt says:

    @Adam Rosenfield: Perhaps you're taking user supplied input in the form of a file path, or expanding an environment variable.  In Windows 3.1 %TEMP% couldn't contain spaces, now it can.

  10. Adam Rosenfield says:

    @Matt: Point taken.  Still, it's a failure in the design of CreateProcess, in that it requires you to combine the program name and arguments into one big string which it then parses. If instead if were more like the Unix exec* family of functions, where each the program name/arguments are passed individually (either as separate function parameters or as separate array elements), there's no ambiguity, and no parsing is required. If suddenly user input or an expanded environment variable contains a space, that space just carries on into the argument as expected without any semantic changes.

    But, since we're stuck with decisions made decades ago, we have to deal with it. In the interest of backwards compatibility, making CreateProcess DWIM in this regard was the smart decision, as it avoided breaking a large number of programs/batch files. We can't all live in an ivory tower.

    [Not sure where you got the idea that CreateProcess requires you to combine the program name and options. You are welcome to pass them separately, in which case no parsing is performed. -Raymond]
  11. Adam Rosenfield says:

    [Not sure where you got the idea that CreateProcess requires you to combine the program name and options. You are welcome to pass them separately, in which case no parsing is performed. -Raymond]

    Blargh, I had a brain fart.  I'd temporary forgotten that, since in the context of the discussion, we were only discussing the case where a program makes CreateProcess do the parsing (and therefore the lpApplicationName == NULL case).  That case should not have been allowed in the first place.

  12. Simon Buchan says:

    Passing the program name and options seperately means argv[0] is the first argument though, which breaks pretty much every program that reads the command line.

    [That's why there's a convention governing the command line. -Raymond]
  13. Evan says:

    @Adam: "That case should not have been allowed in the first place."

    Remember that having CreateProcess parse the command line would be a lot less 'harmful' if you are restricted to old 8.3 file names. So when it was introduced, it was fine in that setting.

    @Simon: "Passing the program name and options seperately means argv[0] is the first argument though, which breaks pretty much every program that reads the command line."

    Or you just pass it twice, just like the Unix exec() functions. (A typical call to exec (taken from a library I wrote) looks something like "execlp(debugger, debugger, exename, pid, (char*)NULL);". Note the repetition of 'debugger' — one for the process to exec, one for argv[0].)

  14. cheong00 says:

    @laonianren: I presume that's the reason why I had the "C:Program" folder on my company installed machine.

    @Adam: Remember that when CreateProcess() was introduced, it's been given the role as the replacement for WinExec(). So it has to support that type of parsing at some point or the old applications would be moving a lot more slowly.

  15. cheong00 says:

    Oh, I'm somehow surprised to see my MSDN forum account and the Blogs account combined here. :O

  16. Gabe says:

    While you can argue that the OS shouldn't be in the business of parsing command lines, you then have to look at the alternative: every single app that accepts a command line has to parse it themselves. What are the odds that every single app that attempts to parse a command line does it correctly?

    If you're a Unix programmer, you have to either parse the command line yourself to pass to exec(), or use a shell (maybe via system(3)). If you parse the command line yourself, you'll probably forget to handle quotes, or do it wrong (How do you handle escaped quotes? What about escaped escape characters?). Since you're a Unix programmer, you've probably never executed a file from a path with spaces in it, so it would never even occur to you to test such a thing.

    Should you decide on punting the parsing task off to the shell, now you have two problems. The user has to know to appropriately escape all special characters (piping, redirection, globbing, environment variables, etc.), meaning that not only do they have to know that the command line is being parsed by a shell, but they have to know which one so that they can use the correct escaping rules. Furthermore, you don't get the PID of the program you're trying to run, so you can't easily debug it or kill it if it uses too much CPU. You also can't reliably redirect its stdio because the shell may do its own redirection due to special characters being on the command line. As has been noted here in the past, ShellExecute on Windows has similar problems.

    The nice thing about CreateProcess is that it allows you to pull a command line from the Registry or a config file and just run it. You can reliably find out if the executable even exists, get the PID of your spawned process to debug it, redirect its stdio, or kill it if it takes too long. The quoting rules for your app will be the same as for any app that calls CreateProcess.

  17. Simon Buchan says:

    @Raymond: I was replying to your comment to @Adam, which implied that you *could* do CreateProcess("winword.exe", "mydocument.txt", …);

  18. Ivan K says:

    Hmm. I've been led to think that DOS, or FAT, did support the ability to create file names with spaces, just practically nobody cared, knew about, or used it correctly? (Informal reference: blogs.msdn.com/…/6785519.aspx).

    And at that time a lot of humans would have been still been happily coding Y2K bugs :)

  19. Adam Rosenfield says:

    @Gabe: How many programs are there out there that accept command lines and are not shells of some sort?  The vast majority of which launch other programs do so using fixed or nearly-fixed command lines (usually with a small handful of configurable parameters such as filenames), in which case nobody should be doing any parsing.  It's like parameterized SQL queries — don't construct the query string using primitive string operations which then need to be parsed, leading to potential SQL injection attacks; instead use parameterization to pass in the intended parameters.

    If your program is a shell, then obviously it follows its own parsing/quoting rules.  If your program is not a shell, why is it accepting entire command lines?  The argument is the same for Unix — more likely than not, you know what your command line parameters are, so you can just pass them directly to exec().  Or maybe you're a shell, in which case you parse the command yourself; if you're not a shell, why are you parsing command lines?

    Yes, many Unix programmers often mishandle filenames with spaces in them, but that's a whole other can of worms (http://www.dwheeler.com/…/fixing-unix-linux-filenames.html is a long but good diatribe on the subject).

    "The nice thing about CreateProcess is that it allows you to pull a command line from the Registry or a config file and just run it. You can reliably find out if the executable even exists, get the PID of your spawned process to debug it, redirect its stdio, or kill it if it takes too long. The quoting rules for your app will be the same as for any app that calls CreateProcess."

    Finding out if an executable exists, getting the PID of the spawned process, redirecting its stdio, and killing it have nothing to do with how it parses command lines.  You can do all of those things just as easily with Unix's fork()+exec().

  20. 640k says:

    APIs in DOS/16-bit windows should have been forward compatible with APIs which can handle names including spaces, even though the old APIs didn't support spaces. That's how a *good* API is designed.

    [Then you will be thrilled to know that MS-DOS was well-designed, because it did support spaces in file names at the API level. Another case of asking for a feature that already exists. -Raymond]
  21. Evan says:

    @Raymond, Ivan

    I didn't know that DOS allowed spaces; that makes my statement "Remember that having CreateProcess parse the command line would be a lot less 'harmful' if you are restricted to old 8.3 file names" wrong, because the same issue happened then.

    @Adam: Finding out if an executable exists, getting the PID of the spawned process, redirecting its stdio, and killing it have nothing to do with how it parses command lines.  You can do all of those things just as easily with Unix's fork()+exec().

    Check out the original statement again. Let me restate it. If you want to run a process, you can do two things: fork/exec directly, or go through a shell (e.g. system()).

    If you go fork/exec, *you* have to parse the command line yourself, because exec won't do it for you. This leads to the potential for bugs and inconsistencies.

    If you go through a shell, then those other things come into play more. If you want the PID for the process you're actually interested in, system() doesn't give it to you. Even if you fork then exec the shell yourself, your child process PID will be that of the shell, not of the process you're actually interested in. (Maybe you could explicitly use the shell's 'exec'. That probably would work.)

  22. Worf says:

    @Evan: Don't think of DOS as dealing with filenames with spaces. Think of it as every filename is composed of 8.3 characters (after all, less and they're padded). The restriction is that they cannot start with the deleted file character. Otherwise all APIs took 8.3 filenames only (and handled the padding gracefully should you supply a short string).

  23. Gabe says:

    Adam Rosenfield: Most programs that accept command lines aren't shells. If you are a programmer, your build utility (e.g. make) and your debugger are examples. If you are a Unix user, inetd, sendmail, cron, procmail, and xargs are all examples. If you are just a regular user, then anything that supports MIME types (web browser, email client, news reader) will have to accept command lines for the viewer programs they execute. Do you ever use a program (like git or svn) that has a "Run my preferred editor" feature? If so, it has a command line for your editor. Heck, nowadays even most Unix kernels have to parse command lines to support the #! syntax.

    Of all those examples, xargs is the only one that possibly doesn't need to actually parse a command line string into separate strings to pass to exec. Every other one either needs to pass the command line to a shell or parse it for exec. I've never, ever seen a configuration file format that allows you to specify command lines as separate arguments so that they don't have to be split up for the call to exec.

    In the Windows world, command lines are all over the registry. Any user of HKEY_CLASSES_ROOT will be dealing with command lines. If you register a service, you give the Service Control Manager a command line for your service. No doubt the Add/Remove Programs control panel stores command lines for updating and uninstalling apps. I think that the registry value that specifies your screen saver is the only place where you may only give a file name rather than a whole command line.

    Quite the opposite from you, I can think of very few examples where a program has a nearly fixed command line hardcoded into it such that it would be able to call exec without having to first parse a string into substrings.

  24. Andreas Rejbrand says:

    Although it has never happened to me, a 'well-known' problem in the Swedish version of Windows (at least in older versions of the operating system) is that the "C:Program" folder is opened every time you log in to Windows. This is probably caused by someone trying to open a program in the "C:Program Files" folder or one of its subfolders, which should not even exist in a Swedish version of Windows. (Perhaps Raymond has already covered this topic in his excellent blog, since he — rightfully, I have to say — is interested in Sweden and the Swedish language.) I think that this is no longer a problem, since the directory is now called "Program Files" in every language, although the name of the directory is now translated 'on-the-fly' by Explorer. (I am sorry if I am not 100 % accurate here.)

  25. David Walker says:

    There are still some bad installers that will create a folder called "C:Program" (sometimes putting files there, and sometimes not).

    I tried to install a TeX or LaTeX interpreter recently, and it insisted on being installed to a short folder name directly under C:.  I don't know if this is because the program was ported from another OS, but come on, Windows has allowed spaces in folder and file names for what, 16 or 17 years now?  Bad programming.

Comments are closed.