When you want to pass a parameter on the command line, don’t forget to pass the parameter on the command line


This happens to me, too. I once got so distracted by the process of purchasing some tickets in person, choosing the performance date and the seats, fumbling for my credit card, signing the receipt, and when I was done, I left the ticket booth without my tickets!

Here is a question that came in from a customer. The details have been changed but the underlying point is the same.

According to the documentation, the taglog command prints a history log of all the tags in the system. I can pass the -d option to limit the date range, and that works too. It also says that that I can pass a specific tag name to limit the listing to just that one tag, but I can't get that part to work. Here's what I'm doing:

for /f %i in (taglist.txt) do taglog -d 2008/05/01,2008/05/31 >> output

But this just dumps the history of all tags during the month of May, not the tags I asked for.

This customer got so excited about passing the date range filter that he forgot to pass the tag name.

for /f %i in (taglist.txt) do taglog -d 2008/05/01,2008/05/31 %i >> output
Comments (26)
  1. Nawak says:

    Raymond, it seems that you have posted the wrong link for the taglog command.

  2. bahbar says:

    @Nawak: I would not be surprised if it was intentional. Slang words coming from tagalog was pretty cool to read about, and the point Raymond is making does not need any understanding of the actual taglog program (which is likely in the “details have been changed” category)

    [Yup. -Raymond]
  3. Maurits says:

    This happens to me all the time.  My first troubleshooting step whenever I get unexpected output from a "for" command is to verify what I’m actually doing inside it:

    for /f %i in (taglist.txt) do ECHO taglog -d 2008/05/01,2008/05/31

    (Hmm, all those commands look the same… d’oh!)

    Actually to be honest I usually do the ECHO version first, before even trying the command "for real."

  4. someone else says:

    And then there’s typing %1 (percent one) instead if %i …

  5. Do you want “%i” with quotes around it in case the entries contain spaces, or does windows do that for you?

    (I’m coming from the unix world where quotes would generally be desirable in that situation)

    [Presumably there is some external knowledge that says that none of the entries will contain spaces or shell metacharacters. -Raymond]
  6. someone else says:

    @Stuart:

    No, Windows does not do that. It may not be desirable (e.g. when outputting to a file). There is, however, a way to get the short name (%~si instead if %i). What’s a short name, you ask? Excellent question.

    PowerShell doesn’t have that problem, due to using objects instead of plain text.

  7. Teo says:

    The CRT team has a similar problem, they got so excited about reading environment variables and building command lines, that at the end forgot to put the correct executable name :) Try calling _wsystem() when the command processor resides in a directory with a space in its name/path.

  8. ParameterPasser says:

    I want to pass the /nobackup parameter to wusa.exe and pkgmgr.exe however how much excited I get, I can’t because it’s not supported.

  9. Mark says:

    someone else: that’s why you use %a in for loops.  It also means that for /f expands to %a, %b, %c, etc.

  10. Anonymous Coward says:

    In Perl, a lot of commands take a missing argument to mean $_, which in turn is set by loops. Which means that you can have for example a "print;" statement inside a loop. I’ve never decided whether I think that’s a good thing; I left Perl for different reasons.

  11. someone else says:

    @Mark:

    It’s not really that much dependent on the variable name. It’s just muscle memory: "This is a percent sign, which denotes a parameter, so it’s followed by a number … damn …"

  12. porter says:

    > Do you want "%i" with quotes around it in case the entries contain spaces, or does windows do that for you?

    UNIX knows about command line arguments, Windows doesn’t. All CreateProcess() does is pass a single string, it’s up to the target application to parse them.

  13. TK says:

    @Stuart:

    "%~i"   (use quotes, the ~ removes them from the value if necessary)

  14. Worf says:

    @porter: Actually, there are several entry points to a Windows program. You have classical WinMain, wWinMain (instead of LPCTSTR, it’s LPCWSTR – wide string). You also have main() and the corresponding wmain (also taking wide strings). The latter two entry points rely on the Windows CRT to do the argument splitting.

    [There is only one true entry point, which is called with no parameters. Everything else is runtime support. “It’s up to the target application to parse them” is correct, since the runtime is part of the application. -Raymond]
  15. someone else says:

    This is a lot more fun if $FOO=-rf /

    Much fun will be had by all.

  16. Karellen says:

    @Evan:

    Shells tend to do that because it allows you to do things like:

    $ export CFLAGS="-Wall -O1 -g"

    $ cc $CFLAGS -o foo foo.c

    If the shell auto-quoted variable expansions, that sort of thing would simply not be possible. Instead, the way it does it, it is at least possible to do things the other way with:

    $ export fname="awkward filename.c"

    $ cc $CFLAGS -o foo "$fname"

    You, as the user of the variable know whether it’s meant to expand to a single arg or many, so can quote it or not as appropriate, in exactly the same way as you quote (or not) all the other args to the command.

  17. Evan says:

    @porter:

    What you say is fairly irrelevant; the question was pointing out that even though the Unix kernel may know about command line arguments, Unix *shells* somewhat act like *they* don’t.

    In particular, if an environment variable FOO’s value contains spaces and you use $FOO on the command line unadorned (e.g. ‘rm $FOO’) then the target command will receive each token as a separate argument. If you want to make sure that it’s passed as one argument, you need to quote the environment variable occurrence. The alternative would be to implicitly put quotes around it.

    I’m not sure which makes more sense.

  18. Karellen says:

    So you’re saying that people would be capable of remembering to make the distinction between $FOO and %FOO, in cases where they are incapable of doing so between $FOO and "$FOO", even though this is almost identical to the distinction they already make between ‘cat foo bar’ and ‘cat "foo bar"’?

    I’m not so sure.

  19. Random832 says:

    What about arguments that contain embedded quotes? I know CommandLineToArgvW uses backslashes (in an odd way, ignoring backslashes that don’t precede a quote mark) and some programs use doubled quote marks. Is there any way in cmd to quote a variable and do escaping in this way?

    At least filenames can’t contain quotes.

  20. Evan says:

    @Karellen:

    I understand why they do that, but there are also cases where it’s equally clear that having it treated as one argument makes sense. If you have something like “for FILE in *; cp $FILE /some/other/place” it would be pretty remarkable if the programmer DIDN’T intend for the $FILE arguments to be passed as one. Sure, the programmer *could* have quoted it to make it work, but many don’t, which leads to a lot of not-very-well-vetted scripts that break when you have spaces in your paths.

    Now, it’s (quite) possible that the current way is the right way, since it is strictly more general, but at the same time there are a couple middle-ground options you could take too. (Though again, I’m not sure which would be the best option in practice.)

    Two such middle ground options would be to have different ways of inserting an environment variable… for instance $FOO inserts it as one argument, and %FOO inserts it as two. Another would be to have things like ‘for’ that generate things that likely should be used as one argument insert the quotes that will eventually cause them to be grouped together.

    [$FOO/%FOO could work. Auto-quoting in “for” wouldn’t though: for /f “delims=” %i in (commandstorun.txt) do %i would be impossible if %i were auto-quoted. -Raymond]
  21. someone else says:

    “At least filenames can’t contain quotes.”

    On unixoid operating systems, they can. In fact, they can contain anything except slash and NUL.

  22. Random832 says:

    Yeah well on "unixoid" operating systems you don’t have to mess around with quoting and escaping to pass any string to execve()

  23. porter says:

    > The alternative would be to implicitly put quotes around it.

    > I’m not sure which makes more sense.

    It doesn’t make sense for it to implicitly make the wrong guess. The shell doesn’t know if your variable with spaces is supposed to contain a list of things, or a single thing with spaces in. Hence it doesn’t try and second guess you.

  24. Evan says:

    Okay, so this was a little more discussion than I anticipated on this topic; sorry for bringing it off course (as I go ahead and fan the flames more).

    @Raymond: "Auto-quoting in "for" wouldn’t though"

    Your example is a good one; I didn’t think of that. (When I do a for loop in a shell it’s almost something like ‘for FILE in *.foo’.)

    @Karellen

    I envision the "autoquote" variant would be sort of the default go-to case, and then the separate tokens version would be used rather less frequently.

    I SSHed into my "work" (I’m a grad student) Linux box to see what environment variables I have set. There are 39 of them. 36 of them I’d want to be treated as a single argument if I were to pass it to a command for some reason. 20 of them are a path or :-separated list of paths, and these are ones where a space could legitimately show up. By contrast, of the remaining three, two of them I’m not sure about (these are $SSH_CLIENT and $SSH_CONNECTION), and only one out of the 39 environment variables would I actually want to be separate tokens.

    The story is similar on the Windows box I’m posting from. 42 environment variables in total. 23 are paths or a collection of paths, many of which in actuality DO contain spaces (e.g. Program Files). In addition, there is at least one more environment variable where spaces make sense (because it has them currently), %PROCESSOR_IDENTIFIER%.

    Finally, back on my Linux box I took a look at my command history. I store 10,000 lines of history, and grepped through it for uses of $. There are 130 such commands. To avoid bias because of some dumb things about the way I have things set up, it’s probably better to count only unique commands, of which there were 71. (I’m pretty sure counting all would strengthen my case BTW, because many of the things I’m excluding are several cases of putting stuff into $LD_LIBRARY_PATH.) Those cases break down as follows: 29 commands where $ didn’t have anything to do with environment variables (grep was a common case, and $(command) falls here), 15 cases where the "autoquoting" wouldn’t matter (almost all because I was just doing ‘echo $FOO’), 4 cases where I’m not sure where they go, and 20 cases where an environment variable is a path, filename, or :-separated list of paths where you’d want it treated as one argument (9 times for ‘configure –prefix=$HOME/.packages’, 5 times to add a path to LD_LIBRARY_PATH, 3 for loops, and 3 misc. other cases). In *no* command in the last 10,000 I’ve run on that computer from an interactive shell do I even think there’s a reasonable chance I have intended an environment variable to be treated as separate arguments.

    (Of course, there are other situations where this is not true, programming Make/SCons scripts being probably the main one.)

    I of course don’t know that my experience is typical, but I do suspect that "treat the environment variable as one argument" is by far the common case. And it seems wrong to me to make the *common* case require extra work (adding quotes) and the *uncommon* and more error-prone case be the default behavior.

    @porter:

    I think this is pretty similar to one of the big distinctions between Unix and Windows (even though both behave the same in this regard). The Unix approach a lot of the time is "we don’t know for sure what the user intends, so it’s better to be easily predictable even if we’re wrong most of the time" while Windows will try to help the user even if it means things are less predictable. (For an example, see one of Raymond’s relatively recent posts about how if Windows detects you turning the screensaver off very soon after it comes on, it will start to increase the idle delay.)

    I sort of view the shell as already making such a decision for me about how I want $FOO treated, so I don’t see trying to make that decision more intelligent as a necessarily bad thing.

    (Now, all that said, yes, I think that if you had to choose between just "treat $FOO as one argument" and "treat $FOO as potentially many arguments", the latter is probably the right choice because it’s strictly more general. I’m just trying to say that I wish that shell designers had put more thought into it and chose some middle ground. I’ve had too many headaches caused by trying to run scripts written for Unix, where spaces are uncommon, under Cygwin on a platform where for a while your equivalent of /home/ was /documents and settings/.)

  25. Mark says:

    Evan: it’s not "strictly more general", forcible quoting would make building a string of command line options impossible without a way of stripping quotes.  $FOO/%FOO might work, but then what would you do about !FOO! ?

    I’m certain the trouble you’re having with Cygwin is exactly why Microsoft chose to have "Documents and Settings" and "Program Files": to force coders to sort their stuff out.

Comments are closed.

Skip to main content