The normal string manipulation functions stop on a null terminator, so be careful when manipulating double-null-terminated strings


One of the many gotchas of working with double-null-terminated strings is accidentally using functions on them which were designed to operate on single-null-terminated strings. Now, you do need to use those single-null-terminated strings, but you also need to know when they won't do what you want.

One of the responses to my psychic detection that somebody passed a single-null-terminated string to SHFileOperation is, "Oh, no, I double-null-terminated it. Look:

sprintf(szDeletePath, "%s\0", file_to_delete);

See, I put an extra \0 at the end."

Well, yeah, you put an extra \0 at the end, but all that does is terminate the format string. The sprintf function accepts its format string as a traditional null-terminated string. When it sees the \0 you stuck into the string, it thinks it found the end of the string. It can't read your mind and say, "Oh, this null is not a terminator. It's an embedded null."

A simple mistake, but something that needs to be called out. To be fair, most people recognize this mistake as soon as it's pointed out to them. You just have to remember to point it out to them.

Comments (20)
  1. Alexander Grigoriev says:

    This works:

    sprintf(szDeletePath, "%s%c", file_to_delete, 0);

  2. Marquess says:

    I tell ya, null-terminated strings are the works of the devil! double-null-terminated string doubly so (naturally).

  3. Joe says:

    Given the choice between pascal strings with a max length of 256 and null-terminated strings, I’d pick null-terminated. Overall, I’m very glad those are no longer the only options.

  4. Voo says:

    @Joe: Given the number of errors made with C and pascal strings I think I’d probably take pascal strings in 9 of 10 cases.

    Though you’re right, both of them are far from perfect..

  5. ton says:

    These last couple of posts have demonstrated one thing very clearly for a fact and that is  these early API designs were truly horrific and should be discarded as soon as possible. This is what is so positive about the virtual mode in Windows 7 it makes it possible to leave some of the worst kludges in the past where they belong. Yessssssssss!!!!!!!!

    [Good thing SHFileOperation was superseded by IFileOperation years ago, so people can stop using the old API designed for a different era. -Raymond]
  6. njkayaker says:

    I’d guess that the programmer who wrote the example is quite capable of making other non-string related mistakes!

  7. Mircea says:

    A Delphi string is ideal IMO. It is O(1) and how often will you need a string bigger than 4 GB? Now don’t forgot to lobby the ISO C and C++ committees to add a new string type. I’m sure they will.

  8. Peter da Silva says:

    Bah, you need buffer-gap strings. buffer length, string length, gap offset. Insert anywhere, delete anywhere, efficient serial inserts and deletes.

  9. W says:

    Delphi strings have their own share of problems. Most of them related to copy-on-write and ref-counting.

    And passing them from one runtime to another leads to problems too.

    And when casting them to PChar(=char*) you need exactly to know what you are doing, or you cause subtle bugs.

  10. It is my hope that Windows 7 will be a kind of watershed release in regards to backwards compatibility.  With XP mode providing a reasonable option, Microsoft can now be aggressive about doing new stuff completely in .NET, and thus rendering double-NULL-terminated strings a thing of the (distant) past.

    [That’s great for new stuff, but what about old stuff? Are you saying the old stuff should be deprecated and eventually purged from Windows? (What’s going to happen to all the programs that still use SHFileOperation?) -Raymond]
  11. Anonymous Coward says:

    The Windows kernel uses counted strings internally, and OLE uses BSTRs. If Microsoft could make a choice, make all functions use those and implement the old functions with some kind of backward-compatibility layer, we would suffer a lot less string handling issues.

    [That sounds backwards. Shouldn’t it be “Write wrappers around the existing functions which take BSTRs?” Or are you saying that we we should reimplement all functions to use BSTRs internally? Won’t that introduce new bugs? -Raymond]
  12. Billy O'Neal says:

    @Mircea: What do you possibly mean by a Delphi string begin O(1)? What operation are you referring to? Things like appending to an existing string must take at least O(min(m,n)) amortized time to copy the string data around, even in a perfect string implementation (O(m+n) in the worst case that a new buffer needs allocated for both strings).

  13. Marquess says:

    @Joe:

    Because these were the only string types ever invented. Even Windows uses two, although rarely the twain meet. There’s the well known null-terminated string (TCHAR*, LPSTR, lpsz) and the UNICODE_STRING of the Native API. Outside Windows we have Delphi strings (32-bit length counter, i.e. the same as for char* on IA32), Ada’s String, Bounded_String and Unbounded_String (with different trade-offs for flexibility and speed) and quite a lot others (linked lists?). Abstraction is the keyword here. (Again, .NET is doing well in that regard)

  14. Gabe says:

    Raymond, I believe that J. Daniel Smith was trying to say that old programs would run in XP mode. The implication is that in order to run in “Win7 native” mode, old software would need to be rewritten to eliminate deprecated APIs.

    [But this still entails reimplementing all functions which previously used null-terminated strings to use the new string format. Rewriting working code is not something undertaken purely for aesthetic reasons. -Raymond]
  15. Gabe says:

    “[But this still entails reimplementing all functions which previously used null-terminated strings to use the new string format. Rewriting working code is not something undertaken purely for aesthetic reasons. -Raymond]”

    The post I was refering to seemed to only be trying to get rid of double-NULL-terminated-strings. Granted, others do seem to think that all null-terminated strings should be eliminated.

    As for rewriting working code, I think Steve Jobs would disagree that it shouldn’t be undertaken for purely aesthetic reasons. Mac application developers probably have to rewrite working code every few years. I recall when it was announced that planned 64-bit support for the backwards-compatible API (Carbon) would be dropped, meaning that anybody who wanted to release a 64-bit version of their app would have to rewrite the whole UI to use a different API (Cocoa). Surprisingly, lots of people were upset that some software vendors would have to delay 64-bit releases of major apps because they had spent the previous seven years merely adding features and fixing bugs, rather than rewriting millions of lines of code to use a shiny new API.

    [Reimplementation means you replace all the bugs you know with new unknown bugs. Sometimes it’s the right thing to do, but it’s not a decision to be made lightly, and I believe that doing it just for aesthetics is not a strong enough reason. That’s why I don’t run Apple Computer. -Raymond]
  16. Yuhong Bao says:

    "I recall when it was announced that planned 64-bit support for the backwards-compatible API (Carbon) would be dropped, meaning that anybody who wanted to release a 64-bit version of their app would have to rewrite the whole UI to use a different API (Cocoa). "

    MS did some of it too when moving to 64-bit:

    http://blogs.msdn.com/larryosterman/archive/2005/04/22/playing-audio-cds-part-3-mci.aspx#411024

  17. 640k says:

    4 GB strings are enough for everyone. Case closed.

  18. Marquess says:

    Well, these days, strings can actually be 16 exabytes. But strings are probably the wrong type for anything even remotely close to 4 GB.

  19. Kilo says:

    Everything is a string.

  20. Cheong says:

    With XP mode providing a reasonable option,…

    Is it?

    That line basically translates to [those who bought "Home" editions of Windows should dump any old programs when upgrading to Windows 7]…… Doesn’t sound to convincing to me…

    You know, my brother still using AutoCAD 2000 at home, because it’s simply too expensive to buy a new verion to justify it’s cost. He’ll probably be upset if he upgraded and see his program doesn’t work, and fall back to WinXP again…

Comments are closed.