This isn’t Highlights magazine: Sort keys and why they change


Some time ago, Ry Jones gave some examples of Quotable Raymond, including the following:

How to make a good doc bug report:

1. Don't embed pictures. ... This isn't Highlights magazine.

What Ry didn't realize is that his "..." totally misrepresented the message. There were actually two separate items, but he combined them into one and replaced the missing parts with "...". I already ranted some time ago about embedding pictures. Today I'll rant about the second item, which is turning a bug report into the Spot the difference between these two pictures game in Highlights magazine.

A customer reported that the LCMap­String function was returning incorrect results on Windows 7. To illustrate, they included a sample program with the Windows XP and Windows 7 output.

#include <windows.h>

char bad[] = "... a very long string...";
 
char dest[10000];
 
int main()
{
    int nChr, i;
 
    nChr = LCMapStringA(1033,  /* English (US) */
                  LCMAP_SORTKEY,
                  bad,
                  -1,    /* source is NULL terminated */
                  dest,  /* destination */
                  sizeof(dest));  /* dest size */
    /* print out returned number of bytes as hexadecimal */
    for (i = 0; i < nChr; i++)
        printf("%02x ", (unsigned char)dest[i]);
    printf("\n");
}

On Windows XP, we get the correct sort key:

0e 48 0e 7c 0e 8a 0e 21 0e 51 07 02 0e 32 0e 7e 0e 91 0e 9f 0e 51 07 02 0e 1a 0e 7c 0e 48 0e 7c 0e 8a 07 02 0e 91 0e 32 0e 99 07 02 0e 02 0e 51 0e 21 0e 99 07 2f 07 02 0e 0a 0e 7c 0e 70 0e 91 0e 21 0e 0a 0e 99 0e 21 0e 99 0e 9f 0e 8a 07 02 0e 02 0e 1a 0e 32 0e 7e 0e 32 0e 91 0e 32 0e 0a 0e 32 0e 70 0e 25 07 02 0e 21 0e 48 0e 32 0e 99 07 2f 07 02 0e 91 0e 21 0e 1a 07 02 0e 1a 0e 7c 07 02 0e 21 0e 32 0e 9f 0e 91 0e 51 0e 7c 0e 1a 07 02 0e 99 0e 21 0e 51 0e 7e 0e 7c 0e 8a 07 02 0e 32 0e 70 0e 0a 0e 32 0e 1a 0e 32 0e 1a 0e 9f 0e 70 0e 99 07 02 0e 9f 0e 99 07 02 0e 48 0e 02 0e 09 0e 7c 0e 8a 0e 21 07 02 0e 21 0e 99 07 02 0e 1a 0e 7c 0e 48 0e 7c 0e 8a 0e 21 07 02 0e 51 0e 02 0e 25 0e 70 0e 02 07 02 0e 02 0e 48 0e 32 0e 89 0e 9f 0e 02 07 33 07 02 08 14 07 02 0e 9f 0e 99 07 02 0e 21 0e 70 0e 32 0e 51 07 02 0e 02 0e 1a 07 02 0e 51 0e 32 0e 70 0e 32 0e 51 07 02 0e a2 0e 21 0e 70 0e 32 0e 02 0e 51 07 2f 07 02 0e 89 0e 9f 0e 32 0e 91 07 02 0e 70 0e 7c 0e 91 0e 99 0e 8a 0e 9f 0e 1a 07 02 0e 21 0e a6 0e 21 0e 8a 0e 0a 0e 32 0e 99 0e 02 0e 99 0e 32 0e 7c 0e 70 07 02 0e 9f 0e 48 0e 48 0e 02 0e 51 0e 0a 0e 7c 07 02 0e 48 0e 02 0e 09 0e 7c 0e 8a 0e 32 0e 91 07 02 0e 70 0e 32 0e 91 0e 32 07 02 0e 9f 0e 99 07 02 0e 02 0e 48 0e 32 0e 89 0e 9f 0e 32 0e 7e 07 02 0e 21 0e a6 07 02 0e 21 0e 02 07 02 0e 0a 0e 7c 0e 51 0e 51 0e 7c 0e 1a 0e 7c 07 02 0e 0a 0e 7c 0e 70 0e 91 0e 21 0e 89 0e 9f 0e 02 0e 99 07 33 07 02 0e 1a 0e 9f 0e 32 0e 91 07 02 0e 02 0e 9f 0e 99 0e 21 07 02 0e 32 0e 8a 0e 9f 0e 8a 0e 21 07 02 0e 1a 0e 7c 0e 48 0e 7c 0e 8a 07 02 0e 32 0e 70 07 02 0e 8a 0e 21 0e 7e 0e 8a 0e 21 0e 2c 0e 21 0e 70 0e 1a 0e 21 0e 8a 0e 32 0e 99 07 02 0e 32 0e 70 07 02 0e a2 0e 7c 0e 48 0e 9f 0e 7e 0e 99 0e 02 0e 99 0e 21 07 02 0e a2 0e 21 0e 48 0e 32 0e 99 07 02 0e 21 0e 91 0e 91 0e 21 07 02 0e 0a 0e 32 0e 48 0e 48 0e 9f 0e 51 07 02 0e 1a 0e 7c 0e 48 0e 7c 0e 8a 0e 21 07 02 0e 21 0e 9f 07 02 0e 23 0e 9f 0e 25 0e 32 0e 02 0e 99 07 02 0e 70 0e 9f 0e 48 0e 48 0e 02 07 02 0e 7e 0e 02 0e 8a 0e 32 0e 02 0e 99 0e 9f 0e 8a 07 33 07 02 0e 21 0e a6 0e 0a 0e 21 0e 7e 0e 99 0e 21 0e 9f 0e 8a 07 02 0e 91 0e 32 0e 70 0e 99 07 02 0e 7c 0e 0a 0e 0a 0e 02 0e 21 0e 0a 0e 02 0e 99 07 02 0e 0a 0e 9f 0e 7e 0e 32 0e 1a 0e 02 0e 99 0e 02 0e 99 07 02 0e 70 0e 7c 0e 70 07 02 0e 7e 0e 8a 0e 7c 0e 32 0e 1a 0e 21 0e 70 0e 99 07 2f 07 02 0e 91 0e 9f 0e 70 0e 99 07 02 0e 32 0e 70 07 02 0e 0a 0e 9f 0e 48 0e 7e 0e 02 07 02 0e 89 0e 9f 0e 32 07 02 0e 7c 0e 23 0e 23 0e 32 0e 0a 0e 32 0e 02 07 02 0e 1a 0e 21 0e 91 0e 21 0e 8a 0e 9f 0e 70 0e 99 07 02 0e 51 0e 7c 0e 48 0e 48 0e 32 0e 99 07 02 0e 02 0e 70 0e 32 0e 51 07 02 0e 32 0e 1a 07 02 0e 21 0e 91 0e 99 07 02 0e 48 0e 02 0e 09 0e 7c 0e 8a 0e 9f 0e 51 07 33 01 01 12 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 12 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 12 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 12 01 01 81 fb 06 82 00

but on Windows 7 the output is wrong:

0e 48 0e 7c 0e 8a 0e 21 0e 51 07 02 0e 32 0e 7e 0e 91 0e 9f 0e 51 07 02 0e 1a 0e 7c 0e 48 0e 7c 0e 8a 07 02 0e 91 0e 32 0e 99 07 02 0e 02 0e 51 0e 21 0e 99 07 2f 07 02 0e 0a 0e 7c 0e 70 0e 91 0e 21 0e 0a 0e 99 0e 21 0e 99 0e 9f 0e 8a 07 02 0e 02 0e 1a 0e 32 0e 7e 0e 32 0e 91 0e 32 0e 0a 0e 32 0e 70 0e 25 07 02 0e 21 0e 48 0e 32 0e 99 07 2f 07 02 0e 91 0e 21 0e 1a 07 02 0e 1a 0e 7c 07 02 0e 21 0e 32 0e 9f 0e 91 0e 51 0e 7c 0e 1a 07 02 0e 99 0e 21 0e 51 0e 7e 0e 7c 0e 8a 07 02 0e 32 0e 70 0e 0a 0e 32 0e 1a 0e 32 0e 1a 0e 9f 0e 70 0e 99 07 02 0e 9f 0e 99 07 02 0e 48 0e 02 0e 09 0e 7c 0e 8a 0e 21 07 02 0e 21 0e 99 07 02 0e 1a 0e 7c 0e 48 0e 7c 0e 8a 0e 21 07 02 0e 51 0e 02 0e 25 0e 70 0e 02 07 02 0e 02 0e 48 0e 32 0e 89 0e 9f 0e 02 07 33 07 02 08 14 07 02 0e 9f 0e 99 07 02 0e 21 0e 70 0e 32 0e 51 07 02 0e 02 0e 1a 07 02 0e 51 0e 32 0e 70 0e 32 0e 51 07 02 0e a2 0e 21 0e 70 0e 32 0e 02 0e 51 07 2f 07 02 0e 89 0e 9f 0e 32 0e 91 07 02 0e 70 0e 7c 0e 91 0e 99 0e 8a 0e 9f 0e 1a 07 02 0e 21 0e a6 0e 21 0e 8a 0e 0a 0e 32 0e 99 0e 02 0e 99 0e 32 0e 7c 0e 70 07 02 0e 9f 0e 48 0e 48 0e 02 0e 51 0e 0a 0e 7c 07 02 0e 48 0e 02 0e 09 0e 7c 0e 8a 0e 32 0e 91 07 02 0e 70 0e 32 0e 91 0e 32 07 02 0e 9f 0e 99 07 02 0e 02 0e 48 0e 32 0e 89 0e 9f 0e 32 0e 7e 07 02 0e 21 0e a6 07 02 0e 21 0e 02 07 02 0e 0a 0e 7c 0e 51 0e 51 0e 7c 0e 1a 0e 7c 07 02 0e 0a 0e 7c 0e 70 0e 91 0e 21 0e 89 0e 9f 0e 02 0e 99 07 33 07 02 0e 1a 0e 9f 0e 32 0e 91 07 02 0e 02 0e 9f 0e 99 0e 21 07 02 0e 32 0e 8a 0e 9f 0e 8a 0e 21 07 02 0e 1a 0e 7c 0e 48 0e 7c 0e 8a 07 02 0e 32 0e 70 07 02 0e 8a 0e 21 0e 7e 0e 8a 0e 21 0e 2c 0e 21 0e 70 0e 1a 0e 21 0e 8a 0e 32 0e 99 07 02 0e 32 0e 70 07 02 0e a2 0e 7c 0e 48 0e 9f 0e 7e 0e 99 0e 02 0e 99 0e 21 07 02 0e a2 0e 21 0e 48 0e 32 0e 99 07 02 0e 21 0e 91 0e 91 0e 21 07 02 0e 0a 0e 32 0e 48 0e 48 0e 9f 0e 51 07 02 0e 1a 0e 7c 0e 48 0e 7c 0e 8a 0e 21 07 02 0e 21 0e 9f 07 02 0e 23 0e 9f 0e 25 0e 32 0e 02 0e 99 07 02 0e 70 0e 9f 0e 48 0e 48 0e 02 07 02 0e 7e 0e 02 0e 8a 0e 32 0e 02 0e 99 0e 9f 0e 8a 07 33 07 02 0e 21 0e a6 0e 0a 0e 21 0e 7e 0e 99 0e 21 0e 9f 0e 8a 07 02 0e 91 0e 32 0e 70 0e 99 07 02 0e 7c 0e 0a 0e 0a 0e 02 0e 21 0e 0a 0e 02 0e 99 07 02 0e 0a 0e 9f 0e 7e 0e 32 0e 1a 0e 02 0e 99 0e 02 0e 99 07 02 0e 70 0e 7c 0e 70 07 02 0e 7e 0e 8a 0e 7c 0e 32 0e 1a 0e 21 0e 70 0e 99 07 2f 07 02 0e 91 0e 9f 0e 70 0e 99 07 02 0e 32 0e 70 07 02 0e 0a 0e 9f 0e 48 0e 7e 0e 02 07 02 0e 89 0e 9f 0e 32 07 02 0e 7c 0e 23 0e 23 0e 32 0e 0a 0e 32 0e 02 07 02 0e 1a 0e 21 0e 91 0e 21 0e 8a 0e 9f 0e 70 0e 99 07 02 0e 51 0e 7c 0e 48 0e 48 0e 32 0e 99 07 02 0e 02 0e 70 0e 32 0e 51 07 02 0e 32 0e 1a 07 02 0e 21 0e 91 0e 99 07 02 0e 48 0e 02 0e 09 0e 7c 0e 8a 0e 9f 0e 51 07 33 01 01 12 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 12 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 12 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 12 01 01 ff 82 82 12 00

This is what I meant by the Spot the difference between these two pictures game. Who wants to sit there and compare two 900-byte sequences byte-by-byte? (On top of that, the customer liaison forwarded the question to an internal peer-to-peer discussion group. Peer-to-peer discussion groups do not come with a service level agreement, and if you ask a question which makes heavy demands on the reader just to understand the question, don't be surprised if nobody answers.)

If you want an explanation of why two things are different, you need to point out where the difference is. This often happens with screen shots, where a customer includes two different screen shots and asks for an explanation of why they are different, without indicating where the difference is. Are they complaining that the font is different? That ClearType is enabled in one screen shot but not the other? The hard drive in the first screen shot has more free space than the one in the second screen shot?

Anyway, back to the sort keys. (As a refresher, you can read Michael Kaplan's explanation of sort keys.) The documentation for LCMap­String says

dwMapFlags [in]

Flags specifying the type of transformation to use during string mapping or the type of sort key to generate. For detailed definitions, see the dwMapFlags parameter of LCMapStringEx.

If you follow the instructions and consult the dwMapFlags parameter of the LCMapStringEx function, you find

LCMAP_SORTKEY Produce a normalized sort key. If the LCMAP_SORTKEY flag is not specified, the function performs string mapping. For details of sort key generation and string mapping, see the Remarks section.

If you follow the instructions and go to the Remarks section looking for LCMAP_SORTKEY, you find

The application can use LCMapString or LCMapStringEx to generate a sort key. To do this, the application specifies LCMAP_SORTKEY for the dwMapFlags parameter. For more information, see Handling Sorting in Your Applications.

(A link to Handling Sorting in Your Applications also exists on the LCMapString page, but let's say you didn't see that.)

If you follow the instructions and read the Handling Sorting in Your Applications page,

Use Sort Versioning

A sorting table has two numbers that identify its version: the defined version and the NLS version. Both numbers are DWORD values, composed of a major value and a minor value. ...

Note For a major version, one or more code points are changed so that the application must re-index all data for comparisons to be valid. For a minor version, nothing is moved but code points are added. For this type of version, the application only has to re-index strings with previously unsortable values.

And there you have your answer. Sort keys generated by different major versions are not compatible. I don't know what NLS version for English was included with Windows XP, though I did check that on Windows Vista SP2 it is major version 0x0405 (minor version 0) and on Windows 7 it is major version 0x601 (minor version 1). So whatever it was on Windows XP, it was presumably less than or equal to 0x0405, which means that it's definitely different from the major version on Windows 7.

And like the documentation says, when the major version changes, you need to regenerate your sort keys because sort keys from different major versions are not compatible.

Michael Kaplan discussed other issues arising from changing the major version as well as various changes to the sort keys over time.

If you find yourself using a program that relies on sort keys remaining stable even across major versions, you can try applying the EmulateSorting compatibility fix.

Comments (25)
  1. Adam Rosenfield says:

    I'm glad to see that LCMapStringEx did away with ANSI string compatibility and only provides a Unicode string version of that function.

  2. Joshua says:

    Maybe I'm missing the point here, but sort stability sounds like something that really should be dependable. See binary tree.

    [So all characters added to Unicode after 1.0 should sort as "unknown" to remain compatible with the Unicode 1.0 sort order? (And the Swedish Academy should be told to revert their 2006 change to Swedish sort rules?) -Raymond]
  3. ErikF says:

    This sounds like when I played Colossal Cave and had to carefully distinguish between the different types of rooms filled with twisty passages. As a brain game, it's kind of fun, but as a support call, it's not what I'm looking forward to! I don't mind too much trying to figure out what was meant in the original question, but hopefully the person asking the question is being billed per-hour on this as I try to puzzle out the differences.

  4. Bob says:

    The output looks like some of the testing the military does during MEPS processing.

  5. anon says:

    Did you really just post a thousand lines to point out the last 4?

  6. I think I just broke my mousewheel.

  7. Mason Wheeler says:

    Of course no one wants to manually go through large dumps of data to find small differences.  Luckily, these days we have computers, which are very good at automating comparisons.

    If I got something like that in a bug report, the first thing I'd do is pull up Beyond Compare and paste the two keys into it.  I just tried that, in fact.  It instantly found the difference between them, no drudgery required.

    [This wasn't a bug report. It was an email message. -Raymond]
  8. Mason Wheeler says:

    [This wasn't a bug report. It was an email message. -Raymond]

    Meh.  It was an email reporting [what the customer saw as] a bug.  And either way, it's not difficult to copy-and-paste large blocks of text into Beyond Compare. :P

    [It also assumes you have Beyond Compare on whatever device you happen to be using when you read that email. (And woe unto you if they post two screenshots.) The underlying principle remains: If you want people to help you, you need to help them. -Raymond]
  9. Henning Makholm says:

    @Bob: To me it looks like a relatively straightforward encoding of an UTS#10 sort key. (And UTS#10 explicitly warns that sort keys are not supposed to be stable across successive versions of the collation-element table (or, for that matter, across a change of locale (well, duh))).

  10. Mason Wheeler says:

    [It also assumes you have Beyond Compare on whatever device you happen to be using when you read that email. (And woe unto you if they post two screenshots.)]

    Yes, it does assume you have Beyond Compare available.  For me, at least, that's a safe assumption.  It's either available, or will be as soon as I get into the office if I happen to be reading it on my phone.  (And if so, the problem can generally wait until I get into the office anyway.)

    And if they posted two screenshots… I'd open them both in Beyond Compare and run an image comparison!  (It can do that too.)  It really is an incredible tool for comparing two mostly similar sets of data and finding the differences.

    [The underlying principle remains: If you want people to help you, you need to help them. -Raymond]

    I suppose.  But the point I'm trying to make is that "spot the difference between these two pictures" is a solved problem these days, and so someone complaining about how that makes it difficult to do their job tells me that they're not aware of that fact.

    [You can't expect somebody to spend more than ten seconds reading a message sent to a peer-to-peer list. If it takes them 15 seconds to copy the content and paste it into a comparison tool, you've lost. Pretend every message begins with the sentence, "Could you do me a favor?" -Raymond]
  11. Xkeeper says:

    I can't believe I read through all 300-odd pages of TONT. Good grief.

    Anyway, one way to get around it is to paste the two dumps into separate text files, then just fc /b d1.txt d2.txt and divide offsets by 3 (unless they were hard wrapped, in which case you can leave the /b of and use the line numbers to help deduce what lines are different).

    Still too much work for something like this, though.

  12. Jeffrey Bosboom says:

    Mason Wheeler: From now on, whenever Raymond gets one of these e-mails, he'll forward it to you.  Congratulations!

  13. Gabe says:

    Mason Wheeler: If you expect everybody to have Beyond Compare at their fingertips, why not expect that the person who submitted the request include output of Beyond Compare? That way only one person has to spend 15 seconds doing the comparison rather than all 500 people on the list, and the person who does the extra work is the one who cares most about the problem.

  14. John says:

    @Maurits:  Please don't ever do that again.

  15. Maurits says:

    The last four bytes are different (not counting the terminating 00.)

    0e

    48

    0e

    7c

    0e

    8a

    0e

    21

    0e

    51

    07

    02

    0e

    32

    0e

    7e

    0e

    91

    0e

    9f

    0e

    51

    07

    02

    0e

    1a

    0e

    7c

    0e

    48

    0e

    7c

    0e

    8a

    07

    02

    0e

    91

    0e

    32

    0e

    99

    07

    02

    0e

    02

    0e

    51

    0e

    21

    0e

    99

    07

    2f

    07

    02

    0e

    0a

    0e

    7c

    0e

    70

    0e

    91

    0e

    21

    0e

    0a

    0e

    99

    0e

    21

    0e

    99

    0e

    9f

    0e

    8a

    07

    02

    0e

    02

    0e

    1a

    0e

    32

    0e

    7e

    0e

    32

    0e

    91

    0e

    32

    0e

    0a

    0e

    32

    0e

    70

    0e

    25

    07

    02

    0e

    21

    0e

    48

    0e

    32

    0e

    99

    07

    2f

    07

    02

    0e

    91

    0e

    21

    0e

    1a

    07

    02

    0e

    1a

    0e

    7c

    07

    02

    0e

    21

    0e

    32

    0e

    9f

    0e

    91

    0e

    51

    0e

    7c

    0e

    1a

    07

    02

    0e

    99

    0e

    21

    0e

    51

    0e

    7e

    0e

    7c

    0e

    8a

    07

    02

    0e

    32

    0e

    70

    0e

    0a

    0e

    32

    0e

    1a

    0e

    32

    0e

    1a

    0e

    9f

    0e

    70

    0e

    99

    07

    02

    0e

    9f

    0e

    99

    07

    02

    0e

    48

    0e

    02

    0e

    09

    0e

    7c

    0e

    8a

    0e

    21

    07

    02

    0e

    21

    0e

    99

    07

    02

    0e

    1a

    0e

    7c

    0e

    48

    0e

    7c

    0e

    8a

    0e

    21

    07

    02

    0e

    51

    0e

    02

    0e

    25

    0e

    70

    0e

    02

    07

    02

    0e

    02

    0e

    48

    0e

    32

    0e

    89

    0e

    9f

    0e

    02

    07

    33

    07

    02

    08

    14

    07

    02

    0e

    9f

    0e

    99

    07

    02

    0e

    21

    0e

    70

    0e

    32

    0e

    51

    07

    02

    0e

    02

    0e

    1a

    07

    02

    0e

    51

    0e

    32

    0e

    70

    0e

    32

    0e

    51

    07

    02

    0e

    a2

    0e

    21

    0e

    70

    0e

    32

    0e

    02

    0e

    51

    07

    2f

    07

    02

    0e

    89

    0e

    9f

    0e

    32

    0e

    91

    07

    02

    0e

    70

    0e

    7c

    0e

    91

    0e

    99

    0e

    8a

    0e

    9f

    0e

    1a

    07

    02

    0e

    21

    0e

    a6

    0e

    21

    0e

    8a

    0e

    0a

    0e

    32

    0e

    99

    0e

    02

    0e

    99

    0e

    32

    0e

    7c

    0e

    70

    07

    02

    0e

    9f

    0e

    48

    0e

    48

    0e

    02

    0e

    51

    0e

    0a

    0e

    7c

    07

    02

    0e

    48

    0e

    02

    0e

    09

    0e

    7c

    0e

    8a

    0e

    32

    0e

    91

    07

    02

    0e

    70

    0e

    32

    0e

    91

    0e

    32

    07

    02

    0e

    9f

    0e

    99

    07

    02

    0e

    02

    0e

    48

    0e

    32

    0e

    89

    0e

    9f

    0e

    32

    0e

    7e

    07

    02

    0e

    21

    0e

    a6

    07

    02

    0e

    21

    0e

    02

    07

    02

    0e

    0a

    0e

    7c

    0e

    51

    0e

    51

    0e

    7c

    0e

    1a

    0e

    7c

    07

    02

    0e

    0a

    0e

    7c

    0e

    70

    0e

    91

    0e

    21

    0e

    89

    0e

    9f

    0e

    02

    0e

    99

    07

    33

    07

    02

    0e

    1a

    0e

    9f

    0e

    32

    0e

    91

    07

    02

    0e

    02

    0e

    9f

    0e

    99

    0e

    21

    07

    02

    0e

    32

    0e

    8a

    0e

    9f

    0e

    8a

    0e

    21

    07

    02

    0e

    1a

    0e

    7c

    0e

    48

    0e

    7c

    0e

    8a

    07

    02

    0e

    32

    0e

    70

    07

    02

    0e

    8a

    0e

    21

    0e

    7e

    0e

    8a

    0e

    21

    0e

    2c

    0e

    21

    0e

    70

    0e

    1a

    0e

    21

    0e

    8a

    0e

    32

    0e

    99

    07

    02

    0e

    32

    0e

    70

    07

    02

    0e

    a2

    0e

    7c

    0e

    48

    0e

    9f

    0e

    7e

    0e

    99

    0e

    02

    0e

    99

    0e

    21

    07

    02

    0e

    a2

    0e

    21

    0e

    48

    0e

    32

    0e

    99

    07

    02

    0e

    21

    0e

    91

    0e

    91

    0e

    21

    07

    02

    0e

    0a

    0e

    32

    0e

    48

    0e

    48

    0e

    9f

    0e

    51

    07

    02

    0e

    1a

    0e

    7c

    0e

    48

    0e

    7c

    0e

    8a

    0e

    21

    07

    02

    0e

    21

    0e

    9f

    07

    02

    0e

    23

    0e

    9f

    0e

    25

    0e

    32

    0e

    02

    0e

    99

    07

    02

    0e

    70

    0e

    9f

    0e

    48

    0e

    48

    0e

    02

    07

    02

    0e

    7e

    0e

    02

    0e

    8a

    0e

    32

    0e

    02

    0e

    99

    0e

    9f

    0e

    8a

    07

    33

    07

    02

    0e

    21

    0e

    a6

    0e

    0a

    0e

    21

    0e

    7e

    0e

    99

    0e

    21

    0e

    9f

    0e

    8a

    07

    02

    0e

    91

    0e

    32

    0e

    70

    0e

    99

    07

    02

    0e

    7c

    0e

    0a

    0e

    0a

    0e

    02

    0e

    21

    0e

    0a

    0e

    02

    0e

    99

    07

    02

    0e

    0a

    0e

    9f

    0e

    7e

    0e

    32

    0e

    1a

    0e

    02

    0e

    99

    0e

    02

    0e

    99

    07

    02

    0e

    70

    0e

    7c

    0e

    70

    07

    02

    0e

    7e

    0e

    8a

    0e

    7c

    0e

    32

    0e

    1a

    0e

    21

    0e

    70

    0e

    99

    07

    2f

    07

    02

    0e

    91

    0e

    9f

    0e

    70

    0e

    99

    07

    02

    0e

    32

    0e

    70

    07

    02

    0e

    0a

    0e

    9f

    0e

    48

    0e

    7e

    0e

    02

    07

    02

    0e

    89

    0e

    9f

    0e

    32

    07

    02

    0e

    7c

    0e

    23

    0e

    23

    0e

    32

    0e

    0a

    0e

    32

    0e

    02

    07

    02

    0e

    1a

    0e

    21

    0e

    91

    0e

    21

    0e

    8a

    0e

    9f

    0e

    70

    0e

    99

    07

    02

    0e

    51

    0e

    7c

    0e

    48

    0e

    48

    0e

    32

    0e

    99

    07

    02

    0e

    02

    0e

    70

    0e

    32

    0e

    51

    07

    02

    0e

    32

    0e

    1a

    07

    02

    0e

    21

    0e

    91

    0e

    99

    07

    02

    0e

    48

    0e

    02

    0e

    09

    0e

    7c

    0e

    8a

    0e

    9f

    0e

    51

    07

    33

    01

    01

    12

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    12

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    12

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    02

    12

    01

    01

    *** 81 ff

    *** fb 82

    *** 06 82

    *** 82 12

    00

  16. Cheong says:

    After reading the first few paragraphs, I was kind of expecting the user attach "images" of resulting sort keys.

    Then again if I receive 2 such image, if I found the font and spacing of characters in it were the same, I'll just open up whatever image conversion program, set pic1 and pic2 to different layer, then decrease opacity of layer 2.

    That's how I cheat at "Spot the difference between these two pictures game". :P

  17. Maurits, I just thought you did it a second time.

    I was getting ready to check if you did this kind of thing on a regular basis on MSDN blogs. Your first post completely exhausted my mouse! Thank the Lord it wasn't necessary to go around again…

  18. Maurits says:

    Oops.  Raymond, can you delete my earlier comment or edit it down to this?

    The last four bytes are different (not counting the terminating 00.)

    0e

    48

    01

    01

    *** 81 ff

    *** fb 82

    *** 06 82

    *** 82 12

    00

  19. Worf says:

    I had no problems going around Maurits' thousand line comment – I opted for a mouse with an inertial wheel, so a quick flick causes it disengage the friction and it freewheels.

    Though, I suspect someone probably opened this up on their mobile device and wore down their fingertips scrolling – the inertia scrolling there doesn't last as long.

  20. chentiangemalc says:

    Even using tools like BeyondCompare or WinDiff regularly I'd still prefer person told us what they're looking for. In peer support groups people who make no effort in the question are lucky to get answers. Some people get so lazy asking questions. Recently someone emailed me "How do I deploy custom libraries in Win7" I replied TechNet site has what you need. Then get back details of some crazy attempt in response and asked why it wasn't working. Google search TechNet page one of first results, email exact article link with step by step instructions. That still didn't work out. Copied exact paragraph emailed back. Hey it works!

  21. mdw says:

    I found the difference between those two blocks very easily, and without using dedicated tools.  The first thing I did was open a new view of the article in another browser window.  Then I picked up the new window and overlaid it carefully on the old one.  My window manager is configured in an eye-candy mode where windows being moved or resized appear translucent.  Result: when I've lined up the two windows properly, the parts that are the same in both come through clearly, and the parts that are different are fuzzy and indistinct.  The human visual system is surprisingly good at picking out the fuzzy parts.

    I mostly use this trick for comparing SSH host key fingerprints and similar, since it's both faster and very much more reliable than trying to compare them by hand.

  22. Simon says:

    @mdw – that's still a lot of unnecessary work, when the person reporting the problem could have just said "the last 4 bytes are different". Raymond's point stands – if you're asking someone to help you, it helps if you try to minimize what you're asking for.

  23. Lars says:

    Is it just me, or is the "peer discussion group" thing a red herring? Raymond wrote "On top of that, <b>the customer liaison</b> forwarded the question to an internal peer-to-peer discussion group."

    How is that the customer's fault?

    [Then the customer liaison should have cleaned it up. I tend to treat the customer+liaison as a single entity, since the customer/liaison distinction is abstracted away. -Raymond]
  24. Joshua says:

    I've been back and forth on analyzing this quite a few times. The only solution is for the API to accept the major version as an argument.

    [That would presumably mean having a copy of collation tables for every version of Unicode that ever existed, just in case somebody asks for it. -Raymond]
  25. There are quite a few cases where the sort key stability is desired. One is when it's used to index a database. Although one could rebuild the index if the DBMS is upgraded.

    Another case is when you need a stable search key in NTFS. You would not want your directory B-tree to become inconsistent with the filenames.

Comments are closed.