Nasty gotcha: VarCmp vs VariantCompare


There are two functions for comparing VARIANTs. One comes from the COM team, called VarCmp. Assuming a successful comparison, it returns

Comparison Return value
Less than 0 = VARCMP_LT
Equal to 1 = VARCMP_EQ
Greater than 2 = VARCMP_GT

This is consistent with the other Var­Xxx functions.

On the other hand, the Variant­Compare function comes from the shell, and it returns

Comparison Return value
Less than −1
Equal to 0
Greater than +1

Yuck.

So be careful which one you're calling and how you're testing the result.

Comments (11)
  1. Nah, it's easy. See, you just remember that VarCmp behaves like CompareString, and VariantCompare behaves like strcmp.

  2. Karellen says:

    I was about to disagree with your conclusion about which one was "yuck", with reference to the decades of precedence set by memcmp(), str*cmp(), strcoll() and wcs*cmp(), when I decided to have a look at the docs for each function (yeah, I know!).

    I think you miss off two important details about VarCmp()'s return values. The first is that they're not just an enum, they're also a valid HRESULT, which is useful in many ways but happens to rule out the possibility of using -1 as a "success" value. The second detail is the 4th possible return value which you elided: if either expression is NULL, the result is 3 = VARCMP_NULL, for which VariantCompare() does not have a documented behaviour.

    (That said, I would have made VARCMP_EQ = 0, with _LT and _GT both positive, to maintain the "0 == match" precedent. Although, that might have been more confusing for devs who would then expect the -1/+1 values by extension.)

  3. FS says:

    I disagree with the "Yuck". IMO: It makes more sense to explain equality with zero because it allows you to interpret any negative value (smaller than zero) as "less than", and any positive value (greater than zero) as "greater than". You could even go crazy and return the difference of the comparison (for example, 5 > 3 returns +2, and 4 < 10 returns -6 -- not sure why you would want to do it; it won't give you the best performance and it is problematic because of a high chance to produce overflow, but still it gives you the idea of thinking of comparion as a vector on a number scale). Maybe I've written too much .NET code... it makes me put the "Yuck" on the COM solution.

    1. The Yuck wasn't saying that −1/+1 is yucky. The Yuck was saying that having two different ways of expressing the result is yucky.

      1. Karellen says:

        Ah, I misread that too.

        The line directly under the VarCmp() result table is a comment on that very table, judging it, and approving of it due to its consistency with other functions. The line directly under the VariantCompare() result table is also a judgement, although negative, but due to example set with the VarCmp() comment appears to be similarly related directly to the contents of the VariantCompare() table, and not to the existence of two different result tables.

    2. dave says:

      and thus -2147483648 > 1 (in a 32-bit signed-arithmetic system)

  4. Joshua says:

    I wonder what these do if the variants are of utterly different types, say a word document and a date.

    1. VariantCompare explicitly states that in such a case the result is undefined and should be ignored. VarCmp doesn't seem to say anything about it. A possibility (implemented e.g. in Python 2) can be to define some strict ordering between the different types, and use it in comparisons between values of different types. This allows sorting and sorted data structures (e.g. RB trees) to work correctly even with non-homogeneous data.

    2. mc says:

      @Joshua The Word document is certainly going to be bigger!

  5. nikos says:

    thanks for this tip, I was using a PROPVARIANT class from the MSDN SHELL SAMPLES, and it looks like it has the bug! (PropVariant.h, class CComPropVariant). It looks somebody from MS copy/pasted code from CComVariant so propvariant comparison code is wrong!

    return PropVariantCompare(*this, varSrc) == static_cast(VARCMP_EQ);

    according to the docs PropVariantCompare behaves like strcmp so checking it agains VARCMP_EQ is wrong

    1. __klg says:

      Yeah, all the operators accepting PROPVARIANTs look broken.

Comments are closed.

Skip to main content