If you try to declare a variadic function with an incompatible calling convention, the compiler secretly converts it to cdecl


Consider the following function on an x86 system:

void __stdcall something(char *, ...);

The function declares itself as __stdcall, which is a callee-clean convention. But a variadic function cannot be callee-clean since the callee does not know how many parameters were passed, so it doesn't know how many it should clean.

The Microsoft Visual Studio C/C++ compiler resolves this conflict by silently converting the calling convention to __cdecl, which is the only supported variadic calling convention for functions that do not take a hidden this parameter.

Why does this conversion take place silently rather than generating a warning or error?

My guess is that it's to make the compiler options /Gr (set default calling convention to __fastcall) and /Gz (set default calling convention to __stdcall) less annoying.

Automatic conversion of variadic functions to __cdecl means that you can just add the /Gr or /Gz command line switch to your compiler options, and everything will still compile and run (just with the new calling convention).

Another way of looking at this is not by thinking of the compiler as converting variadic __stdcall to __cdecl but rather by simply saying "for variadic functions, __stdcall is caller-clean."

Exercise: How can you determine which interpretation is what the compiler actually does? In other words, is it the case that the compiler converts __stdcall to __cdecl for variadic functions, or is it the case that the calling convention for variadic __stdcall functions is caller-clean?

Comments (16)
  1. alegr1 says:

    >How can you determine which interpretation is what the compiler actually does?

    Decode the decorated name.

  2. Adam says:

    The linker will decode the name for you if there's an error:

    error LNK2019: unresolved external symbol "void __cdecl something(char *,…)" (?something@@YAXPADZZ) referenced in function _main

  3. John Doe says:

    I'd prefer that a variadic stdcall function was "caller cleans required arguments, callee cleans the rest".

    I mean, the ellipsis could just rightly mean "caller cleans whatever was in …" in every calling convention, and leave the rest alone.

    [Why create extra work for no reason? The caller has to clean the variadic parameters, so it may as well clean up the fixed parameters at no extra cost. It's not like adding 12 takes longer than adding 8. -Raymond]
  4. VinDuv says:

    After some testing, it appears that for variadic functions, MinGW gcc does not put the @x suffix (so the decoration is as if __cdecl was used). I guess it is the same thing with other compilers.

    This "@<number of bytes of parameters>" suffix used by __stdcall offers some protection on calling a callee-clean function with the wrong number of parameters, which would result in stack corruption, so it makes sense for me to not use this decoration for functions which are not callee-clean (it would be messy if the callee-clean "__stdcall void f(int x)" and the caller-clean "__stdcall void f(int x, …)" had the same decorated name…)

  5. igorsk says:

    Answer: compile that snippet, dump the object file and demangle (undecorate) the symbol name.

    Undecoration of :- "?something@@YAXPADZZ"

    is :- "void __cdecl something(char *,…)"

  6. Eric TF Bat says:

    > Wow that's a really bad explanation. Kind of the lazy developer explanation.

    You speak of laziness as if it were a bad thing.

  7. Joshua says:

    [Why create extra work for no reason? The caller has to clean the variadic parameters, so it may as well clean up the fixed parameters at no extra cost. It's not like adding 12 takes longer than adding 8. -Raymond]

    I can imagine a calle-cleaned variadic. One error in va_args and you unbalance the stack though.

    [Not sure how that's possible, given that int f(int i, ...) { return i; } is a legal function. -Raymond]
  8. bystander says:

    > My guess is that it's to make the compiler options /Gr (set default calling convention to __fastcall) and /Gz (set default calling convention to __stdcall) less annoying.

    > Automatic conversion of variadic functions to __cdecl means that you can just add the /Gr or /Gz command line switch to your compiler options, and everything will still compile and run (just with the new calling convention).

    Wow that's a really bad explanation. Kind of the lazy developer explanation.

    Here's a spec for the warning: "If I explicitely (i.e. in source code) ask for a __stdcall convention, raise a warning if it can't be satisfied and is automatically changed to another convention."

    Note that this warning would not be raised for default conventions such as /Gz. You could say Gz means set the default convention to __stdcall *where applicable*.

  9. Cesar says:

    I can imagine a calling convention with callee-cleaned variadics: just pass a hidden argument with the number of bytes to pop from the stack.

    I wonder how callee-cleaned variadics would work on x86, though. From a quick search, it appears that the ret instruction only takes an immediate, so there is no easy way to pop a variable number of parameters and return, since the return address has to be on the stack. This is unlike for instance ARM, where the return address can be on a register, so you can manipulate the stack freely before returning. That is probably the real reason why the Win32 calling conventions do not have callee cleaned variables: too complex to implement, and would only lose performance.

  10. John Doe says:

    [Why create extra work for no reason? … -Raymond]

    I see it the other way around: less work. A fastcall variadic function could put the required arguments in registers and push variadic arguments on the stack. So, in the case where the variadic arguments are not provided and not used, it's a plain fastcall (I know in x64 everything is cdecl).

  11. John Doe says:

    @Cesar, I've seen implementations of Lisp compile functions that pass the number of arguments in ECX/RCX, mainly for assertion. In theory, they could clean up the stack, and I think at least one actually does.

  12. Joshua says:

    [Not sure how that's possible, given that int f(int i, …) { return i; } is a legal function. -Raymond]

    It's not a legal stdcall function, and neither would it be a legal callee-cleaned variardic function.

    [And how would a function tell the compiler how many bytes to remove upon return? And how would the compiler generate code that did that which didn't mess up the return address predictor? -Raymond]
  13. Joshua says:

    [And how would a function tell the compiler how many bytes to remove upon return?]

    By using the address where the va_list variable ends up of course.

    [And how would the compiler generate code that did that which didn't mess up the return address predictor? -Raymond]

    There's no good reason to use this on a modern processor.

    [And what if there is more than one va_list variable? And if there is no good reason to use this on a modern processor, why are you suggesting it? -Raymond]
  14. John Doe says:

    Looking back to what I stupidly wrote, let me correct; where you read:

    "caller cleans required arguments, callee cleans the rest"

    I really ment:

    "callee cleans required arguments, caller cleans the rest".

  15. Harald van Dijk says:

    > "It's not a legal stdcall function, and neither would it be a legal callee-cleaned variardic function."

    So if the function is declared separately, that declaration requires __cdecl, and perfectly valid standard C or C++ code that compiles on every other system needs Microsoft-specific additions to make it work there too?

    > "By using the address where the va_list variable ends up of course."

    Even ignoring the functions that don't need the variadic parameter values at all, what about those that use va_start and va_end, but nothing else, because they pass the va_list to a function such as vprintf?

  16. Joshua says:

    [And if there is no good reason to use this on a modern processor, why are you suggesting it? -Raymond]

    I can imagine != I think it's a good idea.

    [Oh. This Web site is about practical programming. There's no point discussion a design that is impractical. -Raymond]

Comments are closed.

Skip to main content