You can ask the compiler to answer your calling convention questions


If you want to figure out some quirks of a calling convention, you can always ask the compiler to do it for you, on the not unreasonable assumption that the compiler understands calling conventions.

"When a __stdcall function returns a large structure by value, there is a hidden first parameter that specifies the address the return value should be stored. But if the function is a C++ instance method, then there is also a hidden this parameter. Which goes first, the return value parameter or the this pointer?"

This is another case of You don't need to ask me a question the compiler can answer more accurately.

struct LargeStructure
{
 char x[256];
};

class Something
{
public:
 LargeStructure __stdcall TestMe();
};

void foo(Something *something)
{
 LargeStructure x = something->TestMe();
}

You could compile this into a program and then look in the debugger, or just ask the compiler to generate an assembly listing. I prefer the assembly listing, since it saves a few steps, and the compiler provides helpful symbolic names.

  00015 mov     eax, DWORD PTR _something$[ebp]

; LargeStructure x = something->TestMe();

  00018 lea     ecx, DWORD PTR _x$[ebp]
  0001e push    ecx
  0001f push    eax
  00020 call    ?TestMe@Something@@
                QAG?AULargeStructure@@XZ
                ; Something::TestMe

We see that the last thing pushed onto the stack (and therefore the top parameter on the stack at the point of the call) is the something parameter, which is the this for the function.

Conclusion: The this pointer goes ahead of the output structure pointer.

Comments (22)
  1. Joshua says:

    Now the correct question is what does a COM library do as an internal call can ignore calling conventions. Same procedure to answer.

  2. JM says:

    I love it when people proactively answer their own questions like this even when it's not the whole story — and then fixate on the answer. When you get around to convincing them that what they're observing is not actually guaranteed, but an artifact of how the system happens to work right now, and if they want solid guarantees, please read this document over here, they have the rock-solid defense that documents are just words someone babbled one time, and they've got the code to prove how it works on their system. Theirs is the Living God!

    Now, for a calling convention, this works. But approaches like this need the standard disclaimer of "please make sure you understand in what contexts the answer you have just obtained is valid". Unfortunately, most people are far too overjoyed at having an answer to consider the burdensome task of considering whether it's the right one.

  3. John Doe says:

    That's all very nice when you already have the compiler sitting there somewhere in the PATH eagerly waiting to be run.

    Up until Visual Studio 2005, there was no express edition. You could dig a cl.exe from specific SDK versions. Now, you need the express edition for the compiler, as the newer SDKs are no longer packed with it. What a terrific turn of events. Although, I agree with the latter: Visual Studio is subject to software (read, security) updates, the SDK isn't.

    In short, if you really want to, you can get the compiler for free. But I think it's too much overhead to find out what a calling convention is supposed to do, notwithstanding having to code all possible, interesting or corner-case scenarios.

    "What are you talking about?", you say. Well, more often than not, someone is interested in calling conventions not by mere curiosity or for optimization purposes, but to make a foreign interface support library or define foreign structures and functions (foreign as in not managed, or C-native, or machine raw bit twiddling).

  4. Joshua says:

    [Up until Visual Studio 2005, there was no express edition. You could dig a cl.exe from specific SDK versions. Now, you need the express edition for the compiler, as the newer SDKs are no longer packed with it. What a terrific turn of events. Although, I agree with the latter: Visual Studio is subject to software (read, security) updates, the SDK isn't.]

    And to make matters worse, while an SDK for an arbitrary version of Windows can still be had, old express editions can't be. Way to go and make targeting XP forever much more attractive than it should be.

  5. Jasper says:

    Well, we've only tested one variant of the rules. When I would ask a question like this, I would expect all the ifs, ands, and buts to come as part of the reply. Why should I have to reverse engineer the rules that the compiler already has encoded in it?

    [Usually, this question is asked in the context of "I'm trying to call function X but I can't get the calling convention right. How am I supposed to pass the Y parameter?" In other words, they're not asking for the general rules; they just want to the rules for one particular instance. Answer: Ask the compiler to do X, and see how it passes the Y parameter. -Raymond]
  6. Rushyo says:

    @Jasper It's not Raymond's job to provide on-demand training (at least, I'm pretty certain it's not). If people want to know how it behaves, they can poke it or look up the documentation like everyone else. If something is wrong, then perhaps that's something for Raymond. Otherwise, it's probably not something for Raymond.

    I do find it peculiar that people think support contracts mean that you can just prod knowledge workers for their knowledge whenever they like and that if there's a job in the knowledge worker's domain that's confusing them, that it's the knowledge worker's job to do their job for them. It's like going up to a chef in a restaurant and demanding they tell you how to make all the dishes and the chemical content of all the ingredients, down to the molecular level. Whilst that might be useful to you, all the chef cares about is that you enjoy the food. If you don't enjoy the food then he might want to address it. It does not make him a specialised version of Wikipedia.

  7. alegr1 says:

    This is expected, if you remember that the calling conventions are supposed to allow pseudo-member function call from C (for COM compatibility). For this reason, 'this' pointer takes place of the first C-style call argument, while the return structure pointer goes even before that.

  8. 12BitSlab says:

    Way back in the olden days, I used to write quite a bit of S/360 Assembler.  One of the great things about OS/360 and its follow-ons was that the calling conventions were very consistant and very well documented.

    This is not a slam on the calling conventions for Windows.  Windows and OS/360 are very different animals serving very different types of developers.

    Consistancy was and still is a great thing.

    BR R14            (obscure refernece to saying "goodbye" AND calling conventions)

  9. Henning Makholm says:

    @algr1: Aren't you arguing that the calling convention Raymond shows here is UNEXPECTED? As shown by the assembler, the invisible return struct pointer goes BETWEEN the "this" pointer and the first explicit argument. Remember that arguments are pushed on the stack in reverse of the source order, such that their addresses on the stack will be in the source order.

    A more fundamental problem with your analysis is that COM doesn't care about calling conventions for returning large structures at all, since COM member functions always return HRESULT anyway (or in certain special cases other small types).

  10. John Doe says:

    @Rushyo, what if the documentation is not enough or is incomplete? What if you really need to know all about calling conventions, because, say, you're developing a compiler (no matter the language)? Even by your reasoning, this might be something for Raymond.

    Even by Raymond's reasoning, according to the linked post blogs.msdn.com/…/10190102.aspx , this is something the compiler can answer you in concrete examples, but it might take you more time to find out all about calling conventions this way than to get a detailed answer or request proper documentation.

    BTW, how do you know what is and is not for Raymond? blogs.msdn.com/…/10392028.aspx . Anyway, it's not like the answer would (need to) be "In the beginning, there was a big bang, then … calling conventions … meaning of life …" In the end, I'm sure people become much happier about their spent money on support and/or maintenance from a company that answers even these kind of questions.

  11. David Walker says:

    Yes, the answer you get will be correct (for today).  

    How does this square with Microsoft being forced to spend a lot of money on compatibility hacks to accomodate software vendors who "figured out how things work in practice" instead of finding the appropriate contractual behavior?  And the clear understanding there is that "figuring out how things work in practice" is inferior to figuring out how things are supposed to work.  And if the documentation is hard to find, it's not the programmers' fault that they went with what they could figure it out.

    I think my point is that it's hard for a programmer to know when he SHOULD look at "how things happen to work right now with my specific machine architecture and memory size and version of Windows" and when he should try really hard to find the correct contractual behavior.

    [Suppose you write a program in C++. Now you write another program in assembly language that passes parameters the same way the C++ program does. Those two programs are indistinguishable. Therefore, any dependency you're making in your assembly language is the same dependency you're making in C++. Yet nobody stops and says, "Oh, no, what if Microsoft changes the way the Y parameter is passed to function X? Will that break my C++ program?" You're confusing runtime behavior with compile-time behavior. Compile-time behavior can never change because it's compiled. -Raymond]
  12. Myria says:

    This is one reason why I have IDA Pro at work – to disassemble our own stuff and see how the compiler did things.

  13. Given that the parameters are passed on a/the stack, isn't it more accurate to say that "this" goes (push) behind/after, but COMES (pop) before ?

    Just asking.  :)

  14. Cesar says:

    @Raymond: What @Henning Makholm is asking for is not an exact struct. He is writing a code generator, so he cannot know in advance the exact struct, and must code for all possible alternatives.

  15. Henning Makholm says:

    @John Doe: I think we must distinguish between at least kinds of information sources here. In order of decreasing utility: (1) An authoritative spec that Microsoft (or whichever vendor we're talking about) is formally committed to implement; (2) direct observation of what the thing does in practice; (3) ad-hoc answers to concrete questions from people like Raymond.

    I take your point to be that having (1) is superior to either of the two others. And I agree fully with that, having recently had the fun experience of extending an in-house compiler to generate code for the Windows Runtime API. Here there's essentially no answers of class (1) for the abstraction layers that lie between __stdcall and the fully projected "here's what to write" .NET or C++/CX syntax one finds in MSDN. And yes, that is infuriating.[*]

    However, GIVEN a situation where (1) is not available, then Raymond's point here appears to be that (2) is superior to (3). That is also true, assuming that we're speaking about questions where (2) CAN logically provide complete answer, modulo some common sense. The concrete question in the posting is of that kind — "is is this way or that way?". It's very hard to imagine that the answer would be "it depends" — because what would it possibly depend on? Whether the size of the struct is a prime multiple of its alignment? Or whether there are more than 5 parameters EXCEPT the two invisible ones whose order we're asking about?

    Sure, it would be even nicer if there were a complete self-contained description of the entire calling convention somewhere — but then we're talking (1) versus (2), not (2) versus (3). There doesn't seem to be any advantage in waiting for someone at Microsoft to ask the compiler (because assuming that (1) simply doesn't exist even internally, that's what they'd do) over doing it oneself.

    [*] It would be nice to have a neat conspiracy theory about how Microsoft is deliberately keeping this information secret as part of some convoluted plot to enslave the world, but it doesn't hold water because everything else indicates that they WANT people to create software using this technology and the absence of specs is simply due to resource constraints rather than actively malicious intent. Probably a lot of details were worked out by a bunch of guys around a whiteboard somewhere in Redmond and then they went each to their desks and implemented their end of the decisions, and nobody cared to write a self-contained description of the whole thing because there's a ship date looming and most of the sheeple app programmers just want source examples they can cut-and-paste from without understanding it anyway, and who other than us are ever going to care whether a class object for a runtime class without parameterless public constructors is required to implement IActivationFactory nevertheless?

    [If you want to remove the variability over things like whether the size of the struct is a prime number (in practice: it can vary depending on whether the size of the struct is 8 or less), just run your test with the exact struct you want the answer for. The answer you get from the compiler is necessarily correct because that's how it would've happened if you had simply written a C program. -Raymond]
  16. alegr1 says:

    @myria:

    …or just use the compiler option to generate the .COD file.

  17. Ooh says:

    @Henning Makholm: You know the Visual C++ Team Blog, don't you? They had/are running a series on C++/CX development and compiler transformations (all other parts are linked from the first post):

    C++/CX Part 0 of [n]: An Introduction (blogs.msdn.com/…/cxxcxpart00anintroduction.aspx)

    Other stuff (even in MSDN!) is linked from these. Should get you pretty far, and all the missing pieces should be pretty much like COM.

  18. laonianren says:

    The documentation of calling conventions is very poor, but it tends to be a non-issue because public APIs avoid the corner cases.

    Though the reality of what the compiler does with different size structs is quite bizarre:

    blog.aaronballman.com/…/describing-the-msvc-abi-for-structure-return-types

    It would be interesting to know if this is consistent across compiler versions.

  19. David Walker says:

    Thanks, Raymond.  That explanation helps.

    More text goes here.

  20. Henning Makholm says:

    @Cesar: Actually what I thought I was doing here was to DEFEND Raymond's point, but I must have done it badly, since it didn't register with either him or you. The original question was: GIVEN that there's a hidden parameter that points to memory that the returned struct is written to, will what parameter come before or after the this parameter? I maintain that there's no reason to expect that the parameters will sometimes come in one order and sometimes in another. Therefore one experiment is enough to settle the question asked. Sometimes one or the other of the pointers won't BE in the parameter list, but that is a different question.

  21. Mike Dimmick says:

    @Ooh: There has been a disturbing trend over the last probably five or six years for things that *should* be documented in Henning's Type (1) to only be documented in an MSDN blog, i.e. type (3). The contracts *must* be defined in proper official documentation: we can't be expected to trawl the thousands of MSDN and Technet blogs to find an answer, and then try to discern whether this is a description of how it works now, or an actual contract of how it will continue to work in the future.

  22. 640k says:

    It's also required to consider all the breaking change documents for all APIs between product A and product B to know if they are compatible.

    Promises has been broken in the past.

Comments are closed.