A static_cast is not always just a pointer adjustment

Even without considering virtual base classes, a static_cast to move between a base class and a derived class can be more than just a pointer adjustment.

Consider the following classes and functions.

class A
 int a;
 void DoSomethingA();

class B
 int b;
 void DoSomethingB();

class C : public A, public B
 int c;
 void DoSomethingC();

B* GetB(C* c)
 return static_cast<B*>(c);

void AcceptB(B* b);

void AcceptC(C* c)

Suppose the compiler decided to lay out the memory for C like this:

int a; } A } C
int b; } B
int c;

Now, you would think that converting a pointer to a C into a pointer to a B would be a simple matter of adding sizeof(int), since that's what you need to do to get from the a to the b.

Unless you happen to have started with a null pointer.

The rule for null pointers is that casting a null pointer to anything results in another null pointer.

This means that if the parameter to GetB is a null pointer, the function cannot return nullptr + sizeof(int); it has to return nullptr.

    xor rax, rax
    test rcx, rcx
    jz @F
    lea rax, [rcx+sizeof(int)]
@@: ret

Similarly, if the parameter to AcceptC is nullptr, then it must call AcceptB with nullptr.

    test rcx, rcx
    jz   @F
    add  rcx, sizeof(int)
@@: jmp  AcceptB

A naïve compiler would insert all these conditional jumps every time you cast between a base class and a derived class that involves an adjustment. But this is also a case where a compiler that takes advantage of undefined behavior can optimize the test away: If it sees that every code path through the static_cast dereferences either the upcast or downcast pointer, then that means that if the pointer being converted were nullptr, it would result in undefined behavior. Therefore, the compiler can assume that the pointer is never nullptr and remove the test.

void AcceptC2(C* c)

Here, the test can be elided because the result of the conversion is immediate dereferenced in order to call the B::Do­SomethingB method. The C++ language says that if you try to call an instance method on a null pointer, the behavior is undefined. Doesn't matter whether the method actually accesses any member variables; just the fact that you invoked an instance method is enough to guarantee that the pointer is not null. Therefore, the AcceptC2 function compiles to

    add rcx, sizeof(int)
    jmp B::DoSomethingB

The same logic applies on the receiving end of the method call: A method call can assume that this is never null.

void C::DoSomethingC()

    add rcx, sizeof(int)
    jmp AcceptB

Since this is never null, the conversion from C* to B* can elide the test and perform the adjustment unconditionally.

This means that you could add a dummy method to ever class:

class C : public A, public B
 void IsNotNull() { }
 int c;
 void DoSomethingC();

and call c->IsNotNull() to tell the compiler, "I guarantee on penalty of undefined behavior that c is not null."

void AcceptC3(C* c)

    add rcx, sizeof(int)
    jmp AcceptB

I don't know whether any compilers actually take advantage of this hint, but at least this is a way of providing it in a standard-conforming way.

Now, it looks like the purpose of this article is to delve into optimization tweaking in order to remove unwanted tests, but that wasn't actually the point. The point of the article was to explain what these tests are for. You'll be stepping through some code, and you'll see these strange tests against zero, so here's an explanation of why those tests are there.

Comments (15)
  1. Medinoc says:

    I doubt a compiler like Microsoft Visual C++ could take advantage of this, because some MFC stuff relies on this not being UB, at least on inline methods (such as CWnd::GetSafeHwnd()) that outright include a test on whether ‘this’ is null.

    1. Martin Bonner says:

      @Medinoc: They could:
      a) Add an option to allow the optimization.
      b) Say that GetSafeHwnd only works if CWnd is the *first* base class. (In which case the addition is not required).
      c) Add a magic __attribute__ which says “keep the checks in for adjust a pointer to this class”.

      I bet GCC uses an optimization like this.

      1. Joshua says:

        In this case, it’s an inline function, so the optimizer can see the if (this == NULL) check.

    2. Cesar says:

      The tests on whether ‘this’ is null might be a leftover from pre-standard (before C++98) versions of C++.

      1. Darran Rowe says:

        I’m not sure, it may be also a case of knowing the compiler too well at that point.
        I know for a fact that I have done some silly checks based on knowing the compiler too well.

  2. Joshua says:

    For the guy who wrote if (this == NULL) { /* logic here */ }, his code is starting to look really long in the tooth.

    1. Chris says:

      Bjarne Stroustrup himself did this in CFront (source: http://www.i-programmer.info/programming/cc/9212-finding-bugs-in-the-first-c-compiler-what-does-bjarne-think.html). Granted, that was a very different time.

      1. Joshua says:

        That’s if (this == NULL) { /* error trap here */ } which I still think is reasonable.

        1. Tom says:

          The problem is that the compiler is free to remove that entire conditional block because it only executes when something “impossible” happens and is thus undefined behavior.

        2. voo says:

          The problem is that as soon as you have undefined behavior, it also infects everything else in the program. Meaning the compiler may not just optimize that one check away, it could do anything with the function (realistic possibility I’d say) or the whole program (less likely to happen but theoretically possible).

  3. Krishty says:

    Just use a kind of assert() macro that evaluates to __assume() in release builds.

    assert(c != NULL);

    is easier to understand, does not require changes to the class, and even old Microsoft compilers (early 2000 era) understand __assume() involving null pointers properly.

  4. Kurt says:

    I would guess it’s as likely that the compiler would see IsNotNull is an empty inline and optimise it away first. Maybe you can hint this to compilers by passing around a reference instead of a pointer? Does &static_cast<B&>(*c) remove the nullptr check?

  5. Adam says:

    Clang can complain if you compare this with null, with a message something like:

    error: ‘this’ pointer cannot be null in well-defined C++ code; pointer may be assumed to always convert to true [-Werror,-Wundefined-bool-conversion]

  6. Fuz says:

    The same weird jump instruction exists with placement new, because the compiler skips the constructor call if the pointer is null:

    new (ptr) Type();

    test rax,rax
    je SkipConstructor
    mov rcx,rax
    call Type::Type

    __assume(ptr) (or other UB magic you care to use) will also remove this redundant null test.

  7. Random832 says:

    I like to call this sort of thing “spooky undefined behavior at a distance”. Many people assume these things are optimizer bugs, because they assume that even though the thing they’re doing is undefined by the standard, they assume their architecture’s known behavior for their mental model of how it works means it’s defined by the implementation they are using.

Comments are closed.

Skip to main content