Evil Compiler Tricks, and Checking for Pointer Math

My favorite programming geek hobby being integer overflows, this caught my eye –

"gcc silently discards some wraparound checks" https://www.kb.cert.org/vuls/id/162289

Basically, what it says is that code which looks like this:

============ snip ==============

        char *buf;
int len;

gcc will assume that buf+len >= buf.

As a result, code that performs length checks similar to the following:

len = 1<<30;
[...]
if(buf+len < buf) /* length check */
[...perform some manipulation on len...]

are compiled away by these versions of gcc

============ /snip ==============

Apparently, the compiler may be allowed by the C/C++ standard to do this, as pointers wrapping are not defined. However, the standard also says that optimizations may not change externally observable behavior in an application (aside from making it go faster), so I'm not so sure this is completely legal, but I Am Not A Standards Geek (IANASG), so I'll let them fight it out. The CERT advisory says to do this:

if((uintptr_t)buf+len < (uintptr_t)buf)

being alarmed by this problem, I then consulted what I do in SafeInt, which is:

template < typename T, typename U, typename E >

T*& operator +=( T*& lhs, SafeInt< U, E > rhs )

{

// Cast the pointer to a number so we can do arithmetic

SafeInt< uintptr_t, E > ptr_val = reinterpret_cast< uintptr_t >( lhs );

// Check first that rhs is valid for the type of ptrdiff_t

// and that multiplying by sizeof( T ) doesn't overflow a ptrdiff_t

// Next, we need to add 2 SafeInts of different types, so unbox the ptr_diff

// Finally, cast the number back to a pointer of the correct type

lhs = reinterpret_cast< T* >( (uintptr_t)( ptr_val + (ptrdiff_t)( SafeInt< ptrdiff_t, E >( rhs ) * sizeof( T ) ) ) );

return lhs;

}

So let me explain what exactly we're doing here, and why the CERT advice is bad (except where sizeof(T) == 1). First thing is to cast the pointer to an unsigned type that can always hold the bits in a pointer, and store that in a SafeInt. This happily coincides with the fact that unsigned int overflow behavior is actually specified. The second thing to do is to ensure that the offset multiplied by sizeof(T) does not overflow, and store the result in a ptrdiff_t (we could be moving backwards in the array), and finally check whether the addition ends up overflowing.

If you write code that only checks to see if the result of a pointer addition is greater than what you started with, your compiler might just remove the check, but what is much worse is that your code might not be working even if the compiler does not remove it. To write really correct code, you have to check BOTH the multiplication and the addition operations. As an aside, a problem I like to give to people as a puzzle is to write the correct code to determine if a + b * c yields a valid result or not. Surprisingly few people get it, but I think we can see here just how often we need to get exactly this operation right. Here's a trick that works for 32-bit unsigned code:

Unsigned __int64 result = (unsigned __int64)a + (unsigned __int64)b * (unsigned __int64)c;

If( (unsigned __int32)(result >> 32) )
return error;