What possible use are those extra bits in kernel handles? Part 1: Sentinels

Kernel handles are always a multiple of four; the bottom two bits are available for applications to use. But why would an application need those bits anyway?

The short answer is extending the handle namespace. The long answer will take a few days to play out. (This series was written in response to Igor Levicki being unable to imagine "how this can save anything (in terms of performance)". Then again, who said that it had anything to do with performance? Actually, I'm surprised that my dear readers weren't familiar with the techniques described in this series. Perhaps I shouldn't have written this series and merely replied, "If you can't think of how this could be useful, then you are not my target audience." On the other hand, reader Aaargh! believes that whoever thought to make the bottom two bits of handles available to applications should receive an asswhooping.)

But we'll start with a warm-up. If you need some sentinel values for a HANDLE, you need to make sure your chosen sentinel value will never conflict with a valid HANDLE value. If you decide that your sentinel value is something like

// code in italics is wrong

then your program is going to start acting really strange if the kernel ever gave you handle value of 0x1234. Knowing that kernel handles are always multiples of four means that you can choose a value that isn't a multiple of four and use it as your sentinel value.


Since 0x1233 is not a multiple of four, you can rest assured that no actual kernel handle will have this value, and you can write your logging function like this:

void LogOutput(HANDLE hOutput, LPCVOID pv, DWORD cb)
 if (hOutput == NULL) {
   // logging disabled
 } else if (hOutput == DEBUGWINDOW_HANDLE) {
  AddToDebugWindow(pv, cb);
 } else {
  DWORD cbWritten;
  WriteFile(hOutput, pv, cb, NULL, &cbWritten);

Since you can't WriteFile to a window handle, your logging function has to do something special if somebody decided that their output should go to the debug window. But if they chose to log to a normal kernel object (a file, the console, a serial port, whatever) then you can just write the data to that kernel object.

You've already seen this before; you just didn't realize it. The special values for INVALID_HANDLE_VALUE and kernel pseudo-handles such as GetCurrentProcess are not multiples of four for exactly this reason.

Now, sure, you could have defined your own LogHandle type and have all the logging go through that type instead of just logging to HANDLEs:

class LogHandle {
  static LogHandle *GetDebugLogHandle();
  BOOL IsDebugWindow();
  HANDLE GetKernelHandle();
  static LogHandle *CreateFromKernelHandle(HANDLE KernelHandle);
  ~LogHandle() { }

  LogHandle(BOOL IsDebugWindow, HANDLE KernelHandle);
  static LogHandle DebugWindow;

  BOOL IsLogToDebugWindow;
  HANDLE RegularKernelHandle;

Throughout, your program would use pointers to LogHandles instead of actual handles, using functions like these to convert between them:

// Does not take ownership of the handle
LogHandle::LogHandle(BOOL IsDebugWindow, HANDLE KernelHandle)
    : IsLogToDebugWindow(IsDebugWindow)
    , RegularKernelHandle(KernelHandle)

LogHandle LogHandle::DebugWindow(TRUE, NULL);

LogHandle* LogHandle::GetDebugWindowLogHandle()
  return &DebugWindow;

BOOL LogHandle::IsDebugWindow()
  return IsLogToDebugWindow;

HANDLE LogHandle::GetKernelHandle()
  return RegularKernelHandle;

LogHandle *LogHandle::CreateFromKernelHandle(HANDLE KernelHandle)
  return new LogHandle(FALSE, KernelHandle);

Or you could make everybody pass two parameters instead of one. For example, a class that went

class SomeObject {
  SomeObject(int SomeParameter, BOOL SomeParameter,
             HANDLE LogHandle);
  HANDLE LogHandle; // log to this handle

would have to change to

class SomeObject {
  SomeObject(int SomeParameter, BOOL SomeParameter,
             BOOL LogToDebugWindow, HANDLE LogHandle);
  BOOL LogToDebugWindow; // if TRUE, log to window
  HANDLE LogHandle; // if not logging to window, then log to here

Either way is an awful lot of work just to define a sentinel value. But still, at least you can avoid the need for a sentinel value by just passing more parameters. But sometimes that option isn't available. We'll look at that next time.

Comments (13)
  1. steven says:

    Hmmm… 1234 isn’t a multiple of 4, unless you mean 0x1234, of course.

    [Thanks. Fixed. -Raymond]
  2. keithmo says:

    I wonder if “next time” will concern the disgusting “set the low bit of the event handle in an OVERLAPPED structure to suppress the I/O completion indication” hack…

    [Um, that topic was the inspiration for this one. Try clicking that first link. -Raymond]
  3. John says:

    This whole thing just seems like too much of a hack for me.  I much prefer adding a second parameter to a function.  My personal feeling is that an opaque handle should mean one thing and one thing only.  If you want to associate some extra information with a handle perhaps you should implement SetHandleLong() or something.

  4. Koro says:

    But one must still be careful. "Console" pseudo-handles returned by GetStdHandle have some lower bits set, that WriteFile uses to determine if it should redirect the call to WriteConsoleA.

    All this because console handle are not real handles.

  5. Igor Levicki says:

    I am glad that my "ignorance" on the subject has inspired Raymond to write an educational article.

    As for my imagination shortage — still there is nothing here to convince me that such implementation is actually faster. Why not write two test cases along with some timing code and post them in one of the future blog posts?

  6. Dean Harding says:

    "still there is nothing here to convince me that such implementation is actually faster"

    Quote from Raymond: "Then again, who said that it had anything to do with performance?"

  7. Bruno Martínez says:

    You can use boost::optional to do this cleanly.

  8. Maurits says:

    Of course, just because you /can/ doesn’t mean you /should/.

    // works, but ew

    switch (h) {

    // technically NULL % 4 == 0 but, hey, it’s special

    case NULL: …; break;

    case INVALID_HANDLE_VALUE: …; break;

    // probably need to unwrap this

    case GetCurrentProcess(): …; break;




    default: assert((h % 4) == 0); WriteFile(h, …);


  9. MadQ says:

    I’ve taken advantage of those lower two bits in a different way when I was writing a little debugging tool. It needed to keep track of process IDs (which aren’t handles, but also have the lower two buts set to zero), and it was important to be able to look up process handles by process ID very fast. It used a hash table with the simplest hash function ever: right-shift the process ID by two.

    Nitpickery: "But if the lower two bits weren’t always zero, you wouldn’t need a hash function at all!"

    Yeah, but then I wouldn’t have had a possibly mildly interesting anecdote to tell.

  10. Anon says:

    For some reason this reminds of that Mistakes Demotivator with the caption "It could be that the purpose of your life is only to serve as a warning to others"

  11. Me says:

    John wrote on Wednesday, August 27, 2008 11:10 AM:

    My personal feeling is that an opaque handle should mean one thing and one thing only.

    But if this were always followed, we would not know the FILENOTFOUND value for a boolean enum:


    But I think we all know this same from some other circumstance, don’t we?

  12. Erzengel says:

    I can see what Raymond is getting at, but I personally would have done:

    class LogHandle



    virtual ~LogHandle(){}

    virtual void Write(LPCVOID ToWrite, DWORD Len)=0;


    class LogFileHandle : public LogHandle


    HANDLE regularHandle;




    virtual void Write(LPCVOID ToWrite, DWORD Len)


    DWORD WrittenLen = 0;

    WriteFile(regularHandle, ToWrite, Len, 0, &WrittenLen);



    class LogDebugHandle : public LogHandle



    virtual void Write(LPCVOID ToWrite, DWORD Len)


    AddToDebugWindow(ToWrite, Len);



    __declspec(dllexport) LogHandle* NewLogHandle(HANDLE WriteTo)


    return new LogFileHandle(WriteTo);


    __declspec(dllexport) void FreeHandle(LogHandle* ToDelete)


    delete ToDelete;


    __declspec(dllexport) LogHandle* GetDebugHandle()


    //Yes, I could use a singleton, but I’d need a

    //way to allow the “free” function to recognize

    //the static variable, and that’s beyond

    //the scope of this example.

    return new LogDebugHandle();


    __declspec(dllexport) void LogOutput(LogHandle* LogTo, LPCVOID ToLog, DWORD Len)


    LogTo->Write(ToLog, Len);


    Yes, this may be slower (function pointers rather than if branches), but really, do you need to be fast when outputting to a file, debug window, etc? Wouldn’t the actual writing be orders of magnitude slower than a function pointer? Plus, this is extensible. What if I want to output via WM_COPYDATA? Or across a network? I could create COPYDATA_HANDLE and add an if, but how would I know what window I’m sending to? Using a class allows me a great deal of flexibility.

    So I honestly, sincerely want to get your opinions: how would using a HANDLE sentinal be better than the above?

    [It’s a lot easier if you’re retrofitting an existing code base that originally used HANDLEs. -Raymond]
  13. Extending the namespace to cover new object types.

Comments are closed.