Eventually the window manager simply says that enough is enough

Many window manager operations are recursive, and eventually the window manager will simply say that enough is enough when the recursion goes too deep. We've seen this when you nest windows more than 50 levels deep or nest menus more than 25 levels deep, for example. (Note also that these limits may change in the future, so don't rely on being able to walk right up to the edge. Those values came from 32-bit Windows XP; I don't know if the limits have been dropped even further in subsequent versions of Windows, and I'm not sufficiently motivated to find out.)

A customer had some code which installed a message hook, and they found that the message hook was not called consistently. They tracked it down to another component in their application, and that component also installed a message hook. The contact actually came from the developer who maintained the other component: Did I write my message hook incorrectly? Am I accidentally messing up other message hooks in the system?

The developer included their hook-management code, and it didn't look obviously wrong. All code paths eventually called Call­Next­Hook­Ex, so there shouldn't be any hook-loss.

The customer was kind enough to include a copy of their program with instructions on how to trigger the problem, and stepping through the hook code quickly revealed the source of the problem. Maybe you can see it too. Here's a stack trace when the end of the hook chain is reached:

ChildEBP RetAddr
0011cdc4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011cdf8 768231eb user32!DispatchHookA+0x104 
0011ce38 76824260 user32!CallHookWithSEH+0x21 
0011ce6c 773e642e user32!__fnHkINLPMSG+0x71 
0011ceb0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011ceb4 76844787 user32!NtUserCallNextHookEx+0xc 
0011ced8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011cf14 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011cf48 768231eb user32!DispatchHookA+0x104 
0011cf88 76824260 user32!CallHookWithSEH+0x21 
0011cfbc 773e642e user32!__fnHkINLPMSG+0x71 
0011d000 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011d004 76844787 user32!NtUserCallNextHookEx+0xc 
0011d028 6fbdb1e0 user32!CallNextHookEx+0x71 
0011d064 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011d098 768231eb user32!DispatchHookA+0x104 
0011d0d8 76824260 user32!CallHookWithSEH+0x21 
0011d10c 773e642e user32!__fnHkINLPMSG+0x71 
0011d150 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011d154 76844787 user32!NtUserCallNextHookEx+0xc 
0011d178 6fbdb1e0 user32!CallNextHookEx+0x71 
0011d1b4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011d1e8 768231eb user32!DispatchHookA+0x104 
0011d228 76824260 user32!CallHookWithSEH+0x21 
0011d25c 773e642e user32!__fnHkINLPMSG+0x71 
0011d2a0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011d2a4 76844787 user32!NtUserCallNextHookEx+0xc 
0011d2c8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011d304 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011d338 768231eb user32!DispatchHookA+0x104 
0011d378 76824260 user32!CallHookWithSEH+0x21 
0011d3ac 773e642e user32!__fnHkINLPMSG+0x71 
0011d3f0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011d3f4 76844787 user32!NtUserCallNextHookEx+0xc 
0011d418 6fbdb1e0 user32!CallNextHookEx+0x71 
0011d454 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011d488 768231eb user32!DispatchHookA+0x104 
0011d4c8 76824260 user32!CallHookWithSEH+0x21 
0011d4fc 773e642e user32!__fnHkINLPMSG+0x71 
0011d540 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011d544 76844787 user32!NtUserCallNextHookEx+0xc 
0011d568 6fbdb1e0 user32!CallNextHookEx+0x71 
0011d5a4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011d5d8 768231eb user32!DispatchHookA+0x104 
0011d618 76824260 user32!CallHookWithSEH+0x21 
0011d64c 773e642e user32!__fnHkINLPMSG+0x71 
0011d690 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011d694 76844787 user32!NtUserCallNextHookEx+0xc 
0011d6b8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011d6f4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011d728 768231eb user32!DispatchHookA+0x104 
0011d768 76824260 user32!CallHookWithSEH+0x21 
0011d79c 773e642e user32!__fnHkINLPMSG+0x71 
0011d7e0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011d7e4 76844787 user32!NtUserCallNextHookEx+0xc 
0011d808 6fbdb1e0 user32!CallNextHookEx+0x71 
0011d844 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011d878 768231eb user32!DispatchHookA+0x104 
0011d8b8 76824260 user32!CallHookWithSEH+0x21 
0011d8ec 773e642e user32!__fnHkINLPMSG+0x71 
0011d930 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011d934 76844787 user32!NtUserCallNextHookEx+0xc 
0011d958 6fbdb1e0 user32!CallNextHookEx+0x71 
0011d994 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011d9c8 768231eb user32!DispatchHookA+0x104 
0011da08 76824260 user32!CallHookWithSEH+0x21 
0011da3c 773e642e user32!__fnHkINLPMSG+0x71 
0011da80 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011da84 76844787 user32!NtUserCallNextHookEx+0xc 
0011daa8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011dae4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011db18 768231eb user32!DispatchHookA+0x104 
0011db58 76824260 user32!CallHookWithSEH+0x21 
0011db8c 773e642e user32!__fnHkINLPMSG+0x71 
0011dbd0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011dbd4 76844787 user32!NtUserCallNextHookEx+0xc 
0011dbf8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011dc34 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011dc68 768231eb user32!DispatchHookA+0x104 
0011dca8 76824260 user32!CallHookWithSEH+0x21 
0011dcdc 773e642e user32!__fnHkINLPMSG+0x71 
0011dd20 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011dd24 76844787 user32!NtUserCallNextHookEx+0xc 
0011dd48 6fbdb1e0 user32!CallNextHookEx+0x71 
0011dd84 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011ddb8 768231eb user32!DispatchHookA+0x104 
0011ddf8 76824260 user32!CallHookWithSEH+0x21 
0011de2c 773e642e user32!__fnHkINLPMSG+0x71 
0011de70 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011de74 76844787 user32!NtUserCallNextHookEx+0xc 
0011de98 6fbdb1e0 user32!CallNextHookEx+0x71 
0011ded4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011df08 768231eb user32!DispatchHookA+0x104 
0011df48 76824260 user32!CallHookWithSEH+0x21 
0011df7c 773e642e user32!__fnHkINLPMSG+0x71 
0011dfc0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011dfc4 76844787 user32!NtUserCallNextHookEx+0xc 
0011dfe8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011e024 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011e058 768231eb user32!DispatchHookA+0x104 
0011e098 76824260 user32!CallHookWithSEH+0x21 
0011e0cc 773e642e user32!__fnHkINLPMSG+0x71 
0011e110 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011e114 76844787 user32!NtUserCallNextHookEx+0xc 
0011e138 6fbdb1e0 user32!CallNextHookEx+0x71 
0011e174 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011e1a8 768231eb user32!DispatchHookA+0x104 
0011e1e8 76824260 user32!CallHookWithSEH+0x21 
0011e21c 773e642e user32!__fnHkINLPMSG+0x71 
0011e260 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011e264 76844787 user32!NtUserCallNextHookEx+0xc 
0011e288 6fbdb1e0 user32!CallNextHookEx+0x71 
0011e2c4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011e2f8 768231eb user32!DispatchHookA+0x104 
0011e338 76824260 user32!CallHookWithSEH+0x21 
0011e36c 773e642e user32!__fnHkINLPMSG+0x71 
0011e3b0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011e3b4 76844787 user32!NtUserCallNextHookEx+0xc 
0011e3d8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011e414 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011e448 768231eb user32!DispatchHookA+0x104 
0011e488 76824260 user32!CallHookWithSEH+0x21 
0011e4bc 773e642e user32!__fnHkINLPMSG+0x71 
0011e500 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011e504 76844787 user32!NtUserCallNextHookEx+0xc 
0011e528 6fbdb1e0 user32!CallNextHookEx+0x71 
0011e564 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011e598 768231eb user32!DispatchHookA+0x104 
0011e5d8 76824260 user32!CallHookWithSEH+0x21 
0011e60c 773e642e user32!__fnHkINLPMSG+0x71 
0011e650 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011e654 76844787 user32!NtUserCallNextHookEx+0xc 
0011e678 6fbdb1e0 user32!CallNextHookEx+0x71 
0011e6b4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011e6e8 768231eb user32!DispatchHookA+0x104 
0011e728 76824260 user32!CallHookWithSEH+0x21 
0011e75c 773e642e user32!__fnHkINLPMSG+0x71 
0011e7a0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011e7a4 76844787 user32!NtUserCallNextHookEx+0xc 
0011e7c8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011e804 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011e838 768231eb user32!DispatchHookA+0x104 
0011e878 76824260 user32!CallHookWithSEH+0x21 
0011e8ac 773e642e user32!__fnHkINLPMSG+0x71 
0011e8f0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011e8f4 76844787 user32!NtUserCallNextHookEx+0xc 
0011e918 6fbdb1e0 user32!CallNextHookEx+0x71 
0011e954 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011e988 768231eb user32!DispatchHookA+0x104 
0011e9c8 76824260 user32!CallHookWithSEH+0x21 
0011e9fc 773e642e user32!__fnHkINLPMSG+0x71 
0011ea40 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011ea44 76844787 user32!NtUserCallNextHookEx+0xc 
0011ea68 6fbdb1e0 user32!CallNextHookEx+0x71 
0011eaa4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011ead8 768231eb user32!DispatchHookA+0x104 
0011eb18 76824260 user32!CallHookWithSEH+0x21 
0011eb4c 773e642e user32!__fnHkINLPMSG+0x71 
0011eb90 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011eb94 76844787 user32!NtUserCallNextHookEx+0xc 
0011ebb8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011ebf4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011ec28 768231eb user32!DispatchHookA+0x104 
0011ec68 76824260 user32!CallHookWithSEH+0x21 
0011ec9c 773e642e user32!__fnHkINLPMSG+0x71 
0011ece0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011ece4 76844787 user32!NtUserCallNextHookEx+0xc 
0011ed08 6fbdb1e0 user32!CallNextHookEx+0x71 
0011ed44 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011ed78 768231eb user32!DispatchHookA+0x104 
0011edb8 76824260 user32!CallHookWithSEH+0x21 
0011edec 773e642e user32!__fnHkINLPMSG+0x71 
0011ee30 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011ee34 76844787 user32!NtUserCallNextHookEx+0xc 
0011ee58 6fbdb1e0 user32!CallNextHookEx+0x71 
0011ee94 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011eec8 768231eb user32!DispatchHookA+0x104 
0011ef08 76824260 user32!CallHookWithSEH+0x21 
0011ef3c 773e642e user32!__fnHkINLPMSG+0x71 
0011ef80 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011ef84 76844787 user32!NtUserCallNextHookEx+0xc 
0011efa8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011efe4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011f018 768231eb user32!DispatchHookA+0x104 
0011f058 76824260 user32!CallHookWithSEH+0x21 
0011f08c 773e642e user32!__fnHkINLPMSG+0x71 
0011f0d0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011f0d4 76844787 user32!NtUserCallNextHookEx+0xc 
0011f0f8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011f134 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011f168 768231eb user32!DispatchHookA+0x104 
0011f1a8 76824260 user32!CallHookWithSEH+0x21 
0011f1dc 773e642e user32!__fnHkINLPMSG+0x71 
0011f220 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011f224 76844787 user32!NtUserCallNextHookEx+0xc 
0011f248 6fbdb1e0 user32!CallNextHookEx+0x71 
0011f284 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011f2b8 768231eb user32!DispatchHookA+0x104 
0011f2f8 76824260 user32!CallHookWithSEH+0x21 
0011f32c 773e642e user32!__fnHkINLPMSG+0x71 
0011f370 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011f374 76844787 user32!NtUserCallNextHookEx+0xc 
0011f398 6fbdb1e0 user32!CallNextHookEx+0x71 
0011f3d4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011f408 768231eb user32!DispatchHookA+0x104 
0011f448 76824260 user32!CallHookWithSEH+0x21 
0011f47c 773e642e user32!__fnHkINLPMSG+0x71 
0011f4c0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011f4c4 76844787 user32!NtUserCallNextHookEx+0xc 
0011f4e8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011f524 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011f558 768231eb user32!DispatchHookA+0x104 
0011f598 76824260 user32!CallHookWithSEH+0x21 
0011f5cc 773e642e user32!__fnHkINLPMSG+0x71 
0011f610 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011f614 76844787 user32!NtUserCallNextHookEx+0xc 
0011f638 6fbdb1e0 user32!CallNextHookEx+0x71 
0011f674 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011f6a8 768231eb user32!DispatchHookA+0x104 
0011f6e8 76824260 user32!CallHookWithSEH+0x21 
0011f71c 773e642e user32!__fnHkINLPMSG+0x71 
0011f760 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011f764 76844787 user32!NtUserCallNextHookEx+0xc 
0011f788 6fbdb1e0 user32!CallNextHookEx+0x71 
0011f7c4 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011f7f8 768231eb user32!DispatchHookA+0x104 
0011f838 76824260 user32!CallHookWithSEH+0x21 
0011f86c 773e642e user32!__fnHkINLPMSG+0x71 
0011f8b0 768447aa ntdll!KiUserCallbackDispatcher+0x2e 
0011f8b4 76844787 user32!NtUserCallNextHookEx+0xc 
0011f8d8 6fbdb1e0 user32!CallNextHookEx+0x71 
0011f914 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011f948 768231eb user32!DispatchHookA+0x104 
0011f988 76824260 user32!CallHookWithSEH+0x21 
0011f9bc 773e642e user32!__fnHkINLPMSG+0x71 
0011f9d0 00030000 ntdll!KiUserCallbackDispatcher+0x2e 
0011fa28 6fbdb1e0 0x30000
0011fa64 16846e6e contoso!ContosoWindowEventHook+0x11b 
0011fa98 768231eb user32!DispatchHookA+0x104 
0011fad8 76824260 user32!CallHookWithSEH+0x21 
0011fb0c 773e642e user32!__fnHkINLPMSG+0x71 
0011fb20 00030000 ntdll!KiUserCallbackDispatcher+0x2e 
0011fb7c 768292a9 0x30000
0011fba8 6b2ce010 user32!PeekMessageW+0xfb 

As you can see, a third component in their application installed at least thirty-five hooks. After the thirty-fifth hook, the window manager stepped in and said, "That's it, I'm cutting you off."

Now, the limit isn't actually thirty-five. The window manager keeps dispatching hooks until the kernel stack starts running low, and then it gives up. This happens with a lot of recursive algorithms: The window manager plays the game for a while, but when it looks like you're about to bluescreen, it stops short and says, "Okay, I'm not going to do that any more."

The developers now got to take their problem to the developer responsible for the Contoso component, and figure out why it's installing so many hooks. Maybe that component could try to consolidate identical hooks. Or maybe it's a leak. They never did report back (not that I was expecting them to).

Bonus chatter: Why is hook dispatch done recursively? Shouldn't it be done iteratively?

Remember that windows hooks came from 16-bit Windows, where economy was paramount. And the existing Call­Next­Hook pattern was preserved, though it changed to Call­Next­Hook­Ex, where you pass the hook handle directly instead of its address. One advantage of the Call­Next­Hook­Ex model over an iterative model is that explicitly forwarding to the previous hook lets you do work on the back end. I.e., you can forward the call down the chain, and then do something when control returns. This is the sort of thing you probably use a lot when you subclass a window or override a method in a derived class and call the base class from your override.

Comments (34)
  1. Alex says:

    The limit is lower in 64-bit Windows, apparently because those in charge didn't see fit to increase the kernel stack space when the size of a pointer doubled.  In some cases, the limit can become depressingly low.

  2. alegr1 says:

    My buest guess that Contoso turd was installing the hook again based on some event.

    This reminds Microsoft again to reduce inter-process coupling in Windows, even at a cost of some crapware not working anymore. Enough is enough already.

    [The thread was installing a hook on itself, so your crackdown on inter-process hooks has no effect here. -Raymond]
  3. Lockwood says:

    I'm afraid to say I want to put my pedant hat on.

    I like to avoid the Nitpicker's Corner, but…

    "The developers now got to take their problem to the developer responsible for the Contoso component"

    I needed to reread the start of that a couple of times to make sure I hadn't managed to skip a word, or worse do that thing where you skip to the line below and read on slightly confused but pretty much making sense. (ie this could have a line skip in it as I enter it to say "I needed to reread slightly confused but …")

    The developers have now got to?

    The developers now have got to?

    Not 100% sure which we want, but it seems to want a "have"

    ["get to" = "have the opportunity to". "The developers now had the opportunity to take their problem to…" -Raymond]
  4. Martin Bonner says:

    @Lockwood:  I think Raymond probably needed "The developers then got to …" (being the past tense of "The developers now got to …").

  5. Bob says:

    "didn't see fit to increase the kernel stack space when the size of a pointer doubled"

    The size of a kernel stack doubled in 64-bit windows.

  6. Joshua says:

    Wow my window nest stack is already 16 and threatening to get deeper. (Nested child controls. The apparent stack is more like 3).

    Modularization in .NET tends to create lots of useless child windows that just contain other children.

  7. I think I'd rather the process crashed than have window messages randomly undelivered in these situations.

    The non-delivery can leave the process in an undefined state, so crashing the process there & then is safer and makes bugs easier to detect and fix.

    Window messages are like function calls. Skipping a function call because the stack is low would be unthinkable.

    I suspect this also explains why accidentally making a message handler infinitely recursive will blow the stack and crash your process in some situations (no hooks installed) but result in weird behaviour, and no crash, in others (a hook happens to be installed).

  8. Lockwood says:

    @Martin, Raymond

    That also makes sense, with the "then got to". Surely the present tense would be "now get to"?

    At least I was paying attention to the article to notice such a fine detail!

    ["then got to" is probably better. I sometimes get into this weird narrative state where I use present-tense adverbs with past-tense verbs… -Raymond]
  9. Anonymous Coward says:

    I think Leo is right. After all, when you call a normal function too recursively, your process goes down with an ‘Out of stack space’ error, so I think the same should happen here.

  10. John says:

    In principal I agree with the guys saying you should just exhaust the stack and die, but in reality these hooks can be installed in other programs so from the perspective of the user it's going to be explorer.exe that crashes.

  11. "your process goes down with an ‘Out of stack space’ error, so I think the same should happen here."

    Except it won't be "your process", it would be a blue screen of death because the kernel would blow its stack – right…?  If so, what do you propose if not a BSOD?

  12. alegr1 says:


    Who was talking about kernel mode? So far, we're in the userland.

    [Um, "The window manager keeps dispatching hooks until the kernel stack starts running low, and then it gives up." -Raymond]
  13. 640k says:

    Probably some lousy DRM library which tries to allocate all available hooks to prevent debugging tools.

  14. Why are the limits being *lowered* for hooks, nested windows, menus?  Why is the stack space not increased slightly in this age of 8 GB RAM computers?

    At this rate, there will eventually be a limit of 5 and everything will break.  (I exaggerate, but only slightly.)

    As an application developer, what limit should I target?  How can I even really target a limit, if it's a moving target going *downwards*?

    (Yes, I realize that if you are nesting 50 windows, "you're doing it wrong" – but it's not unreasonable to think that a decently-architected program could go several levels deep – and the question is, how much is too much?  How much will too much be in 10 years?)

  15. Matt says:

    The real problem is doing recursion at all in kernel-mode. Kernel-mode stacks are NON_PAGED, and when you run out your system dies.

    If you're writing kernel-mode code (yes, I'm looking at you, Win32k.sys) and you decide to solve your problem using recursion, then You're Doing It Wrong (TM).

  16. Yuhong Bao says:

    "Except it won't be "your process", it would be a blue screen of death because the kernel would blow its stack – right…?  If so, what do you propose if not a BSOD?"

    I think the commenters were proposing the idea of raising an user-mode exception.

  17. gdalsnes says:

    Callbacks from kernel mode are just evil. Now that all pc's have several cpu cores, maybe it's time to dust of the pre NT4 CSRSS Win32 design. Wine also runs entirely in user mode and seems to get away with it.

  18. Yuhong Bao says:

    arghhhhhhhhhhh: Especially with the introduction of per-session CSRSS with WinFrame.

  19. Random832 says:

    Could the existing API pattern be implemented in a tail-recursive way? Or could a new iterative model be added? We're no longer stuck with the "each process forwards the message to the next one" model for clipboard viewers.

    [See final paragraph. -Raymond]
  20. alegr1 says:

    [Um, "The window manager keeps dispatching hooks until the kernel stack starts running low, and then it gives up." -Raymond]

    What I see in this post is user mode stack.

    [When you're debugging the problem, you typically get to see only the user stack, so that's what I showed. It should be clear that if the user-mode stack is this deely nested in kernel callbacks, the kernel stack must be deeply nested with the callback invokers. -Raymond]
  21. @arghhhhhhhhh says:

    > Wine also runs entirely in user mode and seems to get away with it.

    Given it's (in)compatibility record, it doesn't get away with it well.

  22. "Now that all pc's have several cpu cores, maybe it's time to dust of the pre NT4 CSRSS Win32 design."

    Very good point.  Wikipedia for CSRSS seems to indicate that GDI and window handling is done in win32k.sys, and that they were moved there for Windows NT 4.0 for performance reasons.  Why they can't be moved back to user mode is beyond me…  It's 2012 and I doubt it has to be done for performance reasons any more.  Kernel mode is a dangerous place that has a higher risk of causing a system-wide crash or security failure; it seems like you'd want to keep what's in there to a minimum.

    [Presumably the change was made to improve performance by N%. Reversing it would decrease performance by N%. "Hey, why did performance drop by N%?" "Sorry, I thought you weren't using that performance any more." -Raymond]
  23. Joshua says:

    Given it's (in)compatibility record, it doesn't get away with it well.

    We're talking about performance here. Except when Wine is copying bitmaps between GDI+ and GDI for programs that switch back and forth a lot it gets better performance than Windows. It is thought that this is mainly due to the performance advantage of ext3 over NTFS rather than Wine's architecture though.

  24. alegr1 says:

    [When you're debugging the problem, you typically get to see only the user stack, so that's what I showed. It should be clear that if the user-mode stack is this deely nested in kernel callbacks, the kernel stack must be deeply nested with the callback invokers. -Raymond]

    I don't think the whole chain of calls ever leaves the user mode. It's not possible to interleave usermode and kernel mode stacks. You're either in user mode, or entered the kernel until you return. You can't have part of your callstack in kernel mode and then in usermode again.

    [You can see control leave user mode at NtUserCallNextHookEx and then return to user mode at KiUserCallbackDispatcher. Here's another example of a stack bouncing between kernel mode and user mode. -Raymond]
  25. @alegr1

    The answer to your dilema is in that call stack. KiUserCallbackDispatcher. This is an NTDLL kernel mode to usermode callback function. So this is transitioning between kernel mode and usermode. The transition to kernel mode is in NtCallNextHookEx, and then kernel mode switches back to user mode and calls KiUserCallbackDispatcher.

    So yes, this does leave user mode. Anyway, since the Window Manager has been kernel mode since NT4, and Windows Hooks are obviously handled by the Window Manager, how would you suggest this occurs without the transition to kernel mode?

  26. Joe says:

    @alegr1 "I don't think the whole chain of calls ever leaves the user mode":

    "ntdll!KiUserCallbackDispatcher": This is a callback from win32k. The kernel-mode stack is waiting for a NtCallbackReturn to allow the kernel-mode stack to unwind.

    As a general rule, don't try and criticize Raymond's lack of understanding of Windows. There aren't many people that understand it better than he. If you disagree with you, 98% of the time, it's because you're wrong.

  27. Joshua says:

    Wait what wow that's a kernel callback chain on the stack. Time to dust-off the native call loader and see if it's possible to bluescreen the kernel by overrunning its stack.

  28. alegr1 says:

    That's just great. Let's nest some 30 hooks, and then issue random IOs and IOCTLs.

  29. Joshua says:

    @alegr1: Don't forget IO interrupts. I've got a device driver in archive that wants >4k for its bottom half handler.

  30. Yuhong Bao says:

    @Joshua: AFAIK each time a user-mode callback is done, a new kernel stack is created (on Vista and later) to prevent this. Of course, there is a limit to how many such stacks can be created, and when you hit this limit, win32k get a failure code which is then passed back to the app.

  31. ender says:

    @Joshua: a driver for certain CPU maker's integrated graphics? :)

  32.    "Presumably the change was made to improve performance by N%. Reversing it would decrease performance by N%. "Hey, why did performance drop by N%?" "Sorry, I thought you weren't using that performance any more." -Raymond"

    Actually, looking in MSDN, hasn't something like that already happened with Vista?  msdn.microsoft.com/…/hh437350(v=vs.85).aspx – it looks like the windowing system might be moving in the direction of user mode anyway.

    [Um, DirectComposition is not the window manager. -Raymond]
  33. "DirectComposition is not the window manager."

    Right, but it interacts very closely with it and I'd imagine it is very performance-critical – yet it's in user mode.  If they were able to "get away" with putting something critical like that in user mode, they could probably "get away" with moving the entire window manager out of win32k.sys.

    [The internal performance of DirectComposition is critical, but the performance of the communication channel between DirectComposition and the application is not as critical. And moving the window manager into kernel mode is all about reducing the communication latency. -Raymond]
  34. 640k says:

    "Reversing it would decrease performance by N%."

    And decrease BSODs by N%.

Comments are closed.

Skip to main content