What happens if I have multiple asynchronous ReadDirectoryChangesW calls outstanding on the same directory handle?

A customer ran into an issue with the Read­Directory­ChangesW function. It was rather complicated, but one corner of the issue boiled down to the following: The customer had multiple asynchronous calls to Read­Directory­ChangesW outstanding. What if anything can be said about the results?

First, we'll give the answer while wearing kernel-colored glasses: The kernel completes the calls to Read­Directory­ChangesW in the order they were issued, so that the earliest call receives the first available batch of changes, and the next call receives the next available batch of changes, and so on.

Now let's take off our kernel-colored glasses and see what this means for the application.

Even though the kernel completes the calls in the order they were issued, the fact that the scheduler can preempt a thread at any time means that even though the two operations complete in sequence, the two threads that handle the completion are at the mercy of the scheduler, and they will race against each other, so the end result is unpredictable anyway.

Comments (10)
  1. Killer{R} says:

    but if caller of ReadDirectoryChangesW specified completion routines – they will be called via APC. And APCs are called in same order as they queued. And since kernel queues APC when it completes calls and since calls completed in same order as requested – then completion routines should be called also in same order as corresponding ReadDirectoryChangesW-s were called, aren’t they?

    1. Joshua says:

      Stacking them on APC doesn’t work either because APC can be on top of APC.

      However if they were asynchronous overlapped calls picked up with WaitForMultipleObjects in the order they were issued the code suddenly starts working.

      1. Killer{R} says:

        Whats the problem with APC on top of APC? User-mode APCs always called in alertable wait, so everything in your hands. And it doesn’t matter if user mode APC called from alertable wait executed somewhere in code that executes another APC, if you’re careful enough.

        1. Joshua says:

          Well if you want occasionally the later one to be processed first …

          1. Killer{R} says:

            The main thing about APC queue that it is a queue. When thread comes to waitable wait – scheduler looks at queue, retrieves from it next pending item, and executes it, so flow is it called like a callback inside that alertable wait function. It doesn’t care what is behind current context – normal flow or something inside another APC. Actually I’d not be very wrong saying that thread’s user-mode code execution start from kernel-colored glasses (c) :) actually is just a first user-mode APC queued to thread.
            So APCs executed always in same order as queued, to break things you need to perform alertable wait before actually doing job that particular APC intended to do. Like:
            VOID CALLBACK APCProc(ULONG_PTR dwParam)
            SleepEx(0, TRUE);
            And this again doesn’t break APC order, it just breaks your code :) Of course you should be careful and do not use not only SleepEx(..TRUE) but also anything else that may execute alertable wait inside, that means anything higher level than base kernel32 API. My favorite example of such API-with-surprise that calls alertable wait inside is CreateToolhelp32Snapshot(TH32CS_SNAPMODULE).

      2. I don’t think WaitForMultipleObjects will (reliably) work, because if more than one of the events is already set when you call it there’s no way to determine which event was set first. The same presumably applies if multiple events are set during a period when the thread is running a kernel-mode APC, as described in the documentation for PulseEvent.

        I think you could use APC, carefully. For example, make the APC calls do nothing but push the information into an ordered queue for another thread to take care of.

        1. Joshua says:

          As Raymond said, calls finish in the order issued.

          1. DWalker says:

            Yes, but don’t forget what he said in the last paragraph.

      3. JW says:

        I think you have a misunderstanding about how APCs work. They are only delivered to your thread when you are in an Alertable Wait State (e.g. SleepEx() ), not at any old time the kernel feels like it. This means you will never get an out-of-order APC unless you are specifically doing something incorrect in the APC handler. Generally I’d discourage putting an APC handling routine into any sort of wait state, let alone an alertable one, so I don’t see any situation in decent coding that would result in APC delivery being out of order.

        1. I was assuming the I/O were using an IOCP, not APCs.

Comments are closed.

Skip to main content