Why does my asynchronous I/O request return TRUE instead of failing with ERROR_IO_PENDING?


A customer reported that their program was not respecting the FILE_FLAG_OVERLAPPED flag consistently:

My program opens a file handle in FILE_FLAG_OVERLAPPED mode, binds it to an I/O completion callback function with Bind­Io­Completion­Callback, and then issues a Write­File against it. I would expect that the Write­File returns FALSE and Get­Last­Error() returns ERROR_IO_PENDING, indicating that the I/O operation is being performed asynchronously, and that the completion function will be called when the operation completes. However, I find that some percentage of the time, the call to Write­File returns TRUE, indicating that the operation was performed synchronously. What am I doing wrong? I don’t want my thread to block on I/O; that’s why I’m issuing asynchronous I/O.

When you specify FILE_FLAG_OVERLAPPED, you’re promising that your program knows how to handle I/O which completes asynchronously, but it does not require the I/O stack to behave asynchronously. A driver can choose to perform your I/O synchronously anyway. For example, if the write operation can be performed by writing to cache without blocking, the driver will just copy the data to the cache and indicate synchronous completion. Don’t worry, be happy: Your I/O completed even faster than you expected!

Even though the I/O completed synchronously, all the asynchronous completion notification machinery is still active. It’s just that they all accomplished their job before the Write­File call returned. This means that the event handle will still be signaled, the completion routine will still run (once you wait alertably), and if the handle is bound to an I/O completion port, the I/O completion port will receive a completion notification.

You can use the Set­File­Completion­Notification­Modes function to change some aspects of this behavior, giving some control of the behavior of the I/O subsystem when a potentially-asynchronous I/O request completes synchronously.

Comments (12)
  1. Joshua says:

    Heisenbug!

    Seriously, when using async IO, read the documentation thoroughly. There may be edge cases you need to handle or things blow up in your face.

  2. Adam Rosenfield says:

    Along a similar vein, could you also have an asynchronous read that completes instantly due to the data already being in the cache?  Is there a way to determine how big the cache is?

  3. asf says:

    Set­File­Completion­Notification­Modes is broken on sockets

  4. waleri says:

    @Joshua

    I think documentation on async IO (completion port) could have been much better.

    For example, it may clearly state, that IO can be completed synchronously *and* that event/completion port will be signaled. I am not sure, that the case of synchronous completion *and* signaling is clearly stated anywhere at all.

    List of function, that supports completion port (see GetQueuedCompletionStatus) is incomplete, but it is not mentioned, that is incomplete. Also I read somewhere (now can't remember where) that some async requests may return FALSE with GetLastError() == 0 (no error). All of the above is pretty important and could have been mentioned in once place somewhere (like in this article msdn.microsoft.com/…/aa365198(VS.85).aspx)

  5. Dean Harding says:

    That was always one aspect of async I/O I was never quite sure I got right… good to know there's nothing special I need to do if the operation returns TRUE instead of an error and ERROR_IO_PENDING.

  6. Worf says:

    Yeah, that helped too – even though Windows CE doesn't support async I/O, it does for sockets. All the code I saw (almost no one uses async I/O as it's probably flaky, though I had to simulate the behavior because one program did) checked to see if it returned TRUE (no error) or FALSE with GetLastError looking for WSA_IO_PENDING.

    Didn't help that the documentation implies both were valid return values that the I/O was pending. Of course, it makes sense now (just that the return call took longer than the notification – it is asynchronous) given today's post.

  7. Teo says:

    The most interesting question about async IO is "why there's no async CreateFile?" If you want a non-blocking GUI, you need worker threads to host CreateFile, so you can just skip all the async stuff anyway.

    [How would an async CreateFile signal completion? The completion key is associated with the file handle, but until you have a handle you can't call CreateIoCompletionPort to create the association! (Okay, I guess it could return a "placeholder handle" that you could then use to create the association. Wait, now there's a race condition. Well, I guess you could do something about that too.) (Today, you can issue the synchronous CreateFile and use CancelSynchronousIo to cancel it.) -Raymond]
  8. Teo says:

    AFAIK, no, you can't. for CancelSynchronousIo to work, all drivers involved in a particular IO operation MUST support cancelable ARPs. Besides, this is a Windows 6 API, and all statistics I could find tells me that XP on the desktop and 2003 on the server have the major market share. After 3-5 years, maybe, but definitely not "now". The whole point of existence of an OS is to make application developer life's easier. So I'll leave the race condition problem as an exercise to the kernel team. After all, they needed 20 years to create CancelSyncIo function, and I bet that before they do, there were many excuses like "oh that would deadlock, we cannot do it"

    [Not sure what you're getting at. It sounds like you want Microsoft Research to finish work on their time machine. -Raymond]
  9. Teo says:

    Doh. I obviously meant IRPs (like the stuff drivers jiggle all the time), and not ARP (like the Add/Remove Programs)

  10. Joshua says:

    Hmmm, Async. CreateFile.

    Taking a page from the Linux kernel (the use there is hard umount of active filesystem), have CreateFile return a real file handle immediately, and if the async operation fails, the io completion port signals that and afterwords all operations on that handle fail with ENODEV or its moral equivalent until the handle is closed.

    Oh well, I don't bother with async IO. It's a lot less painful to spawn threads and with a thread pool the runtime cost of creating new threads doesn't outweigh the developer cost of async io anymore.

    [The catch is if the async operation fails before you get a chance to call Create­Io­Completion­Port. Then the operation completes with a null Completion­Key, which means you don't know why you're being notified. -Raymond]
  11. yuhong2 says:

    "Besides, this is a Windows 6 API, and all statistics I could find tells me that XP on the desktop and 2003 on the server have the major market share."

    And it is incorrectly documented to require _WIN32_WINNT >= 0x0501:

    connect.microsoft.com/…/setfilecompletionnotificationmodes-requires-win32-winnt-to-be-0x0600-not-0x0501

  12. Mike Dimmick says:

    @Adam Rosenfield: yes, it can, and (particularly) if the OS is doing read-ahead, it will. You can disable read-ahead with FILE_FLAG_NO_BUFFERING, of course, and FILE_FLAG_RANDOM_ACCESS will reduce it. If you specify FILE_FLAG_SEQUENTIAL_SCAN it makes large read-aheads and aggressively removes data that's already been read from the cache, while not specifying either flag causes it to heuristically determine how much to read-ahead and where.

    @Worf: all the examples for serial I/O on Win32 use asynchronous operations, but CE doesn't support it! It made emulating the DOS behaviour of serial ports, while supporting abort, pretty painful. WaitCommEvent can only be aborted by calling SetCommMask, which is particularly unintuitive.

    @yuhong2: doc bugs should be reported on the MSDN page itself, using the feedback section at the bottom, not on Connect. I've added an annotation to the page.

Comments are closed.