ReadDirectoryChangesW reads directory changes, but what if the directory doesn’t change?

The Read­Directory­ChangesW function reads changes to a directory. But not all changes that happen to files in a directory result in changes to the directory.

Now, it so happens that nearly all changes to a file in a directory really do result in something happening to the file's directory entry. If you write to a file, the last-write time in the directory entry changes. If you rename a file, the name in the directory entry changes. If you create a file, a new directory entry is created.

But there are some changes that do not affect the directory entry. I've heard rumors that if you write to a file via a memory-mapped view, that will not update the last-write time in the directory entry. (I don't know if it's true, but if it's not, then just pick some other file-modifying operation that doesn't affect the directory entry, like modifying the contents of a file through a hard link in another directory, or explicitly suppressing file timestamp changes by calling Set­File­Time with a timestamp of 0xFFFFFFFF`FFFFFFFF.) The point is that since these changes have no effect on the directory, they are not recognized by Read­Directory­ChangesW. The Read­Directory­ChangesW function tells you about changes to the directory; if something happens that doesn't change the directory, then Read­Directory­ChangesW will just shrug its shoulders and say, "Hey, not my job."

If you need to track all changes, even those which do not result in changes to the directory, you need to look at other techniques like the change journal (a.k.a. USN journal).

The intended purpose of the Read­Directory­ChangesW function is to assist programs like Windows Explorer which display the contents of a directory. If something happens that results in a change to the directory listing, then it is reported by Read­Directory­ChangesW. In other words, Read­Directory­ChangesW tells you when the result of a Find­First­File/Find­Next­File loop changes. The intended usage pattern is doing a Find­First­File/Find­Next­File to collect all the directory entries, and then using the results from Read­Directory­ChangesW to update that collection incrementally.

In other words, Read­Directory­ChangesW allows you to optimize a directory-viewing tool so it doesn't have to do full enumerations all the time.

This design philosophy also explains why, if too many changes have taken place in the directory between calls to Read­Directory­ChangesW, the function will fail with an error called ERROR_NOTIFY_ENUM_DIR. It's telling you, "Whoa, like so much happened that I couldn't keep track of it all, so you'll just have to go back and do another Find­First­File/Find­Next­File loop."

Comments (11)
  1. Joshua says:

    Incidentally, ReadDirectoryChangesW doesn't appear to notice a file change until the handle is closed.

    [If you actually read the article, you'll understand why. Put 2 and 2 together. -Raymond]
  2. @Joshua

    Documentation says: "The operating system detects a change to the last write-time only when the file is written to the disk", which imho is not the same as CloseFile.

    Gosh, I wish there was at least "on file close notification"

  3. keithmo says:

    "I've heard rumors that if you write to a file via a memory-mapped view, that will not update the last-write time in the directory entry."

    I've certainly seen cases in which file data is updated but file metadata is not.

    I have a home-grown backup system that uses volume snapshots and robocopy to copy the system volume to an external drive, then computes the sha256 checksum of the source & destination files to verify. There are always a handful of files that fail the verification step. FWIW, the Explorer's thumbcache_*.* files seem to frequently trigger this. Manual inspection of the files shows their data has indeed changed, while their metadata has not.

    Of course, it's possible (but unlikely) that Explorer is updating these files then explicitly resetting the metadata.

    I don't know if this is related to memory-mapped access, or if it's a side-effect of volume snapshot access, or maybe both.

  4. cheong00 says:

    So is .NET FileSystemMonitor functions built on this? I had impression that on Windows OSs the event is guaranteed to be fired.

    On a related note, the Mono implementation on .NET v1.1 is depending on certain kernel extension named inotify that's known to discard event on rapid event firing. The difference caused me pain debuging unexpected bugs at that time.

  5. Gabe says:

    cheong: Yes, the .NET FileSystemWatcher class uses ReadDirectoryChangesW under the covers.

  6. jon says:

    The documentation on ReadDirectoryChangesW fails to mention the existence of ERROR_NOTIFY_ENUM_DIR.

  7. Dave says:

    The documentation on ReadDirectoryChangesW fails to mention the

    existence of ERROR_NOTIFY_ENUM_DIR.…/aa365465(v=vs.85).aspx

  8. Joshua says:

    This comes into play when trying to use an explorer window to watch a large file copy (when something other than explorer initiated the copy). It is necessary to press F5 periodically to determine the file size.

    Note that the CopyFile API advance reserves the disk space so things copied that way cannot be observed at all.

  9. Joe says:

    Gosh, I wish there was at least "on file close notification"

    You and me and my brother and just about everyone who has wanted to use this API.

  10. Leo Davidson says:

    I tried adding ERROR_NOTIFY_ENUM_DIR to the MSDN documentation but the server seems to be b0rked today, at least when it comes to accepting Community Content.

    Will try again later.

    Something to add to the discussion (which I'll also put in the MSDN comment):

    Remember to (re-)start watching the directory for changes before your first FindFirstFile/FindNextFile loop, otherwise you will miss changes that occur during the loop or between the end of the loop and the time you start monitoring for changes.

  11. Leo Davidson says:

    After three days trying to add a comment about ERROR_NOTIFY_ENUM_DIR to the MSDN page and (after lengthy delays at all stages) getting CGI errors, I give up.

    @Dave, maybe you can try doing it. Or maybe Microsoft can update their own documentation.

Comments are closed.