If somebody creates or deletes a file in a directory while I am enumerating its contents, what happens?

Suppose you are enumerating the contents of a directory via Find­First­File and Find­Next­File, and while your loop is running, somebody creates or deletes a file in that directory. What can be said about the result of the enumeration?

Are you guaranteed that the effect of the creation or deletion will not be visible in the enumeration? In other words, will the enumeration show the contents of the directory at the time the enumeration started, even if the directory contents change later? Or maybe the guarantee is the opposite: Are you guaranteed that the deleted file will be removed from the enumeration and the created file will be added to it?

Actually, there is no guarantee either way.

What you can say is that files which were neither created nor deleted during the enumeration will be enumerated. If a file is created during the enumeration, it is unspecified whether it will appear in the enumeration. Similarly, if a file is deleted during the enumeration, it is unspecified whether it will be removed from the enumeration. (And if a file is deleted, then recreated, it may show up more than once!)

We noted some time ago that the Find­First­File function does some batching, and the Find­First­File­Ex function lets you influence the size of the batches. If a file is deleted while it is sitting in a batch, then the Find­Next­File function will return it, because it is just handing data out from the most recent batch. And if an error occurs during the enumeration, you should just give up, because there's no meaningful recovery.

Comments (16)
  1. ZLB says:

    “Are you guaranteed that the deleted file will be removed from the enumeration….”

    Doesn’t this violate causality? A file that has already been returned by a call could be deleted before the enumeration is complete!

    1. skSdnW says:

      Find­Next­File/FindClose cannot fail with a timeout, you could retrieve one file per day if you wanted to. Just having enumerate rights would allow people to prevent deletion of files if there was some kind of guarantee that the returned file was valid until the next call to FindNext/Close.

  2. skSdnW says:

    And what if the ACL on the directory you are enumerating changes and no longer allows you to list the files? I’m guessing it will finish the enumeration every time because it probably has a open handle to the directory…

  3. Antonio Rodríguez says:

    Now, the answer to that question *was* predictable :-) .

    @skSdnW: my guess is you are right: in Windows, permissions are usually checked at the handle opening or the operation start. It’s a good balance between security and performance.

  4. Cesar says:

    Just be happy that Win32 doesn’t have (as far as I know) seekdir()…

    1. Zan Lynx' says:

      I was under the impression that most modern Unix filesystems cheat seekdir() anyway, often just returning to the beginning. Since your code has to assume that a file may show up more than one time anyway…

  5. Simon Clarkstone says:

    Another way of looking at it AIUI: there is no point guaranteeing that directory enumeration is atomic unless your API provides other, greater, atomic facilities. Even if there were a guarantee that enumerating directory contents was atomic, files could still be added/deleted before you used the results. You could use an atomicity guarantee when e.g. checking that XML files and JSON files come in matched pairs, but if something is changing the directory under you then the property you are checking will be changing between true and false continually anyway.

    (And if you want to check whether you have had a consistent view while enumerating, Windows provides mechanisms monitor a directory to see if it has changed during some period of time.)

    1. smf says:

      That is a lot of overhead in the user code setting a monitor, changing all file access to use shadow copy would be interesting (but probably impractical).

  6. AsmGuru62 says:

    Is it safe to use DeleteFile() for an enumerated file within enumeration loop?
    Or maybe first, the list of enumerated files is to be created and then,
    after FindClose() was called – call DeleteFile() for each file name?

    1. Joshua says:

      I can’t say for me but it’s worked for me for ages and ages. I think MS would find the backwards compatibility rather bad to try to change it.

    2. JoeWoodbury says:

      My experience is that you can delete a file. But don’t rename one else funny things may happen.

  7. Jan Ringoš says:

    Can Kernel Transaction Manager be somehow of help here?

  8. Killer{R} says:

    I guess one who worried about consistency of dir enum may execute enumeration twice and use intersection of their results (taking into account file IDs too). I think this will give set of files that are known at least once coesisted all together. But still no guarantee that things will not change until moment you will use this information for whatever else purpose.

  9. “What you can say is that files which were neither created nor deleted during the enumeration will be enumerated”? Is this 100% true. When the iteration is “batching” how does it keep track of the current position? This could be critical. For example

    Directory contains 30 files.

    1) Get first 10 files
    2) Get second 10 files
    3 ) Get final 10 files

    Now if a file in the first group of 10 is deleted, that could “move” all of the files down one position. Thus file #11 would move from the second group (which has not been processed) into the first group (which has alreadyhbeen processed)….and thus “missed”….

    1. Anon says:

      It would only be missed if the batching worked by numerical indices, which it almost certainly does not. That is just not workable for a number of reasons. Instead the system probably maintains a stable pointer to the last entry seen so that the next batch can start after that. By stable pointer I mean something that doesn’t change when other entries are added or removed, such as a unique ID, or some other means known internally by the filesystem.

  10. DWalker says:

    Once an enumeration returns a file object, you don’t know that the file is still there when you try to use it. Enumerations are helpful, of course, but they are not a guarantee that the object will be there forever.

    Which makes me wonder why anyone would worry about whether the results of the enumeration would contain a file that appeared or disappeared DURING the enum.

Comments are closed.

Skip to main content