Psychic debugging: When I copy a file to the clipboard and then paste it, I get an old version of the file


A customer reported the following strange problem:

I tried to copy some text files from my computer to another computer on the network. After the copy completes, I looked at the network directory and found that while it did contain files with the same names as the ones I copied, they have completely wrong timestamps. Curious, I opened up the files and noticed that they don't even match the files I copied! Instead, they have yesterday's version of the files, not incorporating the changes that I made today. I still have both the source and destination folders open on my screen and can confirm that the files I copied really are the ones that I modified and not files from some backup directory.

I tried copying it again but still an outdated version of the file gets copied. Curiously, the problem does not occur if I use drag/drop to copy the files. It happens only if I use Ctrl+C and Ctrl+V. Any ideas?

This was indeed quite puzzling. One possibility was that the customer was mistakenly copying out of a Previous Versions folder. Before we could develop some additional theories, the customer provided additional information.

I've narrowed down the problem. I've found that this has something to do with a clipboard tool I've installed. Without the tool running, everything is fine. How is it that with the tool running, Explorer is copying files through some sort of time machine? Those old versions of the files no longer exist on my computer; where is Explorer getting them from?

Other people started investigation additional avenues, taking I/O trace logs, that sort of thing. Curiously, the I/O trace shows that while Explorer opened both the source and destination files and issued plenty of WriteFile calls to the destination, it never issued a ReadFile request against the source. An investigation of Previous Versions shows that there are no previous versions of the file recorded in the file system. It's as if the contents were being created from nothing.

While the others were off doing their investigations, my head shuddered and I was sent into a trance state. A hollow voice emanated from my throat as my psychic powers manifested themselves. Shortly thereafter, my eyes closed and I snapped back to reality, at which point I frantically typed up the following note while I still remembered what had happened:

My psychic powers tell me that this clipboard program is "virtualizing" the clipboard contents (replacing whatever is on the clipboard with its own data) and then trying (and failing) to regurgitate the original contents when the Paste operation asks for the file on the clipboard.

A closer investigation of the clipboard enhancement utility showed that one of its features was the ability to record old clipboard contents and replay them (similar to the Microsoft Office Clipboard). Hidden inside the I/O operations was a query for the last-access time.

And that's when things fell into place.

Starting in Windows Vista, last access time is no longer updated by default. The program apparently saw that the file was never accessed and assumed that that meant that it also had never been modified, so it regenerated the file contents from its internal cache. (The quick fix for the program would be to switch to checking the last modified time instead of the last access time.)

Upon learning that my psychic powers once again were correct, I realized that my prize for being correct was actually a penalty: Now even more people will ask me to help debug their mysterious problems.

Comments (30)
  1. R. Bemrose says:

    I lost my watch.  Can you use your psychic powers to help me find it? (I am, of course, kidding.)

    I do find it strange that their app was checking the last access time.  I deal more with UNIX/Linux stuff, but the only one of the three Linux file timestamps that's genuinely useful is last modified.  "Creation time" is actually the least useful of the 3, as in Linux it's actually the last time any of the file's metadata (owner, permissions, etc…) were changed.

  2. ERock says:

    "Upon learning that my psychic powers once again were correct, I realized that my prize for being correct was actually a penalty: Now even more people will ask me to help debug their mysterious problems. "

    That's where the phrase "It's a gift, it's a curse" comes from.

  3. davep says:

    "While the others were off doing their investigations, my head shuddered and I was sent into a trance state. A hollow voice emanated from my throat as my psychic powers manifested themselves."

    Must be awesome to behold! You should film it! (I imagine a noise resembling Tuvan throat singing.)

    Of course, your real mistake is using such powers as source materials for this "crappy" blog rather than something much more profitable ($$$)!!

  4. Falcon says:

    You really have to wonder why they were using the last access time. On systems where that timestamp is updated, this would mean that the contents might be unnecessarily read from the source file instead of the cache if it's been accessed but not modified. Yes, the application might also check the last modified time in that case, but why not just do that in the first place?

    Every so often, I see a needlessly stupid design and just scream "WHY?????"

  5. Paul says:

    Once the customer determined that this problem only occurs when a 3rd party tool is installed, why is this still your problem to fix ?

    Shouldn't the customer be following up with the tool vendor first ?

    [It worked on Windows XP. It stopped working on Windows Vista. This is clearly a bug in Windows Vista. -Raymond]
  6. Steve [MSFT] says:

    @Paul –  if you have a problem with 3rd party software running on Microsoft software we're often faced with the "prove it's not you doing this" argument. So, in the interest of providing customers with a quality support experience, we often have to identify the root cause to show it's not us before we can begin to take a hard line on requiring the customer to engage their vendor. A side effect of this is that we often have to do enough investigation to not only prove it's not us, but to also provide a general idea to the vendor where their problem resides.

    Pre-recorded response for Nit Pickers: I will probably ignore any comments regarding our decision to staff our first line of support out of low cost labor markets, your individual support experience that counters my generalization above, your suggestions as to how we should modify our internal processess to accomodate your specific support needs, the cost of opening a support request with Microsoft, etc.

  7. Alex Grigoriev says:

    Every time a program is trying to be unnecessarily "smart" it end up stupid. Happens to all vendors. Usually it's a feature "somebody got a bonus for". Today, it's "smart" way to show selected file information in Windows 7. Select many enough files (>15), it won't show even their total size; you have to click "Show more details". Does that even if you select the files one by one. But I guess, it's done because it has to support offline files.

  8. Alex Grigoriev says:

    About "offline files", it was a sad joke, if you didn't get it.

  9. mikeb says:

    You mean your penalty didn't include writing a compatibility shim on Vista for the program that gives it the file's modified time timestamp when it asks for the last access time?

    More babbling:  If last access time tracking is disabled (for perf reasons), shouldn't it at least still be updated to the last modified time when a file is modified? You're performing  a write to a file attribute anyway, so the performance impact shouldn't be large, and it would prevent this type of confusion for applications that look at last access time.

  10. Joshua says:

    mikeb: actually no. Opening a file for write-only does not change the access time. Unfortunately your shim just made the problem worse. :(

  11. Gabe says:

    It's hard to imagine why an action that changes the modification time doesn't change the access time.

  12. Brian says:

    Because they mean two different things.  One is the last time the file was read, the other is the last time the file was written.

  13. Kyle says:

    @Brian

    But the name "access" implies any sort of access, not simply read access.

  14. Brian says:

    What a variable does and what it implies are not always the same.

  15. Gabe says:

    Brian: If the last access time was the last time the file was read, why is it called something ambiguous instead of LastReadTime? According to MSDN (msdn.microsoft.com/…/ms724320.aspx), "The last access time includes the last time the file or directory was written to, read from, or, in the case of executable files, run."

    Based on the documentation, I would have no reason to expect that after writing to a file its last access time would not be changed.

  16. Leo Davidson says:

    Every so often, I see a needlessly stupid design and just scream "WHY?????"

    You mean the last access timestamps in NTFS? :)

  17. Chewy Chua says:

    So, can anyone confirm that the solution to this is:

    1. Run fsutil behavior set disablelastaccess 0

    or

    2 clearing Clipboard before performing a copy and paste?

    or

    both?

  18. michen says:

    @Chewy – neither :)

    Simply avoid using buggy "clipboard management" tools. One would not get any problem with built-in clipboard.

  19. Falcon says:

    @Leo Davidson:

    That sentence was inspired by the fact that the application used the last access timestamp when last modified clearly seemed more appropriate, but it was more of a general comment.

  20. My psychic debugging powers tell me Brian is wrong. If access time was only updated when file is read, that clipboard app would fail even on XP. The only reason the problem with this clipboard app does not occur on XP is that access time *is* updated (on XP) when the file is modified.

    But on Vista it seems like access time is not updated even when file is modified. I think this is wrong, and agree with mikeb – it is good optimization to not update access time on reads, but it would not hurt at all to update it on writes, and would do much better for compatibility.

    In other words, Vista did equivalent of older 'noatime' Linux option, while better 'relatime' would be preferable (this option updates atime on *writes* only if current atime is older).

  21. asdbsd says:

    This needs tag "raymond_is_being_awesome".

  22. Neil says:

    I always found the last access time to be pretty useless, for example just looking at the properties in Explorer seems to cause a read of the file (presumably to look for summary data) so I have to remember to use dir instead.

  23. JamesNT says:

    Raymond,

    I disagree that this situation is clearly a bug in Vista.  I would think this is clearly a failure of the third party vendor to update their software or of the customer to ensure they are using a compatible version of the program with the version of the OS they have.

    JamesNT

  24. Falcon says:

    @JamesNT:

    I think Raymond is saying that that is how the customer interpreted the situation. He's written about this phenomenon in the past – it's one of the reasons for all those backwards compatibility hacks.

    [Exactly. Especially since the customer didn't have a support contract with the original vendor. -Raymond]

    Regarding your second point – the vendor didn't fail to update their software; they failed to write it correctly in the first place!

  25. Brian says:

    @Gabe: wow, you're right (on windows).  I was thinking of the posix stat() function which states, "The field st_atime is changed by file accesses, e.g. by execve(2), mknod(2), pipe(2), utime(2) and read(2) (of more than zero bytes).  The field st_mtime is changed by file modifications, e.g. by mknod(2), truncate(2), utime(2) and write(2) (of more than zero bytes)."

  26. dogfood says:

    Was the clipboard enhancement utility a ms product?

  27. Daniel says:

    "my prize for being correct was actually a penalty: Now even more people will ask me to help"

    Well, that's a real issue for talented people. My advice:

    a) Become untalented yourself or at least pretend to be

    b) Alternatively, accept that there are untalented people who need your help

    And please, talented people, try to help friendly. This is very much asked, I know, but letting them feel how untalented they are just increases the overall amount of bad feelings – nothing won. Also note that maybe their untalentedness only spans a certain area of expertise.

  28. Sigh... says:

    You go through all that trouble telling us about the issue but don't tell us what the extra "clipboard tool", the root cause of the problem, is?

  29. bryan says:

    ok well, while I can agree that the program in question had a bad design it just screams WTF to me that last access time is disabled by default, especially as it was enabled before. Backwards compatibility problems coming up, also if I am looking at a bunch of files that I read before but didn't change anything in – maybe I want to see the last one I accessed?

    This sounds to me like Someone found there were performance problems with NTFS and disabled a feature as the solution. Premature feature disabling is the root of all my yelling.

  30. Cesar says:

    @bryan: You will notice Linux distributions are also disabling (or at least reducing) atime updates by default, where before they were enabled. The reason is the same as in Windows: performance.

    In particular, an atime update changes a read (potentially from the cache) into a read plus a *write*. Not only are writes slower, you are also wasting disk bandwidth, and on some kinds of media can even cause premature wear (flash-based media has a limited number of writes, which is why it needs wear leveling).

Comments are closed.