Why does the timestamp of a file increase by up to 2 seconds when I copy it to a USB thumb drive?


We saw some time ago that the FAT file system records timestamps in local time to only two-second resolution. This means that copying a file to a FAT-formatted device (typically a floppy drive or a USB thumb drive) can increase the timestamp by up two seconds. And even after the file is copied, the timestamp is not stable. The timestamp changes depending on the time zone employed by the computer that accesses the drive. In particular, if you are in a part of the world which changes clocks during the summer, then the timestamp on the file moves by an hour every spring and then moves in the opposite direction every autumn. (Because you change time zones twice a year.)

Okay, but why does the timestamp always increase to the nearest two-second interval? Why not round to the nearest two-second interval? That way, the timestamp change is at most one second.

Because rounding to the nearest interval means that the file might go backward in time, and that creates its own problems. (Causality can be such a drag.)

For example, suppose you regularly back up files from your NTFS-formatted C: drive to your USB thumb drive mounted as drive F: by typing

xcopy /D C:\Files\* F:\Files\*

If the timestamps rounded to the nearest two-second interval, then half the files on average will have a timestamp on the USB thumb drive older than the files on the C: drive. This means that if you perform the command a second time, approximately half of the files will be copied again. To the user, it looks like the xcopy command never finishes the job, because each time you tell it "Perform an incremental backup" it always finds something to copy. It never says, "All files up to date, you can go home now."

To avoid this infinite loop, the convention is always to round up, so that the copy of a file is never older than the original.

Comments (24)
  1. Adrian says:

    I use RoboCopy with the FFT switch to cope with this.

  2. Xv8 says:

    To say time handling in the computer world is a mess is like saying Hitler was a bit naughty.

  3. Adam Rosenfield says:

    I had a very similar problem with an rsync script of mine.  I use rsync to synchronize files between an HFS+ volume and a FAT volume (an SD card), and rsync by default skips copying files if the file size and timestamp match between the source and destination.

    Because FAT has a 2-second timestamp resolution, I had to add the –modify-window=1 option to tell rsync that "if timestamps differ by up to 1 second, consider them to be equal" in order for half of all files to not be copied again each time the script ran.

    I honestly don't remember why –modify-window=1 is sufficient, when you'd think it should be –modify-window=2, but the documentation recommends using –modify-window=1 when copying to or from FAT volumes, and that's worked just fine for me.

  4. Xv8 says:

    @Adam Rosenfield: +/-1s is enough if your filesystem driver rounds to nearest. 2s is needed if your driver rounds up or down.

  5. I feel obliged to point out that for recurring backups, /M is a better switch to use than /D.

  6. Piotr says:

    Can someone explain the reasoning behind the 2 second accuracy? Is it just about saving one bit or am I missing something?

    [Look at the DOS time format. -Raymond]
  7. Piotr: DOS times were packed 16-bit values; the seconds value only has bits 0 to 4 to encode the seconds.  You can't count up to 59 with only 5 bits unless you half the resolution.

  8. Myria says:

    The last-accessed time on FAT has a resolution of 86400 seconds, so it's more like the last-accessed date.  I don't know what it is for exFAT, though.

  9. A regular viewer says:

    This is known, and many a batch file has worked with/around it. A step back. On the larger picture, does MS on principle work on putting up a method of function straightaway, with unquestionably (trolls, shut up. I know what I am writing on Raymond's blog) good intentions into its products? Or is there a back and forth amongst the larger user base within MS and their partners, that comes up with these functionalities? If the latter, on average, what is the time frame between Option A —–> Usage —–> Feedback —–> Option B —–> Usage —–> Feedback —–> Revert —–> Option A?

    No specific purpose. Just curious. Thank you.

  10. Scott says:

    Stating the obvious for people as confused as I was:  The rounding can result in what appears to be a two second jump, but is really less than two seconds.  If the timestamp is "0 min, 02 sec, 001 ms", since it can't go back in time when the FAT timestamp is created, it needs to update to "0 min, 04 sec".

    It's a jump of less than two seconds technically, but since many tools only show the timestamps in seconds, it'll appear to the user to be a jump of two seconds.

  11. poizan42 says:

    If only there was some way to attach additional metadata to files in FAT without breaking backwards compatibility. Then you could use the same trick to support long filenames as well. Oh well, we are probably going to be stuck with 8.3 filenames forever as well.

  12. Henry Skoglund says:

    Also (perhaps slightly offtopic) is there any way to make Windows behave like Ubuntu or OSX when copying directories?

    I'm thinking of when I mount an NTFS-formatted USB-harddrive and makes a copy of say C:Program Files, if I do it in Windows 8.1 (or earlier versions) the subdirectories will be timestamped with the current date/time, but if I do the same copy operation in Ubuntu or OSX (using the NTFS Paragon drivers) the date and time of the subdirectories will be set to the same as the original.

    (Obviously Windows can do this because normal files are set the same timestamp as the originals, it's just the copied subdirectories themselves that exhibit this syndrome.)

  13. IdahoJacket says:

    @Henry Skoglund

    Robocopy /DCOPY:T

  14. smf says:

    @poizan42

    "If only there was some way to attach additional metadata to files in FAT without breaking backwards compatibility. Then you could use the same trick to support long filenames as well. Oh well, we are probably going to be stuck with 8.3 filenames forever as well."

    There are issues with that, but your sarcasm puts me off discussing them.

  15. Entegy says:

    @poizan42 Aren't 8.3 file names off by default in modern versions of Windows now?

  16. @Entegy: No, 8.3 filenames are still generated by default in Windows 8.1.  You can specify in FORMAT to disable this feature though, or do so after the fact using FSUTIL.

  17. IanBoyd says:

    DOS File Time (16-bits)

    – bits 0–4: Second divided by 2

    – bits 5–10: Minute (0–59)

    – bits 11–15: Hour (0–23 on a 24-hour clock)

    DOS File Date format (16-bits):

    – bits 0–4: Day of the month (1–31)

    – bits 5–8: Month (1 = January, 2 = February, etc.)

    – bits 9-15: Year offset from 1980 (add 1980 to get actual year)

  18. dave says:

    >If only there was some way to attach additional metadata to files in FAT

    Sure, but why bother enhancing an obsolete file system?

    (Particularly since FAT is most useful today as a least-common-denominator interchange format, so *all* implementations will need to change before the enhancement is really worthwhile)

    Also, look at the mess that 'a file has two names' got us into:  'del *.foo' deleting whatever.foobar, for example.

  19. @IanBoyd: Welp, we're screwed in the year 2107 then.

  20. Ancient_Hacker says:

    But Microsoft has such a fine history of making time go backwards.

    If you look into the tick interrupt code in DOS, it checks to see if midnight has just gone by, and if so, it "fixes up" the accumulated time error by subtracting 50 or so milliseconds from the time.

    This tripped up some code of mine that made the rash assumption that time always went forwards.  It got stuck trying to control tasks that had started in the future.  Never assume!

  21. Justa Typer says:

    @Ancient_Hacker – that's what I love about the PC world… opportunities abound!  If you figured out how to control tasks in the future, you'd be rich!

  22. ErikF says:

    @Ancient_Hacker: Your programs have to worry about time going backwards once a year anyways (for many regions anyways; this does not include NTP negative time offsets), so it's not just Microsoft that causes this :-)

  23. Stefan Kanthak says:

    "And even after the file is copied, the timestamp is not stable. The timestamp changes depending on the time zone employed by the computer that accesses the drive."

    Since FAT timestamps are in local time this is wrong.

    Only filesystems where timestamps are not in local time show this behaviour, notably NTFS.

  24. Gabe says:

    Stefan Kanthak: A FAT timestamp will always display as the same time in Explorer, but will be a different FILETIME value in different time zones. An NTFS timestamp will always be the same FILETIME, but will display differently in Explorer depending on your time zone offset.

    The stability that Raymond is referring to is that the FILETIME always stays the same, allowing xcopy to work as intended.

Comments are closed.