Why is the System.DateCreated property off by a few seconds?


If you ask for the System.Date­Created property of a shell item, the timestamp that comes back is up to two seconds different from the file's actual timestamp. (Similarly for System.Date­Modified and System.Date­Accessed.) Why is that?

This is an artifact of a decision taken in 1993.

In general, shell namespace providers cache information in the ID list at the time the ID list is created so that querying basic properties from an item can be done without accessing the underlying medium.

In 1993, saving 4KB of memory had a measurable impact on system performance. Therefore, bytes were scrimped and saved, and one place where four whole bytes were squeezed out was in the encoding of file timestamps in ID lists. Instead of using the 8-byte FILE­TIME structure, the shell used the 4-byte DOS date-time format. Since the shell created thousands of ID lists, a four-byte savings multiplied over thousands of items comes out to several kilobytes of data.

But one of the limitations of the DOS date-time format is that it records time in two-second increments, so any timestamp recorded in DOS date-time format can be up to two seconds away from its actual value. (The value is always truncated rather than rounded in order to avoid problems with timestamps from the future.) Since Windows 95 used FAT as its native file system, and FAT uses the DOS date-time format, this rounding never created any problems in practice, since all the file timestamps were already pre-truncated to 2-second intervals.

Of course, Windows NT uses NTFS as the native file system, and NTFS records file times to 100-nanosecond precision. (Though the accuracy is significantly less.) But too late. The ID list format has already been decided, and since ID lists can be saved to a file and transported to another computer (e.g. in the form of a shortcut file), the binary format cannot be tampered with. Hooray for compatibility.

Bonus chatter: In theory, the ID list format could be extended in a backward-compatible way, so that every ID list contained two timestamps, a compatible version (2-second precision) and a new version (100-nanosecond precision). So far, there has not been significant demand for more accurate timestamps inside of ID lists.

Comments (38)
  1. Erik F says:

    You'll have revamp the date structure in a few years anyways when 2107 rolls around. By that time, people will probably be complaining about the headaches caused by porting their Win256 apps to Win512 and wishing for the good old days of Win128 programming!

  2. Andre says:

    >wishing for the good old days of Win128 programming!

    Win64 ought to be enough for anybody.

    No, seriously.

  3. skSdnW says:

    "the binary format cannot be tampered with" There are already multiple versions of the ID list format used by the default FS IShellFolder. IIRC there are even some undocumented ID list functions that stuff hidden data into the ID list.

  4. 12BitSlab says:

    @ Andre, read "Inside the AS/400" by Frank Soltis.  You will understand why 128bit is very important.

    @ Raymond, thanks for this.  I enjoy reading about how we got from Point A to Point B.

  5. Medinoc says:

    Is there any documentation on the Shell's ID format for files? I've always wondered ever since I first dumped a PIDL...

  6. Ben says:

    In practice of course the precision, as Mr Chen hints, is illusory. Most computers synchronise clocks only weekly, and most computer clocks can easily drift by two seconds or more within a week.

    There is no such thing as "equality" for time or any other Real quantity. There is only "near enough".

  7. voo says:

    @12BitSlab haven't read that book, but that seems incredibly farfetched. There's just no reason even accounting for extra security for ASLR to need more than 2^64 bytes of addressable memory space in the foreseeable future.

    Now having larger arithmetic units is something else, but there we're already far over 128bit anyhow and nobody uses that to address the bitness of systems.

    What's the use case you see for 128 bit address space?

  8. Ben says:

    @voo, More's law suggests an exponential expansion of computational power, whether measured by instructions per second or storage capacity - specifically a doubling every 18 months.

    Since (if) that's true (and I think it is) one bit will have to be added on average every 18 months - or eight bits per twelve years. That being the supposition, the journey from 64 bits to 128 might take at most 100 years or so (and much less, maybe a quarter, for high-performance systems) but we will eventually arrive.

    And even then, some systems will still be running COBOL.

  9. Deanna says:

    I've clearly been reading this blog too long. As soon as you said "two seconds" I knew what the issue is!

  10. 12BitSlab says:

    @ voo,

    The AS/400 was the successor of the S/38.  It uses a concept of single level store.  The OS views the world as a flat 128 bit memory space.  It doesn't even know that disk drives are attached (except the WRKDSKSTS command).  The hardware takes care of paging stuff to/from memory.  A consequence of this is that there is no difference between a page fault for code and a page fault for data.  They are, in fact, one in the same.  There are no more records, files, etc. from an MI view (MI is roughly the equivalent of Assembler on a 400).  There are only spaces.  Cleanest architecture I have ever worked with in my career.

    When the S/38 first shipped in 1981, it was at least 4-5 decades ahead of everything else -- from an architectural view.

  11. voo says:

    @12BitSlab Interesting concept, but even then: The total space of the www in 2013 according to Wikipedia was 4*10^21 byte so around 2^70 byte.

    And just like we can access files with 64 bit offsets on 32 bit machines, the sane skulls work here too, although I can certainly see the elegance in the basic approach. But even then I wager we should be fine for another decade or three ;)

  12. skSdnW says:

    In WinDbg with symbols for shell32 type SHELL32!IL*Hidden* and press tab =) and this is in addition to the various versions of the normal FS PIDLs that I believe exists. Too bad MS cannot even document the layout of the PIDL created by SHSimpleIDListFromPath because these are actually passed across component boundaries, including 3rd-party components...

  13. Some people probably wouldn't be happy unless Windows's precision was Planck time.

  14. Boris says:

    This post is (un?)surprisingly free of reactive snarky commments.

  15. RRR says:

    @Deanna: I thought I was having a deja-vu.

  16. Rob G says:

    @12BitSlab

    There were excellent features in the AS/400 architecture. Sadly, almost all the programs I encountered were written in RPG. I need mind bleach now just to forget about it.

  17. DaveL says:

    >So far, there has not been significant demand for more accurate timestamps inside of ID lists.

    How much demand do you need, and how do we demand it? There's no easy way to get in touch with Windows development.

    This design decision also gives rise to the annoyance that Explorer details view doesn't show timestamps before 1/1/1970 - and yes, such timestamps are used by people doing photo archival for example. However, MS don't seem to think that anyone has any such files based on the response to an outstanding bug (still) in the MFC framework that prevents an application loading a file with a timestamp earlier than 1/1/1970.

  18. Boris says:

    DaveL: a photo _file_ created, modified or accessed earlier than 1/1/1970?

    Surely that's a total misuse of the file property in question; you'd need something more custom than that.

  19. DaveL says:

    @Boris. If the photo was shot in 1960, surely it's perfectly valid for someone to expect its created timestamp to say 1960? There are metadata properties for photo (and other media file types) timestamps, but if you investigate you'll find that most every mainstream application currently does it differently, so the file system timestamp is something that works (and it's what Explorer shows). Besides, if the file system supports the timestamp, the OS UI should show it correctly IMO). :)

  20. Boris says:

    PS. I just rewrote "Date taken" an iPhone photo to 1954. I also customized the Explorer folder view to show this property in one of the columns. If there is a problem, it hasn't been stated clearly.

  21. Boris says:

    PPS. I posted my second comment before I saw yours.

    It's not valid because Explorer deals in files, and users understand the concept of a file. Explorer is obviously doing it correctly because it's using a custom property. So if there is a problem, it's in the applications that assume Date taken == File created. They should learn to work with special properties.

  22. DaveL says:

    @Boris. As to getting applications to work (consistently), yes, I agree they should, but if you investigate you'll find they agree to disagree about the best way to do media metadata :)

  23. 12BitSlab says:

    @ Rob G

    I agree with you re: RPG III.  That stuff will rot your brain.

  24. Boris says:

    @DaveL: I switched my Photos folder to Details, right-clicked on column headers and checked "Date taken" from the right-click menu which appeared. The column appeared and now I can sort by it or whatever.

    Do you have specific examples of which applications are causing problems? I've never really tried scanning old photos, editing their properties and exchanging them between say, iOS, Mac OS X and Windows 7, but the date taken is part of EXIF metadata and the major vendors need to support reading and editing of such metadata. I can't be sure, of course, but I get the feeling you're not talking about current, standardized applications from major vendors (Apple, Microsoft, etc.), but perhaps something freeware or years out of date.

  25. Karellen says:

    @Anon: According to Wikipedia, DecTape[0] supports file creation timestamps[1], so it's possible that someone somewhere has a "hello world" program in PDP-10 assembly language created in the mid-late '60s, that they'd like to transfer/restore to a more modern system with full fidelity.

    How they achieve such a transfer is, of course, an exercise for the reader :-)

    [0] en.wikipedia.org/.../DECtape

    [1] en.wikipedia.org/.../Comparison_of_file_systems

  26. Karellen says:

    @Anon: Ohmygod. I just checked the "supporting operating systems" part of that Wikipedia comparison page, and just learned about AncientFS[0]

    From the DecTape comments: "The tap(1) command in ancient Unix (Editions First through Third) was used to save and restore selected portions of the file system hierarchy on DECtape. Even though the on-tape format was the same across these editions, you need to specify a Unix edition because the epoch changed. (The epoch was not 00:00 GMT Jan 1 1970 until Fourth Edition Unix.)" That epoch fact is a neat bit of Unix trivia I didn't know before!

    So, using Linux, one could legitimately conceivably try to copy a pre-1970 file from a Ver 1 Unix tape image, to an NTFS partition, while preserving the file-creation timestamp. If the filesystem supports it, I don't think it's unreasonable to expect the OS to display it!

    [0] osxbook.com/.../ancientfs

  27. dave says:

    > There's just no reason even accounting for extra security for ASLR to need more than 2^64 bytes of addressable memory space in the foreseeable future.

    That assumes conventional addressing and memory structures.

    In the past, I have worked on the design of a system that needed 128-bit virtual addresses.

    One reason you might need it, and I think this is the AS/400 case too, is that objects have unique addresses. Once an object is deleted, no future object will ever reuse that same virtual address.

    (This does not apply to Win128 of course, it's the general case I am talking about)

  28. dave says:

    >Even OUTSIDE the Windows world, it is statistically a near-impossibility to have any file created prior to 1970, as only three filesystems existed then,

    Maybe I'm lacking context for your remark, but I've stored files on four of those three file systems; here named by the OS:

    George 3

    Eldon 2

    TOPS 10

    E4 (CTL Modular One OS)

  29. Quadko says:

    >What's the use case you see for 128 bit address space?

    High resolution 3D printing. Just as program code still fits in small spaces but data - images and videos especially - require more memory (and more addressable memory), the higher the resolution of 3d printers, the more memory we'll need. I haven't done the math, but atomic scale printing of houses is just "a few" technical issues away from being a reality, I (optimistically) expect.

  30. DaveL says:

    @Boris. "Date Taken" is a metadata property (System.Photo.DateTaken), and you're right, some views of Explorer do show that metadata property correctly, but the (widely used) details doesn't do that, and its view does not show file system dates prior to 1/1/1970. It's perfectly valid for *any* file on a Windows system to have any of its 3 timestamp values be a date prior to 1/1/1970.

  31. voo says:

    @Quadko: You're either completely overestimating the size of whatever image or video (3d, 5d, no matter) or underestimating how enormous 2^64 a number is. To give an example: You can store 1/64th of the total WWW in that memory space if wikipedia is to be believed (or the total WWW around 2010).

    Now the reasons that 12bitslab and dave brought up are more realistic - I mean it's still not going to be a problem for the next few decades, but I could see us running into such limitations for large clusters at the end of this century.

  32. Anon says:

    @DaveL

    It isn't valid for any natively-supported Windows filesystem to display a *FILESYSTEM* date prior to 1/1/1970, because it is impossible for that file to have existed on the filesystem prior to 1970. There was no FAT until 1977, and FAT is the oldest natively-supported Windows filesystem.

    *NON*-filesystem dates (such as "When the picture was taken") have no relationship with filesystem dates.

    Even OUTSIDE the Windows world, it is statistically a near-impossibility to have any file created prior to 1970, as only three filesystems existed then, none of which are actually supported today in any meaningful way.

    (To be more pedantic, and someone else will have to verify, I believe Creation Time wasn't supported until the 90s, so it would be technically impossible to have a file with a Creation Date prior to 199X on a FAT* or NTFS filesystem.)

  33. Anonymous Coward says:

    @voo: Memory-mapping the entire Internet.

    (And if you say that's impossible, or a silly idea... consider that people probably said that about memory-mapped files back when they only had 16 address bits)

  34. Anonymous Coward says:

    @Quadko

    You're talking about what more than 2^64 bytes of *memory* could be used for, not what more than 2^64 bytes of *address space* could be used for.

  35. Anon says:

    @dave

    I didn't know the CTL had a file system. It isn't generally listed.

    Also, did it support file timestamps? (Context is that the file system must support a creation timestamp.)

    @Karellen

    When were Creation timestamps implemented in DECTape?  I thought they were a feature of DECTape II.

    v1 Unix sprang into existence in 1969; While it is conceivable that there are files stored from that year, I can't find evidence that it was in 'use' until the 70s. I also don't see any documentation of the v1 FS capabilities.

  36. Axel Rietschin says:

    The original "decision" was actually taken in 1973 by Gary Kildall, along with a few other things, like 8.3 :-)

  37. Gabe says:

    Axel: CP/M didn't even have file sizes down to the byte. When it had timestamps, they were down to the minute. About the only thing the MS-DOS FAT filesystem format inherited from CP/M is the filename conventions.

  38. Karellen says:

    @Anon - I was just going off the "Metadata" table on Wikipedia's "Comparison of File Systems" page (which I linked to) which has an unqualified "Yes" in the DECtape/Creation timestamps box. I'd be happy[0] to be corrected if you (or anyone else) has a citation or personal recollection which asserts otherwise.

    [0] But also disappointed, because then my "moving a pre-1970 file from a DECtape image to an NTFS partition" thought experiment would no longer be viable.

Comments are closed.

Skip to main content