Why don’t all of my documents show up when I arrange my Documents library by Name?


A customer reported that when they opened their Documents library on Windows 7, some files were missing if they selected Arrange by: Name or Arrange by: Author or in fact any arrangement other than Arrange by: Folder. What's going on?

When you arrange the Documents library by anything other than Folder, the Documents library uses the content indexer to obtain results quickly, rather than kicking off a recursive disk search (ugh). (The Folder arrangement does not require a recursive search, so it can use the traditional Find­First­File/Find­Next­File loop to get the results.

A member of the search indexer team suggested that a common cause for items being missing from the index is if they have the FILE_ATTRIBUTE_TEMPORARY attribute set. The indexer ignores temporary files because the creator of the file said, "Don't worry about me, this is just a temporary file that I will be deleting as soon as I close the handle." No point indexing a file that is going to be deleted soon.

You can check whether the file has the Temporary attribute by viewing its Properties and going to the Details tab. If the file is marked Temporary, the Attributes will include the letter T. (Note that the attrib program does not report the Temporary flag.)

Sure enough, the customer found that the files which were mysteriously missing from the view were all marked as Temporary files. This allowed the customer to proceed to the next step of the investigation, which is figuring out why those files were being marked Temporary in the first place.

Comments (29)
  1. Adam Rosenfield says:

    There's also FILE_ATTRIBUTE_NOT_CONTENT_INDEXED, which means "not a temporary file, so please save me to disk, but also don't index me".

  2. DWalker says:

    I don't need my document contents indexed; I think it's a useless waste of time and resources.  I'll have to see how that affects things like this.

  3. da-bam says:

    I really don't get this: The Windows 7 search in Explorer decides it's more important to get an incomplete (= incorrect) result fast rather a complete one (= correct) slow. Why, oh why, this was already working perfectly in Windows XP. :(

    Who thought that optimizing the wheel to a triangle was a good idea?

  4. Timo Kinnunen says:

    Oh man, I would love to have a query term that would just brute-force every subfolder and file recursively and skip the index. As it is I have to use Eclipse IDE for that. While going over a couple hundred thousand files takes a minute or several at least there's a progress indicator that shows exactly how many files have been searched and how many are still left.

  5. Yuri says:

    Coming from Win9x and XP, the vista search feature is a welcome improvement. I don't mind the index, it's not perfect but works very well when you repeat the same search query again. The Arrange feature I could certainly do without.

  6. DWalker says:

    @James:   Be careful of that tool in a multi-user environment if security is an issue (that is, privacy of files).

  7. Nick says:

    I used to think that indexing file contents was a waste of resources, but then I realized two things: (1) the couple of times I'd spent an hour waiting for a file contents search far outweighed any effort the computer needed to do to index, and (2) making repetitive tasks faster is what computers are supposed to be for.  Whether or not the contents indexing is as useful as it might be (I wish it was better at indexing unknown file types that still contain text) might be up for debate, but I still think it's worth indexing at least user document libraries even if you only use it once a month or so.

    > Try VoidTools' Everything and I guarantee you'll be gob-smacked

    I'll second that.  The speed and abilities of Everything blew my mind.  If Explorer could search the filesystem in this manner it would be amazing.  A nice companion that searches file contents with boolean and regex searching is FileLocator Lite.  It's quite fast at searching our enormous legacy codebase (being on an SSD is a big help).

  8. alegr1 says:

    Oh, the search indexer, where should I start? For example, if I change the sort order by clicking on the header, WHY OH WHY does it have to go and repeat the search? I just changed from sort by name to sort by size, and it's already got all the sizes in the detailed view.

  9. Yuri says:

    > alegr1

    I don't think all directory information is updated in real time in explorer, changing a view settings forces a refresh maybe to guarantee it shows accurate up to date information.

  10. John says:

    My problem with indexers is lack of information.  If I search for something and no results are found, is it because no such file exists or because the file is not indexed?  From the indexer's point of view there is no distinction.  At least with a brute force approach I can be relatively confident about the result.

  11. SomeGuyOnTheInternet says:

    zomg I can get more than a RASH from ATTRIB?

  12. Brandon says:

    Sigh. I don't really understand this exhortation about indexers being evil.

    The indexer provides a function–giving snappy results–at a given cost–must rebuild index; index can get out of date; etc.

    A good majority of users probably appreciate the speed and don't notice or don't care about the costs. If your use case works better without making use of an indexer, then use a different file search tool?

  13. Gabe says:

    The best use of content indexing is to have your whole source tree indexed. I first discovered this when the OFS guys (part of Cairo) had a complete index of the Windows NT source tree. Even back in 1995 it was some GBs of code, yet any function you were looking for in any part of the tree was easily found in seconds.

    Want to know what's putting that cryptic message in the log? Just paste it into the search box and jump instantly to the line that generated it. Looking for comments with "TODO" in them? Just type TODO in the search box and find all the files with tasks left to do.

  14. James says:

    @Timo: Wow, I can't imagine anyone using Eclipse of all things to search for files, not to mention wasting *minutes* on searching a measly hundred thousand or so files! Try VoidTools' Everything and I guarantee you'll be gob-smacked and save precious time you can spend doing something useful. No content indexing, but there are other indexers that do that.

  15. I certainly see the use of the indexer for the common user.

    But why punish the experienced user by removing a really vital part of the shell (aka real file search). I am aware it's slower than indexed search. But now I have to install a 3rd party tool to find content in files? Sorry to sound like xpclient, but at least in XP you could configure the search to search everything.

    And then there's these weird key words you can add to modify the search result (or restrict files by certain properties) which don't really cater to the less computer savy crowd. They seem to be tailored to the experienced user. But they still do not provide the full spectrum of the previous search.

  16. AndyCadley says:

    I think the only issue I've ever had with the indexer is that lots of (mostly development) tools don't register their file extensions as being compatible with the plain text indexing format, so searches in those documents don't work unless you manually go and add the right types in.

    Of course if you don't like it, you can go into Folder Options and tell it not to use indexing and to always search file names and contents in non-indexed locations. At which point you get back XP behaviour and can enjoy the benefits of sitting around drinking tea whilst waiting for your results.

  17. @Georg_Rottensteiner: Temporary attribute is not something that can be set and unset normally, so what you experience is the result of a programming blunder. So: Don't worry, even if Windows did show those files, programmers who do blunders would still have found a way to irritate you, be it in XP or 7.

    This is brings old memories: The time when System attribute was something odd and a bug had caused some program files to be marked as system.

  18. xpclient says:

    Arrange By is really a strange beast and should be improved to behave consistently – it's a mess currently. When a Library is Arranged By, what it's really doing is using the content indexer and presenting the items as either a flat sorted view, grouped view or in stacked view (which was introduced in Vista). However, the unpredictability leads to the confusion.

    When the Documents library is arranged by author, tag or type, it stacks them (what Vista did by "Stack By"). When arranged by date modified, it groups them, no stacking. When arranged by name, it flattens the whole library and only sorts it by name (no grouping, no stacking). The presentation for "arrange by" for each property is unpredictable. Additionally, it also seems to unexpectedly change the view the user has set in Explorer when doing Arrange By. The same is true of other Libraries.

    Plus, the properties for Libraries for "Arrange By" seem to be restricted intentionally in contrast with Vista where sorting, grouping or stacking by any property was allowed.

    My point is that the user never knows or can predict how it's going to present his items – it's just some arbitrary and unpredictable, fancy view. Added to this chaos is more confusion when you consider the fact that in WinXP Explorer (pre-Windows Search days), "Sort By" was called "Arrange By" – so the XP crowd is likely to think Arrange By still means Sort By.

  19. Sorry for bumping but it is possible to solve the problem by having Windows PowerShell unset the temporary attribute. Here is how:

    <code>

    $a=(Get-Item '.filename.ext')

    $a.Attributes=$a.Attributes -bxor [Io.FileAttributes]::Temporary

    </code>

    But I have a hard time imagining which of you might need this, like, ever.

  20. 12BitSlab says:

    Personally, I am not a big fan of the way that Win7 does the file search.  I actually liked the XP search better.  So, I built my own file search that operates in a similar fashion to the way that XP does — sansthe dog with the wagging tail.

  21. Yuri says:

    Wonder where all the love for XP search comes from. I remember it was notoriously buggy and poorly multi-threaded.

  22. Tim says:

    Same place the love for all old features come from. People got used to its quirks and integrated it into their workflow.

    Even though indexed search completes immediately and XP search always took a couple minutes, this isn't enough to overcome the new feature resistance inertia for some people.

  23. 12BitSlab says:

    My need for file search doesn't often involve file contents.  It is far more focused on file name, location, date, etc.  Hence, that is why I liked the older search better.

  24. Klimax says:

    If somebody doesn't want to use indexing, then tell Explorer that. furthermore you can tell Windows what you want/don't want to index. What the hell is then problem?

    You have the control over the feature, use it. (So searching in explorer doesn't work much differently then in XP)

  25. chentiangemalc says:

    Funny people thinking XP search was better, it was terrible. How quickly we forget. And on a 1TB drive… see you next year. If you need slow but thorough searches plenty of free utilities do it great i.e. nirsoft SearchMyFiles. Win7 was such a vast improvement over XP in almost every way, including search.

  26. jader3rd says:

    So is it a bug if there's a file attributed as temporary, but has no file handle open to it?

  27. Daniel says:

    Well, I'm working with TotalCommander. So while I cannot search word or office files by content (which I don't need to in any case) I can search files by text content even if:

    – The file says it's temporary

    – The file says it should not be indexed

    – The registry doesn't contain a "This is a text file" key for the file extension

    – The file is so new that it hasn't been indexed yet

    Yes it's a bit slow if you need to check a whole directory tree. But when I'm searching over a directory tree it's much more important to find ALL files than to finish the search very quickly…

    @jader3rd: nope (Actually when creating a temporary file you almost always have moments where no file handle is open for the file). But generally it should be treated as bug if a temporary file "survives" its process in the regular case (not talking about app crashes here. I'm talking about "forgetting" to cleanup temporary files when closing the application)

  28. morlamweb says:

    Up until today when I read this post, I had never even seen the Arrange By option for the Documents library.  Yet another interesting feature of the shell that I never knew existed.  I do like some of the options available, particularly Arrange By: Author, as it's kind of a record of everybody who's ever sent me a document.  I've seen this information before when viewing file details, but this option really does give a different perspective on things.

    I tend to lump people who turn off the indexer for performance reasons into the same category as people who turn off the visual effects in XP: oh those misguided fools : )  (see Raymond's previous post on Visual Effects and how turning them off can degrade, not improve, system performance).  True, the fastest code is the code that doesn't run at all, but how much performance exactly are you saving by turning off the indexer, and how much time is lost by searches that take much more time due to un-indexed files?

  29. Tools says:

    If you want to search everything regardless of the index, you can:

    * Use dir (for file names and attributes) and/or findstr (for text file content) from the command prompt.

    * Use Windows 8, where there's a progress bar for search and you can use the Advanced Options in the Search ribbon (visible while you're in a search) to search the content of nonindexed files (if you're patient).

Comments are closed.

Skip to main content