Do not access the disk in your IContextMenu handler, no really, don’t do it


We saw some time ago that the number one cause of crashes in Explorer is malware.

It so happens that the number one cause of hangs in Explorer is disk access from context menu handlers (a special case of the more general principle, you can't open the file until the user tells you to open it).

That's why I was amused by Memet's claim that "would hit the disk" is not acceptable for me. The feedback I see from customers, either directly from large multinational corporations with 500ms ping times or indirectly from individual users who collectively click Send Report millions of times a day, is that "would hit the disk" ruins a lot of people's days. It may not be acceptable to you, but millions of other people would beg to disagree.

The Windows team tries very hard to identify unwanted disk accesses in Explorer and get rid of them. We don't get them all, but at least we try. But if the unwanted disk access is coming from a third-party add-on, there isn't much that can be done aside from saying, "Don't do that" and hoping the vendor listens.

Every so often, a vendor will come back and ask for advice on avoiding disk access in their context menu handler. There's a lot of information packed into that data object that contains information gathered from when the disk was accessed originally. You can just retrieve that cached data instead of going off and hitting the disk again to recalculate it.

I'm going to use a boring console application and the clipboard rather than building a full IContext­Menu, since the purpose here is to show how to get data from a data object without hitting the disk and not to delve into the details of IContext­Menu implementation.

#define UNICODE
#define _UNICODE
#include <windows.h>
#include <ole2.h>
#include <shlobj.h>
#include <propkey.h>
#include <tchar.h>

void ProcessDataObject(IDataObject *pdto)
{
 ... to be written ...
}

int __cdecl _tmain(int argc, PTSTR *argv)
{
 if (SUCCEEDED(OleInitialize(NULL))) {
  IDataObject *pdto;
  if (SUCCEEDED(OleGetClipboard(&pdto))) {
   ProcessDataObject(pdto);
   pdto->Release();
  }
  OleUninitialize();
 }
}

Okay, let's say that we want to check that all the items on the clipboard are files and not directories. The HDROP way of doing this would be to get the path to each of the items in the data object, then call Get­File­Attributes on each one to see if any of them has the FILE_ATTRIBUTE_DIRECTORY flag set. But this hits the disk, which makes baby context menu host sad. Fortunately, the IShell­Item­Array interface provides an easy way to check whether any or all the items in a data object have a particular attribute.

void ProcessDataObject(IDataObject *pdto)
{
 IShellItemArray *psia;
 HRESULT hr;
 hr = SHCreateShellItemArrayFromDataObject(pdto,
                                          IID_PPV_ARGS(&psia));
 if (SUCCEEDED(hr)) {
  SFGAOF sfgaoResult;
  hr = psia->GetAttributes(SIATTRIBFLAGS_OR, SFGAO_FOLDER,
                                                 &sfgaoResult);
  if (hr == S_OK) {
   _tprintf(TEXT("Contains a folder\n"));
  } else if (hr == S_FALSE) {
   _tprintf(TEXT("Contains no folders\n"));
  }
  psia->Release();
 }
}

In this case, we want to see if any item (SI­ATTRIB­FLAGS_OR) in the data object has the SFGAO_FOLDER attribute. The IShell­Item­Array::Get­Attributes method returns S_OK if all of the attributes you requested are present in the result. Since we asked for only one attribute, and since we asked for the result to be the logical or of the individual attributes, this means that it returns S_OK if any item is a folder.

Okay, fine, but what if the thing you want to know is not expressible as a SFGAO flag? Well, you can dig into each of the individual items. For example, suppose we want to see the size of each item.

#include <strsafe.h>

void ProcessDroppedObject(IDataObject *pdto)
{
 IShellItemArray *psia;
 HRESULT hr;
 hr = SHCreateShellItemArrayFromDataObject(pdto,
                                          IID_PPV_ARGS(&psia));
 if (SUCCEEDED(hr)) {
  IEnumShellItems *pesi;
  hr = psia->EnumItems(&pesi);
  if (SUCCEEDED(hr)) {
   IShellItem *psi;
   while (pesi->Next(1, &psi, NULL) == S_OK) {
    IShellItem2 *psi2;
    hr = psi->QueryInterface(IID_PPV_ARGS(&psi2));
    if (SUCCEEDED(hr)) {
     ULONGLONG ullSize;
     hr = psi2->GetUInt64(PKEY_Size, &ullSize);
     if (SUCCEEDED(hr)) {
      _tprintf(TEXT("Item size is %I64u\n"), ullSize);
     }
     psi2->Release();
    }
    psi->Release();
   }
  }
  psia->Release();
 }
}

I went for IEnum­Shell­Items here, even though a for loop with IShell­Item­Array::Get­Count and IShell­Item­Array::Get­Item­At would have worked, too.

File system items in data objects cache a bunch of useful pieces of information, such as the last-modified time, file creation time, last-access time, the file size, the file attributes, and the file name (both long and short). Of course, all of these properties are subject to file system support. the shell just takes what's in the WIN32_FIND_DATA; if the values are incorrect (for example, if last-access time tracking is disabled), then the shell is going to cache the incorrect value. But don't say, "Well, if the cache is no good, then I won't use it; I'll just go hit the disk", because if you hit the disk, the file system is going to give you the same incorrect value anyway!

If you just want to order the combo platter, you can ask for PKEY_Find­Data, and out will come a WIN32_FIND_DATA. This might be the easiest way to convert your old-style context menu that hits the disk into a new-style context menu that doesn't hit the disk: Take your calls to Get­File­Attributes and Find­First­File and convert them into calls into the property system, asking for PKEY_File­Attributes or PKEY_Find­Data.

Okay, that's the convenient modern way to get information that has been cached in the data object provided by the shell. What if you're an old-school programmer? Then you get to roll up your sleeves and get your hands dirty with the CFSTR_SHELL­ID­LIST clipboard format. (And if your target is Windows XP or earlier, you have to do it this way since the IShell­Item­Array interface was not introduced until Windows Vista.) In fact, the CFSTR_SHELL­ID­LIST clipboard format will get your hands so dirty, I'm writing a helper class to manage it.

First, go back and familiarize yourself with the CIDA structure.

// these should look familiar
#define HIDA_GetPIDLFolder(pida) (LPCITEMIDLIST)(((LPBYTE)pida)+(pida)->aoffset[0])
#define HIDA_GetPIDLItem(pida, i) (LPCITEMIDLIST)(((LPBYTE)pida)+(pida)->aoffset[i+1])

void ProcessDataObject(IDataObject *pdto)
{
 FORMATETC fmte = {
    (CLIPFORMAT)RegisterClipboardFormat(CFSTR_SHELLIDLIST),
    NULL, DVASPECT_CONTENT, -1, TYMED_HGLOBAL };
 STGMEDIUM stm = { 0 }; // defend against buggy data object
 HRESULT hr = pdto->GetData(&fmte, &stm);
 if (SUCCEEDED(hr) && stm.hGlobal != NULL) {
  LPIDA pida = (LPIDA)GlobalLock(stm.hGlobal);
  if (pida != NULL) { // defend against buggy data object
   IShellFolder *psfRoot;
   hr = SHBindToObject(NULL, HIDA_GetPIDLFolder(pida), NULL,
                       IID_PPV_ARGS(&psfRoot));
   if (SUCCEEDED(hr)) {
    for (UINT i = 0; i < pida->cidl; i++) {
     IShellFolder2 *psf2;
     PCUITEMID_CHILD pidl;
     hr = SHBindToFolderIDListParent(psfRoot,
                HIDA_GetPIDLItem(pida, i),
                IID_PPV_ARGS(&psf2), &pidl);
     if (SUCCEEDED(hr)) {
      VARIANT vt;
      if (SUCCEEDED(psf2->GetDetailsEx(pidl, &PKEY_Size, &vt))) {
       if (SUCCEEDED(VariantChangeType(&vt, &vt, 0, VT_UI8))) {
         _tprintf(TEXT("Item size is %I64u\n"), vt.ullVal);
       }
       VariantClear(&vt);
      }
      psf2->Release();
     }
    }
    psfRoot->Release();
   }
   GlobalUnlock(stm.hGlobal);
  }
  ReleaseStgMedium(&stm);
 }
}

I warned you it was going to be ugly.

First, we retrieve the CFSTR_SHELL­ID­LIST clipboard format from the data object. This format takes the form of an HGLOBAL, which needs to be Global­Lock'd like all HGLOBALs returned by IData­Object::Get­Data. You may notice two defensive measures here. First, there is a defense against data objects which return success when they actually failed. To detect this case, we zero out the STG­MEDIUM and make sure they returned something non-NULL in it. The second defensive measure is against data objects which put an invalid HGLOBAL in the STG­MEDIUM. One of the nice things about doing things the IShell­Item­Array way is that the shell default implementation of IShell­Item­Array has all these defensive measures built-in so you don't have to write them yourself.

Anyway, once we get the CIDA, we bind to the folder portion, walk through the items, and get the size of each item in order to print it. Same story, different words.

Exercise: Why did we need a separate defensive measure for data objects which returned success but left garbage in the STG­MEDIUM? Why doesn't the Global­Lock test cover that case, too?

Comments (22)
  1. alegr1 says:

    I wonder what thinking brought us the Explorer's new stupid behavior in Vista+, when it refuses to calculate and show total size of selected files, if more than 15 selected, which limit was arbitrarily raised (but not removed, regretfully) to 100 in Windows 8.

    "It's too much CPU time to sum those 64-bit numbers and sort those 64 bit FILETIMEs, that we already got in our file list". And don't feed me this "offline file" BS. If there are files offline, which Explorer knows already from WIN32_FIND_DATA::dwFileAttributes, THEN explorer can show partial results or change its behavior. But if the directory was read successfully, which happens in 99.999% of cases, Y U NO SHOW TOTAL SIZE?

    [I think you'll find that people are more likely to respond if you don't insult them. Interestingly, the email next to your comment in my inbox was a spam message titled "How to Deal With Difficult People." -Raymond]
  2. Mordachai says:

    Why doesn't GlobalLock cover garbage in STGMEDIUM?  Because you could get unlucky and have the garbage happen to be a valid HGLOBAL?

    Unfortunately for us we're definitely bound to XP interfaces for another year, at least.  Although MS is dropping support, too many customers still cling to it.  Perhaps in a year, or two.  Once Windows 8 is out, it will be much easier to justify "Vista or above" to our customers.

    Thanks for the explanation.  Bad context handlers are indeed a major source of pain.  Is there an easy way to control one's system to enable/disable IContext handlers as an end user?

  3. Ian Boyd says:

    My Documents contains a folder called "Torrents", which is actually a junction to a location on another (large slow) hard drive. This other drive is set to spin down after a few minutes of inactivity.

    If your context menu handler accesses the disk when i right-click on my "Torrents" folder, i have to wait 7-10 seconds for the drive platters to spin up.

    Similarly, if i open my Documents folder in Thumbnail view, Explorer touches the disk to try to show a preview of file contents in the folder. This causes Explorer to freeze for 5-7 seconds while i hear the platters spin up.

  4. Mike Dunn says:

    Thanks for providing the XP-friendly way of doing it, but the SHBindToFolderIDListParent API was added in Vista.

    [Developing the XP-equivalent version of SHBindToFolderIDListParent is left as an exercise. SHBindToFolderIDListParent does not add any fundamentally new functionality; it's just a helper function, like StrRetToStr. -Raymond]
  5. Alex Grigoriev says:

    @Ian Boyd:

    Move the junction to a subdirectory of Documents, and put a desktop.ini to that subdirectory to provide the icon without having Explorer to dig deeper.

  6. David Walker says:

    Yes, the file system is a hierarchical storage system, but that shouldn't prevent Explorer from getting some information for locally stored files and less information for files stored on tape.  

    (I asked the SQL Server team about returning certain information from a remote data source, and their answer was that the remote data source might not BE a Microsoft SQL Server database, and the information wouldn't exist.  My response was "but then again, it might be, and when it is, that valuable information can be returned".  I suggested that they think of it as marketing:  If the remote data source IS a Microsoft SQL Server database, the user gets more information back.  They were not impressed.)

  7. Wayne says:

    @Mordachai

    The autoruns tool from sysinternals gives the ability to turn off individual content menu handlers.  I go through that every so often and remove any context menu handlers for any apps I really don't need context menus for.

  8. Anonymous Coward says:

    Here's a summary of the threads and blog posts provoked by Igor, Memet and others, in case you don't feel like wading through them, which would be understandable because it's pages and pages of text.

    Igor doesn't want the e-mail resize picture dialogue box to show if its unnecessary. He is not alone. Raymond is immediately convinced of the reason why this is impossible: it would hit the disk. Many people prove that his objection is nonsensical, but Raymond refuses to properly read, let alone understand, their objections and keeps trotting out the same, by now disproven, point. Instead of admitting he's wrong and posting a correction, in the start of a new blog post he links back to the old ones and completely misrepresents what has been said there in a shameful display of dishonesty. (As well as, one assumes, in the hope that no one would be bothered to read that old material.)

    Also, some replies to the old thread. I would have posted them there, but Raymond disabled the comments on that thread. (If you don't plan on wading through that one, you can stop reading now.)

    @Peter: Leo responded to your post, before you even wrote it.

    @Lawrence, Raymond: The dialogue should have a better name. Something along the lines of ‘gathering files to send’, depending on specifics of course.

    @Lawrence: if you have to poke the registry around to do something, it is effectively impossible (for almost everybody).

    [I agree that "Gathering files to send" would be a better title than "Please wait while Windows decides whether or not to show you a dialog." But I thought the request was to show no dialog at all unless necessary. We just replaced one mandatory dialog with a different mandatory dialog. -Raymond]
  9. Alex Grigoriev says:

    "I think you'll find that people are more likely to respond if you don't insult them."

    I think the thought sequence for that misfeature was:

    "We now show the total playback time for the selected files. If multiple files are selected at once, it takes too much time"

    "OK. Let's not calculate accumulated stats if there are more than N files"

    The question that should have been asked:

    "Why not show accumulated stats that don't require additional disk access anyway? Users expect it"

    Another question that should have been asked:

    "Why don't we add/subtract full stats as the selection changes by one/a few files?"

    [Sorry, you lost me when you wrote "misfeature". -Raymond]
  10. Mordachai says:

    In the category of random grumblings, seems like the context handlers should have a user-friendly / easily accessed interface for controlling them built-in-to-windows.  Autoruns is a great suggestion for power-users, but not so much for Ma & Pa.

    And along those lines, having things like MS Office by default install Groove (or whatever it's current marketing name is) which has an unwanted context extension that applies to everything is unfriendly.  That should be an explicit opt-in, rather than by-default slather everywhere.

    However, I note these things here because MS is gargantuan and where else is there that it might be seen by someone of any influence whatsoever?  At least there's a snowball's chance of progress.  But not to rag on our kind benefactor, Raymond.

  11. Rick C says:

    "[I agree that "Gathering files to send" would be a better title than "Please wait while Windows decides whether or not to show you a dialog." But I thought the request was to show no dialog at all unless necessary. We just replaced one mandatory dialog with a different mandatory dialog. -Raymond]"

    I think they're playing a different game than you, Raymond, and theirs is called "Troll Raymond."

  12. David Walker says:

    If there is a way to tell Windows "don't ask me about resizing files to send by e-mail ever again", that's good.  If not, that's not good.  I don't happen to e-mail files in a way that would invoke that dialog anyway, so I don't know.

  13. Leo Davidson says:

    For those asking about tools to manage shell extensions, check out ShellExView from NirSoft, which is also very good.

    Autoruns and ShellExView are both great, free tools and each has its own strengths. I use both regularly.

  14. Medinoc says:

    On the topic of avoiding unwanted disc accesses, whose responsibility is it not to open "slow" files in an icon handler? Will Explorer refrain on its own from calling the icon handler, or does the icon handler need to make the check itself?

  15. Ooh says:

    @Steve Wolf (and off-topic): Totally agreed on the unnecessary Groove extension. Worst of all that the Office team doesn't have any official feedback channel I'm aware of. No site on Connect, no User Voice site or anything else, except Windows Error Reporting. But that's for crashes and hangs, not for feature or change requests…

  16. Random832 says:

    "I agree that "Gathering files to send" would be a better title than "Please wait while Windows decides whether or not to show you a dialog." But I thought the request was to show no dialog at all unless necessary. We just replaced one mandatory dialog with a different mandatory dialog."

    I'm not sure if I am correctly following this discussion, but it sounds to me like what's being suggested is replacing a dialog that always asks a question with a dialog that just has a progress display (requiring no action from the user) until it is determined if the question needs to be asked.

    A dialog that requires no action from the user is really more like a monolog.

  17. Chris says:

    "But this hits the disk, which makes baby context menu host sad." Should this not be "But this hits the disk, which makes baby context menu host cry."? I am assuming you are going for the Simpsons "Lies make baby Jesus cry" inferrence.

  18. Stefan Kanthak says:

    @Steve Wolf & Leo Davidson:

    Neither AUTORUNS.EXE nor ShellExView.exe are "great" tools to manage shell extensions. Both fumble ONLY with the CLSIDs/PROGIDs of shell extensions below [HKLMSoftwareClasses], and need administrative rights for any change.

    To give a rough picture…

    More than 12 years ago but (see MSKB article 216384) the following policy was introduced:

    [HKLMSoftwareMicrosoftWindowsCurrentVersionPoliciesExplorer]

    "EnforceShellExtensionSecurity"=dword:01

    The GUIDs of approved shell extensions have to be entered here:

    [HKLMSoftwareMicrosoftWindowsCurrentVersionShell ExtensionsApproved]

    "{00000000-0000-0000-0000-000000000000}"="arbitrary text"

    The GUIDs of blocked shell extensions can be entered there:

    [HKLMSoftwareMicrosoftWindowsCurrentVersionShell ExtensionsBlocked]

    "{00000000-0000-0000-0000-000000000000}"="arbitrary text"

    And MSKB article 918165 tells you about

    [HKLMSoftwareMicrosoftWindowsCurrentVersionShell ExtensionsCached]

    Yes, changes there need administrative rights too (I haven't checked whether the same registry keys under HKCU are evaluated too).

    If a shell extension implements a shell folder its enumeration can be controlled via (see MSKB articles 292504 & 810869):

    [HKLMSoftwareMicrosoftWindowsCurrentVersionPoliciesNonEnum]

    "{00000000-0000-0000-0000-000000000000}"=dword:00000001

    And if a shell extension implements a shell object that lives on the Desktop, in "My Computer" or on network computers, it can be en- or disabled with

    [HKCUSoftwareMicrosoftWindowsCurrentVersionExplorerHideDesktopIconsClassicStartMenu]

    [HKCUSoftwareMicrosoftWindowsCurrentVersionExplorerHideDesktopIconsNewStartPanel]

    [HKCUSoftwareMicrosoftWindowsCurrentVersionExplorerHideMyComputerIcons]

    [HKCUSoftwareMicrosoftWindowsCurrentVersionExplorerHideRemoteComputerIcons]

    "{00000000-0000-0000-0000-000000000000}"=dword:00000001

    or the resp. entries below

    [HKLMSoftwareMicrosoftWindowsCurrentVersionExplorerHide…]

    JFTR: Internet Explorer 5.x and later has similar settings (see MSKB article 182569):

    [HKLMSoftwareMicrosoftWindowsCurrentVersionInternet SettingsZones#]

    "1200"=dword:0001000

    "2000"=dword:0001000

    [HKLMSoftwarePoliciesMicrosoftWindowsCurrentVersionInternet SettingsAllowedBehaviors]

    [HKLMSoftwarePoliciesMicrosoftWindowsCurrentVersionInternet SettingsAllowedControls]

    Internet Explorer 6.x and later has "Add-On Manager" (see MSKB articles 555235 & 883256):

    [HKLMSoftwareMicrosoftWindowsCurrentVersionPoliciesExt]

    "RestrictToList"=dword:01

    [HKLMSoftwareMicrosoftWindowsCurrentVersionPoliciesExtCLSID]

    "{00000000-0000-0000-0000-000000000000}"="0" ; or "1" or "2"

    Did I miss anything in this glorious distributed mess?

    Of course: if you don't want a BHO to run in Windows Explorer, you have to use:

    [HKLMSoftwareMicrosoftWindowsCurrentVersionExplorerBrowser Helper Objects{00000000-0000-0000-0000-000000000000}]

    "NoExplorer"=dword:00000001

    or

    [HKCUSoftwareMicrosoftWindowsCurrentVersionExplorerBrowser Helper Objects{00000000-0000-0000-0000-000000000000}]

    "NoExplorer"=dword:00000001

  19. r says:

    He wants "instant response UIs", but doesn't think you should mind hitting the disk…

  20. Deduplicator says:

    Explorer/OpenFileDialog/SaveFileDialog/AnyShellNamespaceDialog/Others freeze: Here the non-existence of a guaranteed non-blocking API giving instant completion or otherwise asynchronous notification jumps into the users face.

    If it existed, the core explorer and plugin extensions (namespace/context menu/what have you) could use it to give the best information possible in the short term and improve upon that as fast as the OS can locate the information, while keeping the UI usable throughout.

    Anyone wants all those Pluging/Extensions/Utilities made multithreaded, which is the only other way I see? That would open a really huge can of particularly wiggly worms, epecially keeping in mind the truly spectacular, astounding and (those for the epicurean) subtle ways Joe Average Programmer can easily muck up even simple tasks (as evidence the Daily WTF and numerous topics here).

    So we get back to the topic of just over a week before. ("Why does my asynchronous I/O complete synchronously?" though perhaps it should be "Why does my asynchronous I/O block?")

  21. Cain T. S. Random says:

    The Windows XP code has a problem: SHBindToObject and SHBindToFolderIDListParent are Vista API's (cf. msdn.microsoft.com/…/bb762113(VS.85).aspx).

    [See earlier remark. You can adapt this function. -Raymond]
  22. Sniffnoy says:

    Raymond wrote:

    That's why I was amused by Memet's claim that "would hit the disk" is not acceptable for me. The feedback I see from customers, either directly from large multinational corporations with 500ms ping times or indirectly from individual users who collectively click Send Report millions of times a day, is that "would hit the disk" ruins a lot of people's days. It may not be acceptable to you, but millions of other people would beg to disagree.

    I'm a bit confused; is there a misnegation here?  Currently this seems to say, "You may find hitting the disk unacceptable, but the feedback I get shows that people find hitting the disk unacceptable."

    [There's a double-negative here. Memet claims that the "would hit the disk" justification is unacceptable. (I.e., that hitting the disk is acceptable.) There are lot of people who consider the "would hit the disk" justification acceptable. -Raymond]

Comments are closed.

Skip to main content