ShellExecute fails when called from Internet Explorer Protected Mode

  • Article

Summary

When calling ShellExecute to open of Office document from Internet Explorer, the call will fail with a “File not Found” error if Protected Mode is turned on. The problem occurs after installing Office 2007 Service Pack 1 on Windows Vista.

Cause

Internet Explorer Protected mode is available when running Internet Explorer 7 on Windows Vista. When running in protected mode, Internet Explorer runs as a Low Integrity process. The Office applications on the other hand, run with Medium or High Integrity. The ShellExecute API attempts to establish a DDE connection with the Office application and execute a DDE command to open the Office document. The Office applications no longer allow DDE conversations from Low Integrity applications.

This problem effects Internet Explorer Plug-Ins and COM components instantiated from client script.

Workarounds

Add the site to the Trusted Sites collection (Protected Mode is turned off) or create a broker process which will start with elevated privileges and call ShellExecute from the elevated process. The file to open can be passed on the command line to the new process.

More Information

Understanding and Working in Protected Mode Internet Explorer
https://msdn2.microsoft.com/en-us/library/bb250462.aspx