Unicode MSG Files


[This is now documented here: http://msdn.microsoft.com/en-us/library/bb820947.aspx]

There was a thread a while back in the microsoft.public.win32.programmer.messaging newsgroup about how Outlook 2003’s new Unicode MSG files. The big question was how to create one. One poster even went as far as to reverse engineer the MSG format from scratch. Fortunately, there’s an easier way, which I just got clearance to publish.

We start with the canonical sample for MSG files:
http://support.microsoft.com/kb/171907

The key is the OpenIMsgOnIStg function. This function takes a ulFlags parameter. All you need to do to create a Unicode MSG file is to pass MAPI_UNICODE in this parameter. Once you do that, the resulting message will show STORE_UNICODE_OK in it’s PR_STORE_SUPPORT_MASK and unicode properties will work.

This flag is supported in this function on Outlook 2003 and higher only.

Comments (22)

  1. Pete Maclean says:

    I note that if this flag is used when an older version of Outlook is installed, OpenIMsgOnIStg fails with MAPI_E_UNKNOWN_FLAGS.  This would seem to accord with Stephen’s statement, "This flag is supported in this function on Outlook 2003 and higher only".  The important thing is that the error is not MAPI_E_BAD_CHARWIDTH as one might otherwise suspect.  

  2. scott mcfadden says:

    I noticed that when using extended MAPI to persist emails with large recipient counts to disk, it takes a very long time and the resultant msg file is quite large.  However, it appears that if you save the email / msg to disk using Unicode, the operation goes much faster and the msg file is much smaller.  Why is this?

    thanks

  3. Tim Valez says:

    I’m testing this with Outlook 2007 and OpenIMsgOnIStg() returns MAPI_E_UNKNOWN flags. Puzzling, this is the behavior one would expect from anything older than Outlook 2003 but definitely not from OL2K7. Any ideas?

  4. Stephen Griffin says:

    The flag is definitely supported with 2007. Have you tried the same code and just dropped that flag to make sure it’s not something else in your code triggering the problem?

  5. Tim Valez says:

    Yes, I tried that and it works: without the flag it creates a valid .MSG in the old format that I can load from Outlook. I also tried from OutlookSpy with same results, Dmitry’s code fails to save IMessage objects as Unicode MSG files. It won’t display any errors but the just created file is only 2.5KB long and won’t load in Outlook.

  6. Tim Valez says:

    Update: when I wrote a simple test program that used OpenIMsgOnStg() it worked as expected and I was able to create Unicode .MSG files.

    But the very same code will fail returning MAPI_E_UNKNOWN when executed in my Outlook extension (ECE)!? This reminded me of that old friend (or foe?) the MAPI32 stub DLL…

    The implementation of MAPI loaded for the test program resides in MSMAPI32.DLL and works as reported by Stephen.

    The version of OpenMsgOnIStg() loaded by Outlook 2007 is implemented by OLMAPI32.DLL and does not seem to support the MAPI_UNICODE flag.

    So I managed to work around it by dynamically loading MSMAPI32.DLL in my ECE. But now I’m curious… why Outlook 2007 needs a separate implementation of MAPI?

  7. Stephen Griffin says:

    Tim – the implementation of OpenIMsgOnIStg in msmapi32.dll is just a stub that ensures olmapi32.dll is loaded and then jumps directly into olmapi32!OpenIMsgOnIStg. It doesn’t do any parameter checking or manipulation. So that shouldn’t be any reason for a difference in behavior. I suspect there’s still something else going on here.

  8. Tim Valez says:

    Stephen – obviously I did not manage to communicate my point. I do understand how msmapi32.dll works (just a stub). What I’m saying is:

    msmapi32!IOpenIMsgOnIStg -> supports the MAPI_UNICODE flag

    olmapi32!IOpenIMsgOnIStg -> does not work, returns MAPI_E_UNKNOWN_FLAGS

    That’s why my simple test program worked (msmapi32 was loaded by the stub DLL) but exactly the same code failed when executed from my ECE (olmapi32 was loaded by the stub DLL).

    You can confirm this by directly loading msmapi32!OpenIMsgOnIStg and olmapi32!OpenIMsgOnIStg via LoadLibrary/GetProcAddress and comparing the results.

    As I said in my previous post, the existence of two MAPI implementations (msmapi32.dll and olmapi32.dll) picked my curiosity. Why do we need two of them?Outlook 2003 uses msmapi32.dll, I’ve never heard of olmapi32.dll before Outlook 2007…

  9. Stephen Griffin says:

    There aren’t two implementations. Many applications have hardcoded a path to msmapi32.dll, so for 2007 when we moved our MAPI implementation to olmapi32.dll we left a stub there so as not to break anyone. The stub doesn’t do anything but pass the call on to olmapi32. And this is where I have a problem with your logic:

    How could a stub support a flag when the underlying library, the one that actually does the work, doesn’t support it? The only way that would make sense is if the stub did something to the parameters before calling the underlying libraries, and I already verified both in the code and in the debugger that is not the case.

    There HAS to be something else going on. What are the paths to msmapi32.dll and olmapi32.dll that you’re loading in your tests?

  10. Whenever I find myself repeating the same message over and over again, I have to ask why I haven’t blogged

  11. pcunite says:

    This does not work on Exchange’s version of mapi nor with the new ExchangeMapiCdo.

  12. Hi Steven,

    What’s the situation with using MAPI_UNICODE with OpenIMsgOnIStg on Exchange server (2003)?

    I’m working on an Exchange gateway that needs to support Unicode MAPI. The message I get from Exchange’s MTS_OUT queue allow me to GetProps as Unicode, but our gateway creates an intermediate MSG file using StgCreateDocfile & OpenIMsgOnIStg, and if I specify the unicode flag with that, it fails with MAPI_E_UNKNOWN_FLAGS.

    Is there a server MAPI update that allows me to create Unicode MSG files on Exchange server?

    Assuming there isn’t, what other viable options are there?

    Thanks

    Dave Lowndes

  13. Stephen Griffin says:

    Exchange’s MAPI doesn’t support unicode MSG files. Like I said – only Outlook 2003 and higher.

  14. So, is there is no alternative easy way to create a Unicode MSG file on Exchange server 2003?

  15. Stephen Griffin says:

    No way period.

  16. Vilma says:

    Hi. I have the same problem with the MAPI_UNICODE on OpenIMsgSession.

    I have E2K3 Management Tool and Outlook 2007 SP1 installed on my XP.

    I use LoadLibrary on OLMAPI32.DLL and then call OpenIMsgSession with MAPI_UNICODE. I get MAPI_E_UNKNOWN_FLAGS in return.

    It seems like it is a general issue.

  17. Stephen Griffin says:

    Vilma – you’ve got two conflicting implementations of MAPI installed on that box, one that would take the flag and one that wouldn’t. This error is a result of that conflict. You need to take one of those implementations of MAPI off the box.

  18. Stuart Bray says:

    Has anyone used the canonical sample code from http://support.microsoft.com/kb/171907 in Outlook 2010 beta?  I can create .MSG files OK, but they do not open in Outlook.  

    I get "the operation failed" when trying to open an .MSG from the file system when Outlook is open, and I get "Cannot start Microsoft Outlook" when trying to open an .MSG file from the file system when Outlook is not running.

    Any ideas? Our routine has worked in previous versions of Outlook, up to Outlook 2007.

    Thanks!

  19. Stuart Bray says:

    Correction on my comment above: the sample code in KB171907 actually does still work just fine! 🙂  

    The problem we had was in drag and drop code on Win7.

  20. Surjya Sahoo says:

    Hello,

    We are getting an error 0x80070008 when StgCreateDocfile is called from an NT Service. StgCreateDocfile succeeds for first few calls, but after sometime we start getting this error and then we get it for each and every call. We are calling MAPIInitalize(NULL). Since this is an NT Service, MSDN says we should be calling it with MAPI_NT_SERVICE flags. But, if that is the case, then how is it succeeding for the first few calls? We also have McAfee antivirus installed on the machine: could it be interfering with the file creation? We are calling StgCreateDocFile with flags: STGM_READWRITE | STGM_SHARE_EXCLUSIVE | STGM_TRANSACTED.

    Please help us.

    Thanks in advance.

    ~Surjya.

  21. Stephen Griffin says:

    This appears to be a memory related error. I would look for other signs of resource exhaustion in the process.

  22. Bert says:

    Hello,

    I have managed to get my hands on the bytes of an IMessage by using CreateILockBytesOnHGlobal, StgCreateDocfileOnILockBytes, OpenIMsgOnIStg, WriteClassStg and CopyTo, followed by ILockBytes.ReadAt (Outlook 2010).

    Unfortunately, if I specify MAPI_UNICODE when calling OpenIMsgOnIStg, it does not have any effect. The result is a Non-Unicode compound file.

    Am I missing another Unicode flag somewhere or is this a flaw in the implementation of ILockBytes or even intended to behave this way?

    Please help me, I would prefer to avoid a temporary file.