MAPI Multithreading Rules


I wrote these rules out while debugging a crash in another MS product:

  1. All threads which use MAPI must call MAPIInitialize before doing any MAPI.
  2. All threads which use MAPI must call MAPIUninitialize before ending.
  3. No thread should ever call MAPIUninitialize if it didn’t already call MAPIInitialize.
  4. If MAPI_MULTITHREAD_NOTIFICATIONS is not used, the first thread to call MAPIInitialize should live longer than all other MAPI threads and should be the last to call MAPIUninitialize.

I won’t name the app, but it violated all 4 rules.

Known consequences of violating these rules:

  1. MAPI just can’t work on a thread that hasn’t been initialized. All MAPI calls will fail.
  2. Memory leak.
  3. Either a crash or ‘mysterious’ errors. Thread A initializes MAPI and starts doing some work. Thread B then calls MAPIUninitialize. If no other thread has initialized MAPI, then MAPI assumes everyone is done with it and cleans up. Depending on the timing, thread A will either crash or start getting failures in MAPI calls.
  4. When MAPI_MULTITHREAD_NOTIFICATIONS is not used, MAPI assumes it can tie the notification thread to the thread on which MAPIInitialize was first called. When this thread goes away, a pointer kept by the notification thread is invalidated, leading to a crash the next time we process a notification.
Comments (37)

  1. Konstantin Root (kroot@fusionone.com) says:

    Will there be any potential problems, if i call MAPIInitialize with MAPI_NO_COINIT flag and initialize thread myself with COINIT_MULTITHREADED?

  2. As long as you’re handling the CoUninitialize on the other end, no, that should not be a problem.

  3. I would add a fifth rule:

    5. The application must run a message loop on a thread that calls MAPIInitialize and MAPIUninitialize. And it is probably wise to make this thread the one that is the first to call MAPIInitialize and last to call MAPIUninitialize.

    I learned of this need the hard way after experiencing a hang in IProfAdmin::DeleteProfile and solving it by having my main thread which runs the message loop call MAPIInitialize.

  4. I must amend my fifth rule already — I overlooked something that now seems rather obvious.

    I wrote, "it is probably wise to make this thread [with the message loop] the one that is the first to call MAPIInitialize and last to call MAPIUninitialize." Actually this is not simply wise but essential.

    The documentation for MAPIInitialize states that, as step 6, the function "Initializes the notification engine, creates its window and…" Experimentation shows that this window is actually created only on the first call to MAPIInitialize from a given process — which makes some sense. So this window belongs to the thread that makes that first call, and that’s the thread that is going to receive its messages and must therefore run a message loop to process them.

  5. Hai says:

    I have a couple of more questions 🙂

    1. What happens when a first thread calls MAPIInitialize without MAPI_MULTITHREAD_NOTIFICATIONS and a second thread calls MAPIInitialize with MAPI_MULTITHREAD_NOTIFICATIONS? And the other way around? It that the first call made to MAPIInitialize will always win?

    2. If 1 is true, then all subsequent calls made to MAPIInitialize is only for the purpose of AddRef to MAPI subsystem, right? If this is the case, then as long as the first thread lives longer than every other threads, there should be no need to call MAPIInitialize on other threads. Unless MAPI is doing some extra work for each single thread?

    Thanks!

  6. No – ALL threads that use MAPI must be initialized. MAPI does do per thread work. See consequence 1. As to the first question, I think the first caller will win, but I’m not certain.

  7. Hai says:

    Stephen, thanks for the response. However, after applying this rules to my worker threads, I am stuck with some other kinds of problems. Hope that you could clarify some points 🙂

    In short this is what I am doing: I am writing a service provider, with AddressBook, Transport and MessageStore providers. Worker threads are created at run time to talk to servers. Before I read this blog, there is no MAPIInitialize and MAPIUninitialize pairs in the worker threads, and everything works fine (at least looks like). The consequence 1 above seems not applicable here.

    Then after reading this blog and related MAPI docs in MSDN, I thought "Oops, I am doing something abnormal, maybe it’s better to stick to the rules". Thus I added MAPIInitialize and MAPIUninitialize pairs in my worker threads. Everything continues to work fine, UNLESS if one worker thread does not end when the client shutdown. Sometimes when the worker thread stuck in talking to server, its MAPIUninitialize does not get a chance to be called. It looks like because of the missing MAPIUninitialize the MAPI subsystem refuses to shutdown our service providers. For instance, the IABProvider::Shutdown() is not called by MAPI.

    I am wondering, is the service provider supposed to be called in its Shutdown() from MAPI to be notified to call MAPIUninitialize()? Or, is there some other way that the service provider can be told to call MAPIUninitialize() when the client initiates the shutdown process?

    Any information will be appreciated. Thanks!

  8. Hai – I’m not sure what the point of confusion is. You started conforming to rule 1, but you note that you have problems when you don’t conform to rule 2. Here’s my suggestion – if you’ve got operations that may get stuck making calls out to servers, do those on threads that have nothing to do with MAPI. Then you’ll be free to do whatever you want with those threads without affecting MAPI’s stability.

  9. Tusi says:

    What is the situation, if i have a Windows Service and before the message loop i call MAPIInitialize (MULTITHREAD NOTIFICATIONS, NO_COINIT, NT_SERVICE) and after the message loop i call MAPIUninitialize. E.g. the MAPI subsystem will be kept alive as far as my service is alive. Will it be a problem? Since all the threads using the service are working fine with MAPI – until some strange errors appear after a very long run (4-5 days).

  10. MAPIUnitialize requires the message pump as part of shutting down MAPI.

  11. Kevin says:

    Stephen, comments made here are making me confused. Can you clarify where I’m misunderstanding things? Here’s my understanding:

    1. The first thread to call MAPIInitialize requires a message pump because it creates the hidden notification window.

    2. All other threads that subsequently call MAPIInitialize do not require a message pump.

    3. All threads also call a corresponding MAPIUninitialize

    4. The first thread eventually should call MAPIUnitialize. Because it owns the message pump, it is able to correctly shut down MAPI.

    Is this right?

  12. Kevin,

    Mostly right – you’ll also want to make sure that first thread is the last one to call MAPIUninitialize.

    Steve

  13. Kevin says:

    Stephen,

    Thanks for the response, that helps a lot. I think it’s safe to assume that if we’re an Outlook plugin, then Outlook owns the initial MAPIInitialize and last MAPIUninitialize call, right? I might still be out on a limb, but I’m figuring that while it’s great to be aware of this behavior, most Outlook plugins that properly init/uninit probably don’t have to worry about message pumps…

    -Kevin

  14. In that case – yeah – Outlook’s thread 0 would be handling the initial init/uninit as well as the message pump.

  15. Kevin says:

    Cool, you’re awesome, Stephen, and great blog.

  16. Jan Halama says:

    Stephen, great article. How would you handle using Outlook 2000 MAPI in multithreated enviroment? Outlook 2000 MAPI doesn’t support NoCoInit flag – which makes it imposible to use in in threads. Jan

  17. Actually, MAPI_NO_COINIT just tells MAPI not to call co-initialize (http://support.microsoft.com/kb/239853). This only relates to multithreading IF you’re also using COM and want to initialize it with COINIT_MULTITHREADED.

    So the answer there is: If you want to use Outlook 2000’s MAPI in a multithreaded environment, don’t use COM. 🙂

  18. Ok – not exactly. But there are a few things you need to configure on your SQL box if you want SQLMail

  19. Peter Katz says:

    Hi Stephen,

    Maybe you can help me this…

    Here is my case: In my COM Object, hosted in a COM+ application, I call MAPIInitialize in the FinalConstruct method (that happens in thread A). This object can be pooled (I allow object pooling to optimize MAPI connections) so this object is reusable. Next time the object is used it probably used from thread B. When the object is disposed that probably happens in thread C.

    My previous logic was to call MAPIUninitialize when the object was destroyed (thread C). As you can see this violates a few rules. As expected, I am having weird problems, like MAPIInitialize starts to fail after a few minutes, hours, days or never. When it happens the solution is to restart the CAS server.

    My key implementation problem here is I don’t have a way to control when the threads are created or destroyed to initialize/uninitialize MAPI. The only place I can have an idea of the life-time of a thread is in the WinMain function of my hosting DLL, but that is not a good place to call MAPI Initialize/Uninitialize. Right?

    My plan B solution is: When an object is created I check if MAPI is not initialized in the current thread (using TLS) and then I Initialize MAPI. When an object is destroyed I check if MAPI was initialized in the calling thread (using TLS) and then I uninitialize MAPI. So far it is working, but I am not sure what is going to happen tomorrow.

    Question: If a thread that initialized MAPI finishes without uninitializing MAPI, what happens? The application leaks memory?

    I know this is complicated, but if you can drop any idea, I will appreciate it.

    Thanks!

    PKatz

    PD: this is how I initialize MAPI

    mapicfg.ulVersion = MAPI_INIT_VERSION;

    mapicfg.ulFlags   = MAPI_NO_COINIT | MAPI_MULTITHREAD_NOTIFICATIONS | MAPI_NT_SERVICE;

    HRESULT hr = MAPIInitialize(&mapicfg);

  20. >Question: If a thread that initialized MAPI finishes without uninitializing MAPI, what happens? The application leaks memory?

    Or worse – you could end up with a crash since MAPI as a whole might still think that thread is valid.

  21. Peter Katz says:

    Stephen:

    Lets say in every method of my service objects I initialize MAPI, do some work, then uninitialize MAPI.

    My point (prev. post) was to have a bunch of service objects to optimize MAPI connections (with COM+ object pooling). If I initialize MAPI, create a some MAPI objects (e.g. a session), if after that I uninitialize MAPI my MAPI objects will be hanging, no?

    It is hard to follow the 4 rules, I am trying really hard here. 🙂

    PKatz

  22. ?? The rules shouldn’t be that hard to follow. If you initialize MAPI, uninitialize it. And of course, at the point where you’re uninitializing MAPI, you should have already cleaned everything up – so there wouldn’t be any objects "hanging around".

  23. Shoot says:

    Hi man!

    Can I initialize MAPI twice? (and then uninitialize then twice) on ine thread.

    There is my problem: after all work, when application completed work (MAPI had uninitialized) my App crashed on some thread with with stack:

    > EMSMDB32.DLL!35c053e3()

    [Frames below may be incorrect and/or missing, no symbols loaded for EMSMDB32.DLL]

    ntdll.dll!7c926abe()

    ntdll.dll!7c9268ad()

    ntdll.dll!7c91056d()

    MSMAPI32.DLL!35f72505()

    MSMAPI32.DLL!35f724e6()

    ntdll.dll!7c90fb6c()

    ntdll.dll!7c90fb71()

    ntdll.dll!7c90e9c0()

    ntdll.dll!7c91901b()

    ntdll.dll!7c90fb6c()

    ntdll.dll!7c90fb71()

    advapi32.dll!77dd6d3c()

    advapi32.dll!77dd7b5e()

    MSMAPI32.DLL!35f75cb0()

    EMSMDB32.DLL!35c05c86()

    MSMAPI32.DLL!35f76ba3()

    MSMAPI32.DLL!35f7673d()

    MSMAPI32.DLL!35f7662e()

    kernel32.dll!7c80b50b()

    kernel32.dll!7c8399f3()

  24. MFC says:

    Can a single thread call MAPIInitialize and then MAPIUninitialize and then again call MAPIInitialize and then MAPIUninitialize?

  25. MFC – yes, technically that’s OK, though in practice I wouldn’t recommend it. Most versions of MAPI have leaks of some form or the other that only surface when you call MAPIInitialize/MAPIUninitialize in a loop like that.

  26. MFC says:

    Thanks for the reponse.  I have an object that calls MAPIInitialize in the constructor and MAPIUninitialize in the deconstructor.  Another app creates and destroys the object.  The second time the object is created from the same instance of the application there is a failure to open the message store on Open Message Store.  Would my workflow be the cause of this?

  27. MFC – I’m working with the owner of the case you opened. He’ll send you instructions for gathering some data to figure out the cause of the failure.

  28. We had an issue recently where DDE broadcasts were being blocked on a system. The customer noticed that

  29. David Lowndes says:

    Hi Stephen,

    We have an Exchange server gateway (based on the old EDK sample code) which only calls MapiInitialize/Uninitialize on the main Windows message processing thread. It doesn’t use MAPI_MULTITHREAD_NOTIFICATIONS. Multiple worker threads in the gateway use MAPI calls.

    With reference to your rule & consequence #1, we do not see any MAPI call failures in the worker threads – I don’t understand why it works for us.

    Can you explain what benefit calling MAPIInitialize on each thread may have in our multi-threading gateway?

    I’m asking because we can see that there’s apparently a lot of contention in our worker threads on MAPI operations. Randomly breaking into the application inevitably shows multiple threads waiting inside MAPI operations on a critical section and only 1 thread that’s doing some rpc operation in the depths of MAPI.

  30. Somebody must be calling MAPIInitialize for you on the worker threads. That or you’ve been very lucky.

  31. David Lowndes says:

    Someone – who? We know our code and can’t think of anything the worker threads do that would automatically get MAPI initialised.

    I don’t think luck comes into it. We’re not being very lucky with this project. 😉

    Can you elaborate on the contention we’re seeing inside the MAPI calls on our worker threads? Is that normal in any multi-threaded application’s heavy use of MAPI?

  32. David Lowndes says:

    Hi Stephen,

    In doubting my sanity over this issue I’ve subsequently run our gateway under debug with a breakpoint set on _MAPIInitialize@4. It’s hit once – from the service initialisation message processing loop. I’m as certain as I can be that there are no per-thread calls to MAPIInitialize. Nevertheless we don’t experience any issues making MAPI calls from the worker threads.

    Are there perhaps significant differences between the threading/MAPIInitialize requirements of the server version (Exchange 2003) of MAPI and the client version of MAPI?

    Also, we’ve tried explicitly calling MAPIInitialize from the worker threads and while it hasn’t caused any issues, it’s not had any noticable effect on the contention we see inside MAPI calls. BTW, the parallel stacks debugging view in VS2010 is very illuminating for these situations.

  33. The contention has nothing to do with initializing MAPI (though you should still initialize it on every thread). The contention is because many operations in MAPI and in the ems providers are serialized.

  34. Renald says:

    Hi Stephen,

    First, good job with this blog.

    I have a problem with my MAPI program (writen in C++.NET).

    My program is a service, so I initialize MAPI with MAPI_NO_COINIT and MAPI_NT_SERVICE flags.

    But MAPIInitialize returns me the error 0x80004005.

    I wrote a second program (console) that make a MAPIInitialize but with MAPI_NO_COINIT flag only.

    MAPI has initialized with no problem.

    Do you know if it could be a problem of multi processor?

    And if I remove the flag MAPI_NT_SERVICE in my service program, what could be the consequently?

    Environment (new computer, office installed only):

    XP SP2

    Office 2003 v11.0.8173.0

    Administrator of my computer

    Thank you

    Renald

  35. Shanmuga says:

    Hi Stephen,

    Please let me know in your opinion, general considerations in improving performance of OUTLOOK Addin using MAPI?

    Thanks

    Shanmuga

  36. vipaw says:

    Hi Stephen

    MAPIInitialize fails with error MAPI_E_NOT_ENOUGH_RESOURCES abruptly for a binary 'XYZ' (MAPIInitialize with MAPI_MULTITHREAD_NOTIFICATIONS parameter).

    I have another service running all the time which uses MAPI initialized with MAPI_MULTITHREAD_NOTIFICATIONS.

    If this service is closed or restarted, the MAPIInitialize is successful again for few instances of binary 'XYZ'.

    It was working fine when binary 'XYZ' was not passing MAPI_MULTITHREAD_NOTIFICATIONS to MAPIInitialize.

    Do you think MAPI_MULTITHREAD_NOTIFICATIONS cause any king of MAPI issue if 3-4 binaries are run in parallel.

    Thanks

    vipaw

  37. Hi Stephen,
    Greetings!!
    I am working on a requirement where we need to import e-mails in MSG format into PST files. I have written a tool in C++ and used MAPI API to import MSG files into PSTs. The tool consists of a .dll that uses different MAPI APIs to read MSG file and import it into a PST and a .exe that passes MSG filename and PST filename to the .dll. We need to invoke .exe thousands of time to import thousands of MSG files into PSTs.

    I am observing following behaviour in my testing with outlook 2010:-

    1) When we are running single .exe repeatedly thousand of times by passing different MSG filenames and PST filenames each time, it works well without any error.
    2) When we are running multiple .exes simultaneously (thousand of times) by passing different MSG filenames and PST filenames each time, I am seeing random MAPI API failues as follows:-

    m_pIProfAdmin->CreateProfile() failed :: -2147024891 (MAPI_E_NO_ACCESS)
    MAPILogonEx failed :: -2147221231 (MAPI_E_LOGON_FAILED)
    m_pSvcAdmin->CreateMsgService() failed :: -2147467259 (MAPI_E_CALL_FAILED)
    OpenMsgStore() failed :: -2147221226 (MAPI_E_DISK_ERROR)

    Such failures are random in nature and about 10 transactions out of 500 are showing such failures.
    In single process mode run, I never saw any failure at all. I also tried putting inter-process locks around MAPI API calls so that they become serial in nature, still I am getting these errors when I am invoking multiple .exes simultaneously.

    Are you aware of any known issue related to multi-process invocation of MAPI API related to profile management? How to address such failures in multi-process execution?

    Your inputs will help me to proceed ahead with my implementation.

    Thanks in advance,
    Yashpal