Exchange's MAPI repeatedly prompts for credentials
I had a case recently where the customer's symptoms where that his application was causing a security credential dialog over and over again even though correct credentials were supplied. Upon further investigation we found that the LmCompatibilityLevel registry value was set to 1. Since the application was using Windows RPC over HTTP (ROH) and the LmCompatibilityLevel registry value was set to less than 2, Windows RPC would not use the default credentials of the thread or process. This was causing the security prompts. However, the way that Exchange's MAPI is structured, it will never recover if a security dialog is shown.
There are two ways to allow the LmCompatibilityLevel to be 1 and use WIndows ROH. The first is to use mutual authentication. Since setting the LmCompatibilityLevel to 1 allows for using the LAN Manager (LM) security protocol, Windows RPC will require that the channel be secured with SSL and that the ServerCertificateSubject member of the RPC_HTTP_TRANSPORT_CREDENTIALS structure be present. Supplying the ServerCertificateSubject member will tell Windows RPC the expected server principal name. It will verify the server certificate against the principal name. If they don't match, it will not work. Unfortunately, Exchange's MAPI doesn't support mutual authentication. It provides no mechanism to set the ServerCertificateSubject from MAPI. Outlook's MAPI consults the PR_ROH_PROXY_PRINICPAL_NAME property for this information. Exchange's MAPI doesn't support this property.
The other option is to specify the credentials to be used via the PR_PROFILE_AUTH_USER_W and PR_PROFILE_AUTH_PASSWORD. This will set the transport credentials for the RPC_HTTP_TRANSPORT_CREDENTIALS structure and will circumvent the check for mutual authentication, allowing it to work For more information on setting these two properties see the "MAPI Download configuration guidance" document that comes with MAPI \ CDO download.
It's important to note that the LM security protocol uses weak password hashes. Therefore, customers are encouraged to use level 3 or higher.
More Resources
LmCompatibilityLevel registry value
technet.microsoft.com/en-us/library/cc960646.aspx
See the section labeled Automatic Logon Policy. Windows RPC uses WinHTTP internally when making the HTTP requests. When the transport credentials are NULL and the LmCompatibilityLevel is less than 2, the Automatic Logon Policy will be set to WINHTTP_AUTOLOGON_SECURITY_LEVEL_HIGH.
msdn.microsoft.com/en-us/library/windows/desktop/aa383144(v=vs.85).aspx
RPC_HTTP_TRANSPORT_CREDENTIALS structure (see ServerCertificateSubject and the Remarks section)
msdn.microsoft.com/en-us/library/windows/desktop/aa378624(v=vs.85).aspx
Security Watch
The Most Misunderstood Windows Security Setting of All Time
technet.microsoft.com/en-us/magazine/2006.08.securitywatch.aspx