Best Practices: EWS Managed API coding for Exchange

Article
04/12/2016

There are issues which we seem to get a lot of cases on which are tied to the way a customer writes their code. Often there are sections of code which were missing or that they otherwise should have implemented. If your running into an issue or writing new code, then please read all points below and each linked article. Note that this article is not meant as a primer on EWS development but rather to point out things which many developers miss and seek support assistance on.

If you are looking for primer articles, then please check out the following:

About: Exchange Web Services (EWS)
https://blogs.msdn.com/b/webdav_101/archive/2015/05/03/getting-started-with-ews.aspx

Getting started with EWS
https://blogs.msdn.microsoft.com/webdav_101/2015/05/03/getting-started-with-ews

Version of the EWS Managed API

If your using a version of the EWS Managed API older than 2.2 then you should upgrade. There have been many fixes between prior versions of this API and the issue you may be fighting may have already been fixed.

The 2.2 release of the EWS Managed API can download here:

Microsoft Exchange Web Services Managed API 2.2 (15.00.0913.022 – Date Published 5/20/2014)
https://www.microsoft.com/en-us/download/details.aspx?id=42951

The EWS Managed API is open-sourced now. This means you can download the code from GIT and build it. The GIT version has changes and fixes not in the released 2.2 version of the EWS Managed API.

EWS Managed API… open source? YES! It's now Open Source!!!
https://blogs.msdn.microsoft.com/webdav_101/2014/09/29/ews-managed-api-open-source-yes-its-now-open-source

Authentication vs Authorization.

Let's start from the beginning with some basic information on authentication and authorization. The first thing to keep in mind is that Exchange itself does not authenticate – IIS handles that part. Authorizing access to things in a mailbox is handled by Exchange. So, if you got an issue like using NTLM or BASIC authentication in an EWS POST then consider how the Exchange IIS is configured, how the client code is written to authenticate and what's happening between the client application and Exchange's IIS server that might mess with the call. If Impersonation or delegation don't work but you can authenticate then the issue is most likely going to be with Exchange or the client code and not with IIS as it would be an authorization issue.

When you authenticate against Exchange Online then you would use the mailbox owner's UPN and not the SMTP address. Usually the UPN and SMTP address are the same – however, they are not in all cases. Default windows credentials cannot be passed to Exchange Online – they must be specified. You have the choice of NTLM, Basic and oAuth for Exchange Online – there are no other supported options.

So, keep in mind that you authenticate with a UPN and not an SMTP address. You can add a UPN to match an SMTP server suffix.

Add User Principal Name Suffixes
https://technet.microsoft.com/en-us/library/cc772007.aspx

One of the most common issues is getting a 401 error – which means the call did not get authenticated. The usual cause is that the credentials are wrong, EWS/Exchange is not configured for the type of authentication or the code is going against the wrong URL. So, if you get this error or a similar one then try using OWA, Microsoft Remote Connectivity Analyzer (EXRCA), EWSEditor and even Outlook to test using the same credentials.

Testing connectivity:

If your code is failing to authenticate then consider using one of the following to help rule out your code as being the cause:

This is the official tool for testing connectivity:

Microsoft Remote Connectivity Analyzer (EXRCA)
https://testconnectivity.microsoft.com

Note: This site also has a downloadable tester which can work for servers which do not expose their EWS end points to the web.

The sample app below is a sample which my team wrote and it uses the same API for working against Exchange. You can download both the run time and the source.

EwsEditor
https://ewseditor.codeplex.com

AutoDiscover:

Its best to use AutoDiscover to locate the Exchange Web Service being accessed. The best practice is to do AutoDiscover and cache the EWS URL then set it on the service object the next time you need to set the EWS URL on the service object until calls fail using that URL.

The EWS URI for Exchange Online by default is "https://outlook.office365.com/EWS/Exchange.asmx". Some companies though have a custom URI which takes into account their company domain. If AutoDiscover is used, then your code would work if the URI for your code gets changed due to your company using a custom URI.

AutoDiscover is a series of processes. The EWS Managed API will first try to do in-network SCP AutoDiscover lookups against your AD then if that fails it will try using POX AutoDiscover. Exchange Online is out of network, so you can avoid the SCP lookup part by turning it off on the service object. This applies to any Exchange server which is not in-network with the machine running your code.

// Exchange Online is out of network to you, so turn off SCP Autodiscover lookups.
// If you had an on-premise serer then this likely would be set to true.
service.EnableScpLookup = false;

There is a callback on the EWS Managed API which your code MUST handle. This callback is used to allow your code to allow or block URL changes during the AutoDiscover process. As an example most developers would only want URI starting with HTTPS to be accessed at a minimal. Some developers may wish to add additional checks.

ExchangeService.AutodiscoverUrl method (String, AutodiscoverRedirectionUrlValidationCallback)
https://msdn.microsoft.com/en-us/library/office/dd635285(v=exchg.80).aspx

Example from article above:

static
bool RedirectionCallback(string url)
{
// Return true if the URL is an HTTPS URL.
return url.ToLower().StartsWith("https://");
}

static
void GetUsersEwsUrl(string userEmailAddress, SecureString userPassword)
{
ExchangeService service = new ExchangeService();

// Set specific credentials.
service.Credentials = new NetworkCredential(userEmailAddress, userPassword);

// Look up the user's EWS endpoint by using Autodiscover.
service.AutodiscoverUrl(userEmailAddress, RedirectionCallback);

Console.WriteLine("EWS Endpoint: {0}", service.Url);
}

SSL Redirection Callback:

There is an SSL callback which your code should handle. In most cases this would be a MUST implement item. Under .NET when SSL is involved with HTTP communication and there is no locally stored certificate then the certificate being used needs to get an "OK" from your code. Exchange Online and pretty much every Exchange server (hopefully all) have SSL as being required when EWS is used. You should implement at least minimal checking and check into additional checks you can do to help make this check more robust.

How to: Validate a server certificate for the EWS Managed API
https://msdn.microsoft.com/en-us/library/office/jj900163(v=exchg.150).aspx

Here is code from the article above:

ServicePointManager.ServerCertificateValidationCallback = CertificateValidationCallBack;

     private
static
bool CertificateValidationCallBack(
         object sender,
         System.Security.Cryptography.X509Certificates.X509Certificate certificate,
         System.Security.Cryptography.X509Certificates.X509Chain chain,
         System.Net.Security.SslPolicyErrors sslPolicyErrors)
    {
      // If the certificate is a valid, signed certificate, return true.
      if (sslPolicyErrors == System.Net.Security.SslPolicyErrors.None)
      {
        return
true;
      }

      // If there are errors in the certificate chain, look at each error to determine the cause.
      if ((sslPolicyErrors & System.Net.Security.SslPolicyErrors.RemoteCertificateChainErrors) != 0)
      {

        if (chain != null && chain.ChainStatus != null)
        {
          foreach (System.Security.Cryptography.X509Certificates.X509ChainStatus status in chain.ChainStatus)
          {
            if ((certificate.Subject == certificate.Issuer) &&
               (status.Status == System.Security.Cryptography.X509Certificates.X509ChainStatusFlags.UntrustedRoot))
            {
              // Self-signed certificates with an untrusted root are valid.
              continue;
            }
            else
            {
              if (status.Status != System.Security.Cryptography.X509Certificates.X509ChainStatusFlags.NoError)
              {
                // If there are any other errors in the certificate chain, the certificate is invalid,
                // so the method returns false.

return
false;

              }
            }
          }
        }

        // When processing reaches this line, the only errors in the certificate chain are
        // untrusted root errors for self-signed certificates. These certificates are valid
        // for default Exchange server installations, so return true.

return
true;

      }
      else
      {
        // In all other cases, return false.
        return
false;
      }
    }

Connection Limits:

There is a default maximum connection limit for different types of .NET applications. For a WinForm application its 2 and for ASP.NET its 10. This is purely a .NET client connection limit imposed on your application and does not reflect the settings on the Exchange server.

ServicePointManager.DefaultConnectionLimit Property
https://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.defaultconnectionlimit(v=vs.110).aspx

EWS Impersonation:

Thread impersonation is not supported against Exchange. EWS however has its own impersonation scheme. Permission needs to be granted to the account which will impersonate. Also, permission needs to be granted to the mailbox or storage group which will be accessed with EWS Impersonation.

How to: Configure impersonation
https://msdn.microsoft.com/en-us/library/office/dn722376(v=exchg.150).aspx

How to: Configure impersonation (2013)
https://msdn.microsoft.com/en-us/library/office/dn722376(v=exchg.150).aspx

Configuring Exchange Impersonation in Exchange 2010
https://msdn.microsoft.com/en-us/library/office/bb204095(v=exchg.140).aspx

Exchange Impersonation vs. Delegate Access
https://blogs.msdn.com/b/exchangedev/archive/2009/06/15/exchange-impersonation-vs-delegate-access.aspx

When EWS Impersonation is used the X-AnchorMailbox always should be correctly set. Without doing so you may get 500 or 503 errors at times. It is critical for performance and also for notifications with Exchange Online/Exchange 2013. Not setting it can double or more the time it takes to complete the call. In some cases, you can also get timeouts. The rule is to always set this header when using impersonation – this will make your EWS Impersonated code from Exchange 2007 work better with Exchange 2013. It should be set to be set to the mailbox being accessed with the exception of when streaming notifications are being done and in that case it should be set to the first mailbox in the subscription group. With Exchange Online there are additional headers which need to be set for affinity.

Example: service.HttpHeaders.Add("X-AnchorMailbox", targetSmtp);

Also see:

How to: Route public folder hierarchy requests
https://msdn.microsoft.com/en-us/library/office/dn818490(v=exchg.150).aspx

Things to try if Impersonation is not working:

Be sure the credentials are correct.
Navigate to the EWS service URI in a browser and see if you get a log-in dialog box. This would likely be https://outlook.office365.com/EWS/Exchange.asmx
for Exchange Online.
Try accessing the mailbox of the authenticating account using the owner's credentials using EWS and OWA.
Try accessing the mailbox being accessed using the its owner's credential using EWS and OWA.

X-AnchorMailbox

This header always should be set when using EWS Impersonation. If you do not, then you may experience one of several issues which include slowness and errors such as a 500 or 403 error. When Impersonating is used, Exchange will be using the mailbox server of the impersonation account to move the calls through and as such can be slow if you don't tell it to use the server of the mailbox your accessing, which is done by using the X-AnchorMailbox header. Further, some calls may fail if you don't use this setting.

Example: service.HttpHeaders.Add("X-AnchorMailbox", targetSmtp);

Combining EWS Impersonation and Delegate Access is not working:

When code uses both EWS Impersonation and delegate access the call is subject to limitations and restrictions for both.

Be sure to try the following:

Check the Impersonation and general EWS throttling settings to be sure you're under the limits set.
Because delegate access is in the mix your code will be subject to delegate access limitations, which include the maximum MAPI connection limit (MapiExceptionSessionLimit) which kick-in when too many delegate operations happen at the same time. If you don't need to be using delegate access in the EWS call then consider removing it.
Be sure to set the X-AnchorMailbox header to the mailbox being impersonated or you might run into errors on some EWS calls when working with items – such as creating a meeting in a delegate calendar.
Be sure that EWS Impersonation and Delegate access work without a combined call.

Delegate access results in a MAPI connection limit error (MapiExceptionSessionLimit):

Yes, this says MAPI – the throttling limits for delegate access with EWS are tied the same limits for MAPI on Exchange. These errors are usually intermittent. In the response Below are the reasons which I see show up often from our customers:

The code is exceeding the limit.
The limit is smaller than you think it is. Recheck it and adjust as needed.
There are other applications using the same account which your application is using. Its best that service style applications have their own account to run under to avoid this situation.
The code is mixing EWS Impersonation and delegate access and don't realize it.
The code is accessing an item by ID and it actually is for an item in a mailbox other than you think it is and implicit delegate access is being used.
There is a load balancer which is causing the issue. With some load balancers this can happen when its CPU hits 85%. This happens when the load balancer drops the connection and Exchange is not notified, so it keeps the connection alive based-upon the "keepalive" setting – the default is two hours. Try a test by bypassing the load balancer to see if the issue goes away so that you can rule it out – this is a necessary troubleshooting test.

You can the MAPI connection limit for your on-premise server. However, its best to work within the default.

Exchange Store Limits
https://technet.microsoft.com/en-us/library/ff477612(v=exchg.141).aspx

Multithreading:

Be sure that none of the EWS managed API objects are declared as "public static" – that would make them available to be used threads and using them that way is known to cause issues and is also not supported. Each EWS Managed API object needs to be used on the thread it was created on.

Also be aware you need to work within the throttling limits of Exhange. One easy way to run into trouble with multithreading is if your application has too many concurrent calls at one time – yes, Exchange will throttle for that. There are limits and your code needs to self-throttle so Exchange won't have to.

Throttling:

There are several types of throttling involved with EWS calls with the EWS Managed API. Throttling is not your enemy. It's there to help assure everyone plays nice – even the bullies. All applications need to work within the bounds of Exchange throttling.

Exchange connection throttling: This is controlled by an Exchange Administrator via PowerShell calls. These settings cannot be changed for shared Exchange Online.

Exchange MAPI connection throttling: You will see this limit hit when EWS is doing some sort of delegate access. The setting can be overridden for on-premise Exchange servers via a registry key.

.NET connection throttling: All .NET applications have a default connection limit. You can override this in code by setting ServicePointManager.DefaultConnectionLimit. It can also be overridden with an application configuration setting.

If you are being throttled due to Exchange connection throttling then you may see information on what that throttling is by looking at the response body, response headers and the error response collection.

Please review:

Exchange throttling is your friend… well, more like a police officer.
https://blogs.msdn.microsoft.com/webdav_101/2015/11/18/exchange-throttling-is-your-friend-well-more-like-a-police-officer

Best Practices – EWS Throttling
https://blogs.msdn.microsoft.com/webdav_101/2015/05/03/best-practices-ews-throttling

EWS Calls are slow:

There are many things which can cause slowness. Here are a few points:

Ask for the minimal: Only ask for the properties you absolutely need. Request IdsOnly plus the properties needed – no more. Everything requested in a call has a performance cost and the fewer the properties requested the better. Only think about pulling all properties if your building some sort of explorer application.
Search smart: Be smart on what you search for and what criteria your using. The search criteria can cause slow searches – especially when not using indexed fields. The data being retrieved can make a big difference also.
Searching in a vast folder or too deep: The depth of your search can make a huge difference. The more there is to look through the slower the search. If search indexes cannot be used, then the time to complete a search can take a very long time.
Think about the overall call: One scenario I see every few months is where a customer is searching a folder with an extreme amount of items while asking for a lot of data (usually including the message body) and using inefficient search criteria. What usually happens is that they get a timeout. We make the recommendations as noted above and often get push back and we have to go over each point. In the end they usually make the recommended changes to their code. So please think about the overall operation being performed.
Be realistic: Be realistic about how long a call can take. You may find that you're getting sub second responses most of the time but not all the time. Don't let this tempt you into setting a 1 second timeout. Some calls can and will take much, much longer. Sometimes servers will be slower due to load and thus your calls come back slower. A call which completes well under a second 99% of the time may take much longer (example 7 seconds) if the server is tied up.
Performance can vary: If you're going against a shared server (such as an Exchange Online shared server) or virtualized server then keep in mind that performance can vary because you are not the only one using that server and may not be the only company using that server.
Target the mailbox: Set the set the X-AnchorMailbox header when using EWS Impersonation. I've seen this cut the response time for a call in half.
Behave or be throttled: Be aware of throttling you might encounter. If you slam the server then it may react by throttling your calls.

See:

OUTBOX: Understanding and Fixing Slow Exchange Web Services Code (Part 1)
https://blogs.msdn.microsoft.com/mstehle/2008/07/17/outbox-understanding-and-fixing-slow-exchange-web-services-code-part-1/

EWS Best Practices – Searching
https://blogs.msdn.microsoft.com/webdav_101/2015/05/03/ews-best-practices-searching/

Coding for failure:

It's important to have be prepared for failures when your code goes into production. One thing to consider is to add client side logging so that you can see what goes over the wire. This is often much better than looking at what you have coded in your application since it shows what was actually sent versus what your code is building to send. Another thing is to add settings to the service object which helps in tying your application logs with Exchange server logs.

Sometimes calls may fail. Your application should be able to handle every failure appropriately. It's better to assume that any call would fail at some point – so code for it and if it does fail then either your application will not be as impacted or you will be able to troubleshoot the issue faster.

Be sure to review this article:

EWS best practices – Tracing and logging
https://blogs.msdn.microsoft.com/webdav_101/2015/05/03/ews-best-practices-tracing-and-logging

Things to test with:

This is the official tool for testing connectivity:

Microsoft Remote Connectivity Analyzer (EXRCA)
https://testconnectivity.microsoft.com/

The app below is a sample which my team wrote and it uses the same API for working against Exchange.

EwsEditor
https://ewseditor.codeplex.com/

Other:

EWS Best Practices
https://blogs.msdn.microsoft.com/webdav_101/2015/05/03/ews-best-practices/

Best Practices – EWS Authentication and Access Issues
https://blogs.msdn.microsoft.com/webdav_101/2015/05/11/best-practices-ews-authentication-and-access-issues/

Getting started with EWS
https://blogs.msdn.microsoft.com/webdav_101/2015/05/03/getting-started-with-ews/

About: Exchange Web Services (EWS)
https://blogs.msdn.com/b/webdav_101/archive/2015/05/03/getting-started-with-ews.aspx

Best Practices: EWS Managed API coding for Exchange

Additional resources