Backup of Individual mailbox items


There are a lot of well -written and well-behaved item level backup applications. If you are writing one, then you will encounter many difficulties to get to the stage where the application works well. This type of application is challenging to build; however, you need to do a lot of homework and testing. I'm going to point out some of the hurdles you will need to get past, but please don't get discouraged as others have had to do so with their own applications.

There is no developer support/advisory for building a brick level backup application (backing up one item at a time); however, developer support can be provided on a per call basis for each call made.  Developers of such software usually struggle with throttling. The most common scenario is that they get a server busy error or slow syncing when starting the initial backup. A server busy back-off penalty is often 3-5 minutes. So, developers will need to write code to highly scale and not trigger throttling.  Changing throttling policies is not appropriate – applications really should work within the server limits.

Another great challenge is calendars – usually item level backup apps I've seen avoid this area because calendars are living things (especially meetings). Even doing something as simple as moving a calendar item to another folder and back break a meeting/appointment (the UID gets reset). MIME created and property created calendar items will have their organizer set to the account running the code (no way to override as Exchange is coded to work this way). UploadItem may get past this, however I've never tested to see its behavior. Restoring a meeting may cause issues even if everything is correct per the copy made earlier because it will not have updates send by recipients who would have accepted or rejected attending the meeting.

Support Policy:

Overview of Exchange Server Backup Methods
https://technet.microsoft.com/en-us/library/aa996125(v=exchg.65).aspx#MailboxBrickLevelBackups

Mailbox (Brick) Level Backups
There are several third-party backup programs that can back up and restore individual mailboxes, rather than whole databases. Because those backup solutions do not follow Microsoft backup guidelines and technology, they are not directly supported.

There are known problems with some versions of mailbox-level backup programs including loss of free/busy data and incomplete restores. Every effort is made to help, and to look at the errors and issues that you are having with this type of backup and restore, to determine if there are Exchange problems documented that could cause this error.

EWS is not a backup API:

EWS is not a backup API.  However, it does have a way to export an item and load it back with full fidelity through its Export Items and Upload Items calls.

One thing to consider if you want full data and easier handling is to pull needed properties they will search on in the archival then use ExportItem to get the full item as a blob.  The blob generated an only be reloaded into Exchange and there is no API to parse it.  EWSEditor (on GitHub) has implemented code they can look at (Search on ExportItem).  The EWS Managed API does not implement ExportItem/UploadItem, however you can do the calls with a raw EWS POST. For monitoring a mailbox, EWS Streaming Notifications is usually used – it was made for high scaling and to be very efficient.

Using EWS (or any API) to export MIME and recreate items from MIME is usually bad approach as properties will be lost – i.e. you won't end up with the same item in the end.  Keep in mind that Exchange does not store MIME – it generates it on the fly and that processing is server intensive.

Pulling all properties is also not a good idea – you never get all properties set when you ask for them, so you won't be able to properly recreate the item later. It is possible to loop try to specify every possible property in with an immense amount of EWS calls, but that's going to either take a very long time – in for a large mailbox the needed calls may not finish until after your retired (half kidding).

Backing up and restoring individual items is known as a brick-level backup. There is no developer support for brick-level backup. However, developer support can usually be provided for issues with most of the calls involved in a brick-level backup.

This article talks about supported backup APIs for Exchange:

About: Exchange APIs to backup and restore data
http://blogs.msdn.com/b/webdav_101/archive/2015/06/01/about-exchange-apis-to-backup-and-restore.aspx

EWS: If you need to save an item and load it elsewhere, avoid MIME:

While email MIME is an internet standard, you will lose properties when you have Exchange create MIME based-upon an email. EWS provides a way to save off things like email with full fidelity and load it back – this is done with ExportItem and UploadItem. Note that these calls are not meant as calls for implementing a brick-level backup application.

How to: Export items by using EWS in Exchange
https://msdn.microsoft.com/en-us/library/office/dn672317(v=exchg.150).aspx

Exporting and importing items by using EWS in Exchange
https://msdn.microsoft.com/en-us/library/office/dn672316(v=exchg.150).aspx

EWS Throttling:

This type of application will encounter throttling if its not written to properly scale to get pack throttling limits.

Exchange throttling is your friend… well, more like a police officer.
https://blogs.msdn.microsoft.com/webdav_101/2015/11/18/exchange-throttling-is-your-friend-well-more-like-a-police-officer/

You are doing too much at one time: EWSMaxConcurrency – "too many concurrent connections opened"
https://blogs.msdn.microsoft.com/webdav_101/2018/06/02/you-are-doing-too-much-at-one-time-ewsmaxconcurrency-too-many-concurrent-connections-opened/

EWS ServerBusyException – The server is too busy – for you
https://blogs.msdn.microsoft.com/webdav_101/2018/03/20/ews-serverbusyexception-the-server-is-too-busy-for-you/

Graph/REST:

Graph and REST do not have calls to backup data and were not made for individual item backups. You can pull item properties and MIME. There are no calls like ExportItem/UploadItem which EWS has. Throttling will also be an issue, so you will need to master self-throttling for your application in order to not get throttled.

Graph & REST Throttling:

Microsoft Graph throttling guidance
https://developer.microsoft.com/en-us/graph/docs/concepts/throttling

Microsoft Graph throttling guidance
https://developer.microsoft.com/en-us/graph/docs/concepts/throttling

Throttling coming to Outlook API and Microsoft Graph
https://developer.microsoft.com/en-us/office/blogs/throttling-coming-to-outlook-api-and-microsoft-graph/

Microsoft Graph error responses and resource types
https://developer.microsoft.com/en-us/graph/docs/concepts/errors

Best practices with Microsoft Graph: Making your applications more performant, resilient, and reliable
https://azure.microsoft.com/en-us/resources/videos/best-practices-with-microsoft-graph

Issues with Microsoft Graph
https://developer.microsoft.com/en-us/graph/docs/concepts/known_issues

Setting some properties:

Some properties can only be set by the Exchange and are stamped by the Exchange Store. Be sure to check if a property is read-only to see if you can set it.

Items created by MIME may be missing things like extended properties. Not all properties on an item are expressed in MIME when Exchange generates it from stored property data. Extended properties will become x-headers in MIME if created properly.

Provision x-headers by using EWS in Exchange
https://docs.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-provision-x-headers-by-using-ews-in-exchange

Calls to EWS's ExportItem and UpdateItem will allow you to get all properties and recreate an item with those properties. So be sure to check out these calls.

On-premise VSS requesters:

On prem can use a VSS requester for backups, however customers should be doing backups through replication of data.  I have not seen a VSS case in years and don't know if its viable for 2016.

About: Exchange Backup and Restore API
https://blogs.msdn.microsoft.com/webdav_101/2015/05/18/about-exchange-backup-and-restore-api/

About: Exchange VSS Writer – Exchange backup and restore
https://blogs.msdn.microsoft.com/webdav_101/2015/06/01/about-exchange-vss-writer-exchange-backup-and-restore/

Other types of backing up items I've seen:

Backing up items as MSG files is not possible with EWS/REST/Graph. Only Outlook Object Model and MAPI can create such files.

About: EWS/REST/Graph and MSG file
https://blogs.msdn.microsoft.com/webdav_101/2018/05/01/about-ewsrestgraph-and-msg-files/

Reading Exchange's data directly from the files or by a database connection is very much not supported:

HOWTO: Read unmounted Exchange EDB files.
https://blogs.msdn.microsoft.com/webdav_101/2008/09/03/howto-read-unmounted-exchange-edb-files/

Backing up items in Outlook to a PST file using OOM or MAPI can be problematic also. Things are not always restored exactly as they were. Code running in Outlook which uses OOM will need to run on the main thread – running on a background thread with OOM may cause a hang when OOM/Outlook tries to display a dialog. MAPI code can run in a background thread in Outlook. Both OOM and MAPI are not supported for advisory on individual item backup though support can be provided for the individual calls being made.

Further reading:

About: EWS/REST/Graph and MSG files
https://blogs.msdn.microsoft.com/webdav_101/2018/05/01/about-ewsrestgraph-and-msg-files/

 

 

 


Comments (0)

Skip to main content