Transitioning to Exchange Web Services Notifications

Jason Henderson announced in his Exchange Developer Roadmap blog post that we are officially deprecating, among other things, Store Event Sinks and SMTP Event Sinks. This means that Exchange developers will have to rely on Exchange Web Services (EWS) notifications and transport agents to perform many of the tasks previously handled by these event sinks. To help with the transition, I’m going to discuss how to manage the notification subscription lifecycle. Specifically, I’m going to focus on push notifications. I’m going to assume that the reader is familiar with both the Notifications documentation that is available on MSDN (Event Notifications) and the Exchange Web Services API. Also, Inside Microsoft Exchange Server 2007 Web Services is a great book that was written by some of the people who developed Exchange Web Services and has a lot of detailed information on this and other topics.

Architecture

Notification subscriptions (both push and pull) are stored in memory on the Client Access server (CAS). In the case of push notifications, when a subscription comes in, the CAS begins to poll the Mailbox server every couple of seconds for new events. If any events have occurred since the last poll, the Mailbox server responds to the CAS with notifications about the events and the associated watermarks. The CAS then filters these events according to the information that is provided in the subscription, and creates a notification response that is populated with those events. Along with the events, the Client Access server appends the watermark of each event, as well as the “previous” watermark – the watermark of the last event of the previous message (the Mailbox server maintains the watermarks and updates them as events occur). Finally, the CAS sends this message to the client.

Ordering of Events

We are frequently asked – are notifications guaranteed to arrive in the order in which the events happened? The answer is yes! When the CAS sends a notifications message to the client application, the client must reply with either an “OK” or an “Unsubscribe” response. Until the CAS receives an “OK” response, it will not send out any more NEW notifications messages (even if new events occur). It will, however, go into a retry mode, if it doesn’t receive a response within the timeout amount of time, which is set on the subscription request (this is the topic of the next section). Thus, no new notifications message will be sent out by the CAS until the previous one has been acknowledged by the server. This guarantees that the order is preserved.

Timeouts

We are also often asked how push notifications timeouts work. Suppose your client subscribed to push notifications, and then unexpectedly goes offline. The server will not be able to push out the next set of events as notifications to the client (or, more importantly, the server will not get a response to its notifications message). Rather than give up right away, the server will retry to push them out up to three times. After the first failure, the server will wait the timeout amount of time (that’s the timeout that you set in the subscription request) and try again. If that attempt fails, the server will wait twice the timeout time and try again. If that fails, the server will wait three times the timeout time and try again. Finally, if the third retry attempt fails, the server will give up and delete the subscription. There is no Web service method to check on the life of the subscription, but the CAS sends StatusEvent messages every timeout amount of time, so if you don’t get a StatusEvent message within the specified time, you can assume that the subscription was deleted.

If your client application goes offline for long enough, such that the CAS went through its retry procedure and deleted the subscription, it is still possible (in almost all cases; see the next section for clarification) for your client to receive notifications that happened since the last notifications message that the client processed. To do this, you must issue a subscribe request and pass in the watermark from the point where you want to get notifications. Most of the time, you will want to pass in the last watermark seen (thus we recommend you keep track of it).

In summary, there are really two ways to end a subscription on a CAS:

  • You can send “Unsubscribe” in the response message (as a response to a notifications message from the CAS).
  • You can become unresponsive to notifications messages.

We recommend the first method (sending an unsubscribe message) rather than the second method, which causes the Client Access server to perform unnecessary work.

Reliability

Reliability of notifications is another area of confusion. Notifications are reliable – that is, all events for a mailbox that happen on the back end can be seen by the client, unless the backend server goes down. A client can always re-subscribe with the last watermark (a thing to keep in mind here: watermarks are good for about 30 days, after which they expire) and in this way get any potential events that have happened since that last watermark. The one exception to this is mailbox moves. Remember that watermarks are unique to mailbox databases, and only the Mailbox server that the watermark came from knows about that watermark. In the case of a mailbox move to another server, after the mailbox is moved, the original server notifies the CAS that no more events for that mailbox will occur, and the CAS deletes any subscriptions for that mailbox. Prior to deleting the subscriptions, the CAS will send a failure message to the client application, thus notifying it that the subscription will be deleted. The client application may now try to re-subscribe with the latest watermark. However, because the mailbox is now on another server, the previous watermarks are no longer valid (sending a subscription that includes an invalid watermark will result in an error returned by the server). Therefore, the client will have to create a new subscription from that point on. This means that any events that happened to that mailbox between the mailbox move and the creation of the new subscription creation will not be delivered.