Understanding Persistence Points in Biztalk Orchestration

Often ignored, but a really important point when designing orchestrations is of course the persistence points. Understanding of the persistence points becomes really an important factor if you are dealing with orchestration performance. Lets take a glance at what they are and how we can take care of them.  

The SQL Server does bulk of the job at the backend to keep the BizTalk server running.  BizTalk has this inherent behavior of saving the orchestration state to the SQL Server so as to quickly and easily recover from any failures. Persistence points are those points in the orchestration that save the orchestration state to the database.

However, this behavior induces latency in the execution of the orchestration because of several database trips that are needed when executing the them. Although its quite difficult to avoid the persistence points when designing the orchestration, understanding them and using appropriate shapes and patterns can ensure that you keep them to minimum. For example, the orchestration persists its state after end of every transaction, at the end of orchestration, after every send shape etc. However, if the Send shapes are enclosed within atomic transactions, they do not persist the state. Instead, the state is then persisted at the end of the atomic transaction.

In a usual scenario, the persistence in the orchestrations occur at the following points -

• End of a transactional scope (atomic or long running)
• At the execution of other orchestrations through the Start Orchestration shape
• At the Send shape (except in an atomic transaction)
• At the end of the orchestration

Apart from that, the engine might decide to persist the orchestration in the following cases -

• When an Orchestration Instance is suspended
• When the system shutdowns in a controlled manner
• When the engine determines it wants to dehydrate
• When an orchestration instance is finished

Persistence can also occur at debugging breakpoints when the orchestration debugger is being run.

Lets just illustrate with a simple example how persistence points can increase the latency during the execution of the orchestration. 

Consider orchestration scenario that has 2 send shapes sending messages to two different systems.

This can be implemented in the following ways -

A. A parallel branch that has two Send shapes.

B. Two send shapes being executed sequentially in the orchestration.

C. Two send shapes being executed sequentially in the atomic shape.

Question: Now which of these do you think would work the fastest?
Of course the (B) scenario is out of question. The two Send shapes will have their persistence points. If the Send shape is the last shape in the orchestration, then the persistence point of the Send shape and the end of the orchestration will be batched together to result in a single persistence point instead of two.

With scenario (A) even we have the parallel branches in place, Biztalk by nature executes them in a sequential manner. Parallel branches do not necessarily mean they are executed on separate threads. Hence, even in this case, it results in sequential execution with two persistence points. Infact, in this case, the persistence point will not be batched with end of the orchestration and hence it results in the slowest of the three.

The scenario (C) takes the fastest position of the three. The send shapes are enclosed in the atomic shape and hence the state is persisted only at the end of the atomic shape. Again, if it is at the end of the orchestration, the persistence point will be batched with the persistence point at the end of the orchestration. Hence resulting in a single persistence point.

Hence, when designing the orchestration from the performance perspective, it is imperative to take a look at what points the orchestration will be persisting the data. Reducing the database trips helps increase the performance of the orchestration.