Understanding Persistence Points in Biztalk Orchestration


Often ignored, but a really important point when designing orchestrations is of course the persistence points. Understanding of the persistence points becomes really an important factor if you are dealing with orchestration performance. Lets take a glance at what they are and how we can take care of them.  

The SQL Server does bulk of the job at the backend to keep the BizTalk server running.  BizTalk has this inherent behavior of saving the orchestration state to the SQL Server so as to quickly and easily recover from any failures. Persistence points are those points in the orchestration that save the orchestration state to the database.

However, this behavior induces latency in the execution of the orchestration because of several database trips that are needed when executing the them. Although its quite difficult to avoid the persistence points when designing the orchestration, understanding them and using appropriate shapes and patterns can ensure that you keep them to minimum. For example, the orchestration persists its state after end of every transaction, at the end of orchestration, after every send shape etc. However, if the Send shapes are enclosed within atomic transactions, they do not persist the state. Instead, the state is then persisted at the end of the atomic transaction.

In a usual scenario, the persistence in the orchestrations occur at the following points -

  • End of a transactional scope (atomic or long running)
  • At the execution of other orchestrations through the Start Orchestration shape
  • At the Send shape (except in an atomic transaction)
  • At the end of the orchestration

Apart from that, the engine might decide to persist the orchestration in the following cases -

  • When an Orchestration Instance is suspended
  • When the system shutdowns in a controlled manner
  • When the engine determines it wants to dehydrate
  • When an orchestration instance is finished

Persistence can also occur at debugging breakpoints when the orchestration debugger is being run.

Lets just illustrate with a simple example how persistence points can increase the latency during the execution of the orchestration. 

Consider orchestration scenario that has 2 send shapes sending messages to two different systems.

This can be implemented in the following ways -

A. A parallel branch that has two Send shapes.

B. Two send shapes being executed sequentially in the orchestration.

C. Two send shapes being executed sequentially in the atomic shape.

Question: Now which of these do you think would work the fastest?
Of course the (B) scenario is out of question. The two Send shapes will have their persistence points. If the Send shape is the last shape in the orchestration, then the persistence point of the Send shape and the end of the orchestration will be batched together to result in a single persistence point instead of two.

With scenario (A) even we have the parallel branches in place, Biztalk by nature executes them in a sequential manner. Parallel branches do not necessarily mean they are executed on separate threads. Hence, even in this case, it results in sequential execution with two persistence points. Infact, in this case, the persistence point will not be batched with end of the orchestration and hence it results in the slowest of the three.

The scenario (C) takes the fastest position of the three. The send shapes are enclosed in the atomic shape and hence the state is persisted only at the end of the atomic shape. Again, if it is at the end of the orchestration, the persistence point will be batched with the persistence point at the end of the orchestration. Hence resulting in a single persistence point.

Hence, when designing the orchestration from the performance perspective, it is imperative to take a look at what points the orchestration will be persisting the data. Reducing the database trips helps increase the performance of the orchestration.

Comments (16)

  1. Oly says:

    Excellent explanation!

  2. Pam says:

    Thanks for the explanation!  Very well stated.

  3. magda says:

    I finally got this! Thanks for this great post.

  4. Poc says:

    explanation is so clear, thank you for this great lesson.

  5. JBA says:

    thanks a lot !! this is something which is not visible to the developer. but after reading, it is very much visible.

  6. Jitender Chauhan says:

    Thanks a Lot for this excellent explanation.

  7. ¿How can I avoid any persistence in may orquestrations?

    I have this behaviour in my deployed solution:

    Sometimes, the sql server machine has "memory problems" like this:

    The following stored procedure call failed: " { call [dbo].[bts_FindSubscriptions]( ?)}". SQL Server returned error string: "Could not perform the requested operation because the minimum query memory is not available. Decrease the configured value for the ‘min memory per query’ server configuration option.".

    And , if an orquestration is running, in that moment, the engine repeats all the process, so I have other problems due to this behaviour, I should prefer that the process would terminate…

    Is that possible?

    Thanks in advance!

    Ana

  8. Jas says:

    I don’t think you can completely remove persistence from your orchestration b/c your orchestration needs to persist its state to Messagebox. However you can try to reduce the # of persistenace points as stated in this post. The problems with your SQL server might be due to hardware or other issues. How much memory do you have on the box? What else is running on the box besides SQL server? How many databases it holds? etc

  9. bhavik says:

    I am new in BizTalk. and not able to Understand what is Persistence Point. But your Article have given clear Understanding. Thanks man.

    I want to give little bit light no Understanding Dehydration and Persistence Points.

    Dehydration is one of the most misunderstood orchestration topics and, ultimately, it’s not that hard to comprehend.

    Imagine that your BizTalk Server is processing 1,000 messages a minute. Not only is it receiving and mapping, it is also sending those messages to an external application process that will approve or deny them based on business criteria. However, that process could possibly

    be a time-consuming one. Should you ask BizTalk to simply wait, with all of those orchestration instances idling and chewing up your valuable resources? Certainly not! Instead of just waiting, the orchestration will dehydrate, and the state of the instance is taken out of memory and stored in SQL Server. BizTalk does this to free up valuable resources for other processes. A special subscription will be created for the dehydrated instance of the orchestration. This subscription will monitor the BizTalk message box for context properties that match your unique correlation ID. When the subscription comes across that message, the orchestration will “wake up,” and the state of the instance will be taken out of the database

    and reconstituted in memory. The point at which the orchestration will, essentially, come alive is known as a persistence point.

    Persistence points are created throughout the orchestration and provide a method of bookmarking activities, so that when the orchestration is revisited (for correlation, for example), BizTalk will pick up right at the last persistence point. A send port is automatically a persistence point, and when you think of it in terms of correlation, that makes perfect sense. These restore points also provide an avenue of recovery for orchestrations in the event that the service or hosting computer suffers a catastrophic failure.

  10. Sanket B says:

    Bhavik,

    Thanks a lot for your comments and a really good description for dehydration.

    –Sanket

  11. Mit says:

    Well explained and expect more on near future

  12. Abhishek says:

    The explanation Sanket gave and the one by Bhavik are somewhat contradictory. According to Sanket, the point at which the orchestration state is saved to the database is Persistence point and going by Bhavik's waty, the point at which the data is taken out of the database is persistence point.

    So which of the two is the right one?

  13. Sanket B says:

    Abhishek, As I see it, both of us are saying the same thing – at the persistence point, the state is saved (persisted) in the database. When the orchestration needs to be reconstituted in the memory, thats when it starts again from the last persistence point that was saved from the database. Hope this makes it clear.

  14. Anand S B says:

    Thanks a lot for this clear cut explanation ! It helped me a lot

  15. Priya says:

    Thanks for this. Cleared a lot of doubts.

Skip to main content