BizTalk and Hyper-Threaded Processors

This great new article from the BizTalk Performance Blog has reminded me of a potentially nasty issue when you have Hyper-Threading enabled processors on your BizTalk box.

BizTalk comes out of the box with some default performance tuning settings which seem to work well on the whole, you can see these in the adm_ServiceClass table of the BizTalkMgmtDb, you might need to adjust these during your performance tuning – the normal disclaimers apply! They are all documented in the Performance Characteristics paper

I’ve pulled out some of the key ones that I tend to adjust during scale and perf exercises:

Low and High Watermarks specify how many instances of that “service type” can execute at any one time, so for XLANG/S it refers to Orchestrations. The High Watermark being the maximum number of Orchestrations that can be in flight at any one time – once the BizTalk engine hits this watermark it will throttle back to the number specified by the Low Watermark and then resume normal operation – you’ll see this throttling logged in the Event Viewer.

Low and High Sessionmarks specify the number of concurrent database sessions that can be persisting inbound messages at any one time for each “service type”.

The key thing to remember is that all of these settings are per processor, so if you have a dual processor box then 40 Orchestrations can be in flight at any time, quad processor boxes can have 80 and so on. BizTalk however isn’t “Hyperthreading aware” so it will treat a Hyperthreaded Processor as a physical processor despite it not giving dual processor like performance (we tend to use 25% as a rule of thumb increase – although this will depend on your scenario).

Therefore you may want to think about scaling these numbers back to reflect this, you may not have to of course and this is an exercise to perform during your scale and perf exercise and not in live! Please don’t go changing these just in case J

The scenario where we successfully did this was where we saw high context switching on the BizTalk Server boxes which can be an indication of too many threads fighting for time on the processor causing everything to take longer than usual to execute, the typical solution being to reduce the number of threads. 

On our Hyper-Threaded boxes we reduced the number of threads to cater for the ~20% improvement (rather than 100%) and we saw a positive improvement as less threads were fighting over the processor leading to improved throughput.

Something to watch out for, but again it won’t necessarily help in all scenarios – be careful!