|Distributed architecture can mercilessly backfire at you. In my case flexible architecture, elegant design patterns, and smart code led to abuse of the flexibility, resulting in very poor performance. Free Microsoft Network Monitor (Netmon) helped to identify the root cause of the||
|performance hit. It showed that over-distributed-ness can cost you in terms of performance.
Customer Case Study
The customer complained about poor response times in his web application. The application's architecture was similar to the Web Application Archetype. Notice Services Agent box that connects your application to downstream services? Our assumption was that the services agents are too chatty causing the performance hit. Netmon only made it clear.
We took captures on the application server where the Server Agents are to identify what other downstream servers are accessed. In no time we get very clear picture - the application server was accessing other three downstream resources:
There are two well known ports - 443 and 1433 - so we could safely assume there is communication over SSL/HTTPS and SQL Server. The other one - 1414 - turned out to be MQ.
Next step was to identify which one of the conversations is causing us troubles the most - either by chatty communication or by just long running transaction.
Looking at Time Delta column for HTTPS stateless communications we found nothing exciting regarding the latency:
For MQ communications we used magic ContainsBin(FrameData, 0,"StringToFindGoesHere") to identify XML messages going back and forth over MQ transport. For example, to find the beginning of the XML message we used the following filter:
Similar technique was used to correlate request and response XML messages.
To identify SQL communication we used shiny new TDS parser available for free download on Codeplex - SQL Parser in Latest CodePlex Package. I particularly like this one, it shows SQL Server communication without using SQL Server Profiler:
This post is made with PracticeThis.com plugin for Windows Live Writer