page contents

Benchmark your Dynamics AX solution with Load Testing


One of the most crucial time when implementing your Dynamics AX solution is the few weeks prior the Go Live. At that stage, the infrastructure has been configured with all components and all settings have been reviewed to match best practices. The functional team is running all test cases to sign off all design changes and the development team is fixing the remaining issues before Code Freeze.

It is the time when stakeholders ask you to validate the overall performance of your Dynamics AX solution. The primary goal is to make sure the solution will accept the future load and no major incident will occur. Load Testing is the right approach here to model the expected usage by simulating multiple users who access the application at the same time.

There are mainy objectives that can be achieved by running such Load Test, but here are the three major ones:

  1. Create a baseline of your core business scenarios
  2. Simulate load testing in term of concurrency and volume of transactions (see figure 1)
  3. Execute Performance Tuning and Optimization

Figure 1 shows the number of records created during one run versus the target. Here is a great way to calculate the table growth per company.

Once the goals have been defined and clear expectations validated, you should make sure the following pre requisites are met:

  • Identify the core business scenarios from production and purchase to sales order. It is crucial that these scenarios match the ‘Day in the Life’ usage to get relevant output.
  • Security of all user profiles is ready. The performance benchmark is not a good time to fix such issue and having all users as System Administrator is not recommended
  • Functional testing of the targeted scenarios is completed, and configuration of all components has been verified (SSRS, Enterprise Portal, Batch…). This is crucial for the efficiency of the benchmark because these core scenarios will be extensively repeated.
  • Code development is complete and Data is available for testing (Master data like customer account, product and stock availability or transactional data is required).

Please note that the scope of the load testing should be targeted. It is key to ensure the stability of the test cases and reach high sucessful test case execution. For example, Disaster Recovery and High Availability should not be tested during this load testing.

A positive side effect of such a project is to deploy all the tools necessary to troubleshoot performance issue after Go Live. During the performance benchmark, these tools will collect detailed trace and the support team will get comfortable on their analysis:

  • Windows Performance Monitoring (memory, processor, network, disk latency).
  • Performance Analyser for Dynamics (DynamicsPerf database for number sequences consumption, top tables by rows, query tuning analysis for missing indexes, index physical statistics).
  • SQL Trace for major SQL Events like Deadlock, Lock escalation, data file auto growth. Please note that Extended Events are now recommended from SQL Server 2012.
  • Dynamics AX Long Running Queries (for example above 1000 ms) to collect longest queries per user and search (with X++ call stack and SQL Statement information). See figure 2 for example.
  • Dynamics AX Client Access Log to collect objects usage for example Forms opening and Menu item clicked.

Figure 2 shows an example of long running queries collected during one run.


Note: Trace Parser is only recommended for creating and optimizing the baseline of each business scenarios. Trace Parser analysis should be run prior to the Load Testing because it adds overhead on the execution and will impact the results.

There are several tools to capture the scenario and run them automatically:

  • The Microsoft Dynamics AX 2012 Performance Benchmark Software Development Kit allows you to develop managed tests that simulate multiple-user activity. It leverages the Load Test functionality of Visual Studio® 2010 to run performance tests. Please read the White Paper for details on creating Test Data as well.
  • The Coded UI feature of Visual Studio allows you to capture the scenario directly on Dynamics AX rich client with the Test Recorder. The end to end tests must be straightforward without any custom control that requires X++ customization. You also need to plan for the repetition of the test case with mixed data input and error handling (see the figure 3 below)
  • If you plan to only test the Enterprise Portal, you should use the Visual Studio Web Performance tool to capture the HTTP traffic and simulate the work load on the SharePoint Server.

Figure 3 shows one example of degradation statistics between baseline run and one stress run.

Remember that Performance Testing is all about iterations to fine tune your scenario and secure the code optimization until you reach an acceptable threshold. Therefore, the automated framework will only be beneficial if you can repeat several run. In the figure 4 below,  we can see the load of concurrent users and number of transactions per scenarios during one run executed with Visual Studio Load Test, with a warm up of 10 minutes.

Note: you can also leverage the “AX CUIT Extension” available on CodePlex for Coded UI: https://axcuitextension.codeplex.com/

 

Regards,

@BertrandCaillet

Principal Premier Field Engineer

 

Comments (13)
  1. Please be aware of the latest R3 version of the tool now available on Lifecycle Services:

    http://www.microsoft.com/…/details.aspx

  2. Nishant says:

    Thanks for your all support. I emailed you my query.

    Regards

    Nishant

  3. Hi Nishant,

    I appreciate your interest in this topic. Microsoft Services is currently building a workshop to help Customer and Partner to work with the Performance Benchmark SDK and I will let you know once that is available.

    In the meantime, I I think we should keep this conversation offline, feel free to email me directly at bcaillet at microsoft dot com.

    Regards,

    Bertrand

  4. Michael says:

    Hi Michael,

    Thanks for your quick response.I registered for the webinar. As per client requirement, I need to simulate the 3,000 Users load and moreover stress testing is also in the scope. Yes you are absolutely correct that in case of Benchmark we need assistance of X++ developer.

    I have seen one more tool called AXeptance. How it is? any idea. And more importantly client is more focused on identifying the bottleneck of Infrastructure and AX applications.

    Could we plot the graph of response time graph of each form and other performance metrics like CPU, Memory, Disk etc.?

    Do we need to have Citrix farm or Microsoft terminal server to run the load test?

    Any idea about the licensing cost of this tool with 3000 Virtual users?

    Regards

    Nishant

  5. Michael Roedeske says:

    Dear Nishant,

    Executive Automats logs execution time of each action step (press a button, entry data, etc.), in regression testing mode you set a previous testrun as reference and as a result on every further regression testrun you get the comparison of the exection time and possible error situations.

    Compared to the Benchmark Toolkit you always have included the full GUI interaction, further you do not need a a developer. To record a sales order Scenario you possible need less than 5 minutes and another 10 minutes to parametrisize the script, thats it. You can join a free webinar, feel free to register at: http://www.dynamicsautomated.de/webinar.

    But depending on the amount of concurrent users you want to simulate, you need to dig into the Benchmark Toolkit. We did several years ago a test with 6000 concurrent users, with limited hardware the Benchmakr Toolkit is the best option.

    Regards,

    Michael

  6. Nishant says:

    Dear Bertand,

    I have gone through the performance Benchmark Toolkit. There is no doubt that this is the best tool for performance testing of  AX Rich Client application. But I am very new to this tool and don't have expertise on this tool. Do you have any document or sample code step by step for the execution of any customized scenario (example:- sales order , purchase order etc.) . If there is, please share with me (sinha.nish01@gmail.com)

    Regards

    Nishant

  7. Nishant says:

    Dear Michael,

    Does Executive Automats is a relevant tool for the performance testing of AX rich client application? If it is then could we extract or configure response time, hits per second etc graph? Please do the needful.

    Regards

    Nishant

  8. Hi Nishant,

    You can download the tool and the White Paper via Lifecycle Services or TechNet;

    technet.microsoft.com/…/dn248439.aspx

    Regards,

    Bertrand

  9. Michael Roedeske says:

    Dear Nishant,

    take a look on Executive Automats from XPLUS http://www.dynamicsautomated.com, this makes Dynamics AX function-, process-, daily regression- and further load- and stresstest so easy and simple compared to all other options that are available on the market.

    Regards,

    Michael

  10. Nishant says:

    Hi,

    I want to do the performance testing of Microsoft ax Dynamics 2012 based application. Please suggest me relevant tool for this. Is there any guide available for Microsoft benchmark toolkit, How to work on it?

    Thanks

    Nishant

  11. Hi Lisa,

    The typical example is the sales order creation with the header and 2 lines. You can test the Baseline with only one user and analyze performance with Trace Parser (Client and Server calls). Then, after the optimization is completed, you are ready to test the concurrency model with several users creating sales order (with a mix of Customer and products of course).

    Hope that makes sense, let me know.

    Regards,

    Bertrand

  12. Lisa says:

    What would a sample baseline be for a simple versus complex scenario, I know this depends on what you're trying to do exactly but I'm just looking for a best practice baseline.

    Thanks.

Comments are closed.

Skip to main content