One of the most crucial time when implementing your Dynamics AX solution is the few weeks prior the Go Live. At that stage, the infrastructure has been configured with all components and all settings have been reviewed to match best practices. The functional team is running all test cases to sign off all design changes and the development team is fixing the remaining issues before Code Freeze.
It is the time when stakeholders ask you to validate the overall performance of your Dynamics AX solution. The primary goal is to make sure the solution will accept the future load and no major incident will occur. Load Testing is the right approach here to model the expected usage by simulating multiple users who access the application at the same time.
There are mainy objectives that can be achieved by running such Load Test, but here are the three major ones:
- Create a baseline of your core business scenarios
- Simulate load testing in term of concurrency and volume of transactions (see figure 1)
- Execute Performance Tuning and Optimization
Figure 1 shows the number of records created during one run versus the target. Here is a great way to calculate the table growth per company.
Once the goals have been defined and clear expectations validated, you should make sure the following pre requisites are met:
- Identify the core business scenarios from production and purchase to sales order. It is crucial that these scenarios match the ‘Day in the Life’ usage to get relevant output.
- Security of all user profiles is ready. The performance benchmark is not a good time to fix such issue and having all users as System Administrator is not recommended
- Functional testing of the targeted scenarios is completed, and configuration of all components has been verified (SSRS, Enterprise Portal, Batch…). This is crucial for the efficiency of the benchmark because these core scenarios will be extensively repeated.
- Code development is complete and Data is available for testing (Master data like customer account, product and stock availability or transactional data is required).
Please note that the scope of the load testing should be targeted. It is key to ensure the stability of the test cases and reach high sucessful test case execution. For example, Disaster Recovery and High Availability should not be tested during this load testing.
A positive side effect of such a project is to deploy all the tools necessary to troubleshoot performance issue after Go Live. During the performance benchmark, these tools will collect detailed trace and the support team will get comfortable on their analysis:
- Windows Performance Monitoring (memory, processor, network, disk latency).
- Performance Analyser for Dynamics (DynamicsPerf database for number sequences consumption, top tables by rows, query tuning analysis for missing indexes, index physical statistics).
- SQL Trace for major SQL Events like Deadlock, Lock escalation, data file auto growth. Please note that Extended Events are now recommended from SQL Server 2012.
- Dynamics AX Long Running Queries (for example above 1000 ms) to collect longest queries per user and search (with X++ call stack and SQL Statement information). See figure 2 for example.
- Dynamics AX Client Access Log to collect objects usage for example Forms opening and Menu item clicked.
Figure 2 shows an example of long running queries collected during one run.
Note: Trace Parser is only recommended for creating and optimizing the baseline of each business scenarios. Trace Parser analysis should be run prior to the Load Testing because it adds overhead on the execution and will impact the results.
There are several tools to capture the scenario and run them automatically:
- The Microsoft Dynamics AX 2012 Performance Benchmark Software Development Kit allows you to develop managed tests that simulate multiple-user activity. It leverages the Load Test functionality of Visual Studio® 2010 to run performance tests. Please read the White Paper for details on creating Test Data as well.
- The Coded UI feature of Visual Studio allows you to capture the scenario directly on Dynamics AX rich client with the Test Recorder. The end to end tests must be straightforward without any custom control that requires X++ customization. You also need to plan for the repetition of the test case with mixed data input and error handling (see the figure 3 below)
- If you plan to only test the Enterprise Portal, you should use the Visual Studio Web Performance tool to capture the HTTP traffic and simulate the work load on the SharePoint Server.
Figure 3 shows one example of degradation statistics between baseline run and one stress run.
Remember that Performance Testing is all about iterations to fine tune your scenario and secure the code optimization until you reach an acceptable threshold. Therefore, the automated framework will only be beneficial if you can repeat several run. In the figure 4 below, we can see the load of concurrent users and number of transactions per scenarios during one run executed with Visual Studio Load Test, with a warm up of 10 minutes.
Note: you can also leverage the "AX CUIT Extension" available on CodePlex for Coded UI: https://axcuitextension.codeplex.com/
Principal Premier Field Engineer