Declare the default protocol (HTTPS) and server name (F&O Sandbox environment) in the HTTP Request Defaults. This settings value based on User Defined Variables for all HTTPS requests within the Test plan.
- Protocol [https]: ${scheme}
- Server Name or IP: ${host}
bzm - Correlation Recorder
As previously mentioned, we will be utilizing the Correlation Recorder to record scenarios in Dynamics 365 finance and operations apps. Proper configuration in the correlation recorder is crucial for accurate recordings. The recorder acts as an HTTP(S) proxy server, capturing the interactions between JMeter and the application.
-
• Grouping: leave default Do not group samples
-
• Capture HTTP Headers: Checked
-
• Regex Matching: Checked
-
• Type: Java
-
• Use Keep-Alive: Checked
-
• Follow Redirects: Unchecked
Download the filters template DefaultRequestFilters.txt from here. Copy all filter rules from the text file (Ctrl + C). In JMeter, go to the filters section, and click on Add from Clipboard to paste the filter rules. This template helps exclude unnecessary URLs, optimizing the recording process.
We have prepared a template of correlation rules, these are the rules that will determine what information will be extracted and replaced from each response and request message when recording scenarios in Dynamics 365 finance and operations apps.
-
1. Download the correlation rules template file (D365Login_correlation_template-1.0.2.json) from here.
-
2. Move the downloaded template to the correlation-templates folder within the JMeter directory \apache-jmeter\correlation-templates\.
-
3. In the JMeter, go to Correlation Recorder, access the correlation rules and click on Load template.
-
4. In the Templates Manager, select the Available tab, choose the downloaded D365Login_correlation_template-1.0.2.json, and click Install. (Restart JMeter if you cannot find the downloaded template).
-
-
Noted: If you see a blank page in the Installed or Available tab, and your JDK version is higher than 13, you may need to uninstall it and install JDK 13 instead. Newer JDK versions may affect the Templates Manager.
5. In the Templates Manager, go to the Installed tab, select D365Login_correlation_template-1.0.2.json, and click Load.
6. All correlation rules have been added from the template and were grouped under Group-1.
Noted: Please enable the "Enable Correlation (Legacy)" option in the latest version (e.g. V2.1) to achieve a similar experience to the older version of Correlation Recorder.
View Result Tree Configuration
Under the bzm - Correlation Recorder section, set the desired file path for the results location in the View Result Tree \bin\FnO_Sample\CreatePOAndSubmitWorkFlow.xml.
Click on button Configure next to the Filename and ensure that all checkboxes are selected.
Similarly, under the Test Plan, set the desired file path for the results location in the View Result Tree \bin\FnO_Sample\CreatePOAndSubmitWorkFlow_Run.xml.
Starting the Tasks Recording
Before starting the scenario recording, it is crucial to ensure that the setup and configuration in the above sections are completed. Furthermore, it is essential to confirm which specific scenarios need to be captured. To achieve this, it is highly recommended to collaborate with your business team to define the end-to-end scenarios.
In this blog post, we will showcase a simple scenario: “Create purchase order and submit purchase order to approval workflow”, using Contoso data. The scenario has the following flow:
-
• Go to Procurement and sourcing module then Create a Purchase Order with 2 items.
-
• Save the Purchase Order and submit to workflow.
-
• Sign out from the application.
-
1. Go to the bzm – Correlation Recorder section. Click on the Start button to initiate the scenario recording process.
-
2. When you come across the Recorder: Transactions Control prompt, you are clear to proceed with walking through the designated scenarios.
-
3. Let's get started by inputting the transaction name as LoginD365. After that, proceed to access the URL: https://<environment_name>.sandbox.operations.dynamics.com/. Following this, provide your email address and password on the credential page.
4. Once you've successfully obtained access to the Dashboard, designate the transaction name as GoToPurchaseOrderListPage. Then, go to Procurement and Sourcing > Purchase orders > All purchase orders.
5. Input the transaction name as CreateNewPO and proceed by clicking the New button to generate a new Purchase Order.
6. Input 1001 in Vendor account field, leaving all other settings as their default values. Subsequently, click the OK button to continue.
7. Enter FirstLineDetail as the transaction name. Then, input the subsequent details into the line:
- Product number: 1000 (Surface Pro 128 GB)
- Site: 1
- Warehouse: 11
- Quantity: 20
8. Enter AddSecondLine as the transaction name. Then click the button Add line to add a new line below first line.
-
9. Enter SecondLineDetail as the transaction name. Then input the subsequent details into the line:
- Product number: A0001 (HDMI 6' Cables)
- Site: 1
- Warehouse: 11
- Quantity: 10
10. Enter the transaction name as SavePO and subsequently, click the Save button to ensure that the Purchase Order is successfully saved.
11. Input the transaction name as SubmitWorkFlow and go to the Workflow section. Proceed by selecting the Submit button, and within the prompted dialogue box, input the comment “Please help to approve my purchase order by JMeter”. Finally, click the Submit button to initiate the submission of the purchase order workflow.
12. Confirmed the PO has been in review state, then Input the transaction name as SignOut and proceed to perform the sign out action.
13. Once you have successfully signed out, click the Stop button.
Upon finishing the recording process, it is important to ensure that you save the file. Once the recording is concluded, you can the opportunity to review the recorded samples within the Recording Controller.
After recording the scenario in JMeter, there are several essential tasks that should be carried out.
Correlation using PowerShell script for Dynamics 365 finance and operations transactions
Correlation involves capturing and preserving dynamic responses from the server and effectively passing them to subsequent requests. Recognizing the limitations of the bzm-correlation recorder in constructing correlation rules, we have developed a dedicated PowerShell script tailored for automatic correlation of control IDs. These control IDs are encountered in requests other than the login operation and are distinctive for each session.
To achieve automated correlation, follow these steps:
-
1. Exit the JMeter before proceeding with the following steps. This prevents overwriting the test plan after executing the Correlation matching.
-
2. Begin by downloading the “CorrelationScript.ps1” from here and save it within the JMeter directory's \bin folder.
-
2. Open the automatic correlation script, “CorrelationScript.ps1,” using PowerShell.
-
3. Utilize the Browse button to select the script file path, as well as the path of the test script (.jmx).
-
4. Similarly, click the Browse button to specify the result file, which is the .xml (Noted: .jtl if you are using higher version e.g. V2.1 of correlation recorder) located within the view result tree under the bzm-correlation recorder.
5. Click the OK button. The auto-correlation script will initiate a scan of both the .jmx file and the .xml result file. Please refrain from closing the PowerShell window until the script's execution is complete.
-
6. Once the previous step is completed, then open JMeter again.
-
• Check for any correlation failures, if found, assign parameters manually.
-
• Review and identify any invalid Requests that need to be disabled.
-
• Add Think time intervals within the script if necessary.
-
• Organize Requests by grouping them according to transaction names or types, and then add these groups to the Transaction Controller. This approach simplifies recording management and enhances the ability to reuse the test plan for different scenarios.
Execution of Single-User Tests (GUI Mode execution)
Once you've completed the JMeter script and provided all necessary parameters, it's time to verify the validity of the recorded test. This verification step is important to ensure the test has the necessary elements to run or to identify if any further adjustments are required. Alternatively, you can proceed with a run-through of the process to confirm its functionality.
It's worth noting that performing multi-user testing in GUI mode is not recommended. GUI mode is primarily used for scenario recording and validation purposes. In this context, it's important to confirm that the Thread Group is set up correctly.
Go to Test Plan > Thread Group and ensure that Number of Threads, Ramp-up Period, and Loop Count are all set to 1. This ensures an accurate representation of a single-user scenario. For more detailed information on each parameter refer to this link.
Initiate the verification of the test script by clicking on the Start button. To obtain an overview of the execution, refer to the Summary Report. For more detailed insights into each transaction, especially when debugging is necessary, consult the View Result Tree section.
Execution of Multi-User Tests
Within JMeter, the execution of multi-user testing can be initiated through either the GUI or command mode. The recommended approach is to utilize the command mode, primarily due to its advantages such as efficient resource utilization, separation of controllers and agents, and lower resource overhead.
Prior to commencing multi-user testing, it's imperative to include test users within both the target environment and Azure Active Directory (AAD). This inclusion is essential to simulate concurrent user activity. Before proceeding with the import of users into Dynamics 365 finance and operations apps, ensure the testing user accounts are created within Azure Active Directory. Please make sure to collaborate with an Azure Administrator to gain a clear understanding of any potential additional charges.
-
1. Go to Azure Active Directory, then proceed to All users. From there, click on Bulk operations, followed by selecting Bulk create. Download the CSV template provided.
-
2. Utilize an editor to open the downloaded CSV template. Input the necessary information into the designated columns.
-
3. Upload the completed CSV template and click the Submit button. This action will lead to the creation of testing users in Azure Active Directory upon successful validation.
-
4. After successfully creating the testing users in AAD, move on to Dynamics 365 finance and operations apps. Go to System administration > Users. Click on Import Users.
-
5. Choose the previously created testing users from the preceding steps and initiate the user import process.
-
6. Assign the Purchasing Agent role to all the created testing users.
-
1. Create a CSV file (the headless comma separated list of userEmail, password) duplicating the authentication credentials that were established in Azure Active Directory (AAD).
-
2. To create a list of testing users for testing, move to JMeter and Right-click on the Test plan, Add > Config element > CSV Data Set Config.
-
3. Outline the particulars as follows:
- Filename: Indicate the path to the CSV file containing the credentials
- Variable Names: userEmail, password
-
4. Go to Thread Group and expand group “1. Access to finance and operations apps”
-
5. Identify the request responsible for transmitting the credentials to login.microsoftonline.com/tenantId/Login.
-
6. Subsequently, using the CSV Data Set Config variables, substitute the occurrences of login, loginfmt and password in the above-mentioned request.
In our defined scenarios, the performance objective requires utilizing 10 test users to individually initiate the creation of Purchase Orders and subsequently submit them through to workflow. The target response time for each of actions is set at under 2 seconds.
-
1. To configure this, Go to Test Plan and access the Thread Group. Make sure to set the Number of Threads parameter to 10, and retain all default settings for the other relevant parameters.
2. Deactivate all listeners and non-test elements, retaining only the executable transaction script within the Test Plan.
-
3. Indicate the host name, save the project, and subsequently close JMeter.
-
4. Launch Command Prompt with Administrator privileges.
-
5. Navigate to the \bin directory where jmeter.bat execution application stored.
-
6. Execute the following command, referring to additional information here for command parameters:
-
Jmeter.bat -n -t “C:\apache-jmeter\bin\FnO_Sample\CreatePOAndSubmitWorkFlow.jmx” -l “C:\apache-jmeter\bin\FnO_Sample\Result\Results.csv” -e -o “C:\apache-jmeter\bin\FnO_Sample\Result\Report”
-
7. A successful execution of the command will be indicated as follows:
8. The successful test outcome is represented as follows: all Purchase Orders have been generated and submitted to the workflow successfully, and they are presently awaiting approval.
9. Verify the workflow history, you can access detailed information within it. This includes the workflow name, version, the individual who submitted it, as well as any accompanying comments.
Since the parameter -o is utilized, an analysis report will be generated automatically upon the completion of testing.
To access this report, go to the directory located at \bin\FnO_Sample\Result\Report\. Inside this folder, you will find the dashboard containing the relevant information.
Double click on index.html to open it. There is performance information provided in the report. The JMeter HTML report serves as a comprehensive summary of performance test results, presenting essential metrics such as response times, throughput, and error rates in a visually accessible format. This report aids in analyzing the performance of the tested application, identifying bottlenecks, validating performance goals, and making informed decisions for optimization and further testing efforts.
In the Response Time overview, we can observe most response times where less than 1,500ms which aligns with our performance goal (less than 2 seconds). A more in-depth analysis of the results will be available in Part 3 – Dynamics 365 Finance and Operations Apps Performance Testing with JMeter – Result Analysis.
Azure load testing is a load testing service designed to facilitate the generation of extensive load for your applications. This service adeptly simulates traffic, irrespective of the hosting location of your applications. For further details, you can access the comprehensive documentation available here.
In our defined scenarios, the performance objective requires utilizing 100 test users to individually initiate the creation of Purchase Orders and subsequently submit them through the workflow. The target response time for each of actions is set at under 2 seconds.
-
1. Begin by establishing the Azure Load Testing resource. Detailed guidance can be found here. Go to Azure portal > Azure Load Testing, create new Azure Load Testing resource, Instance name is D365FOPerfTests
-
2. Navigate to the newly created Load Testing resource D365FOPerfTests. Access the Tests section and initiate the creation process. Choose to upload a JMeter script to shape the test. Comprehensive steps can be referred to here.
-
3. Fill in the essential information on the Basics tab, including the Test name and Test description. For instance, "Create PO and Submit PO Workflow".
-
4. In the Test plan tab, upload the script files, namely CreatePOAndSubmitWorkFlow.jmx and Login.csv. Successful validation will be indicated by a "Completed" status.
5. While optional, parameterizing variables is advantageous for executing the JMeter .jmx file across different environments. In JMeter, access User Defined Variables to incorporate environment-specific parameters below, then save it. Find detailed steps here.
6. With the test configured, initiate the execution by clicking Run. Offer a concise description for this test run, such as "Create PO and submit Workflow with 100 users."
7. A successful execution of the Azure Load Testing will be indicated by specific information shown below. In this case, the test concluded in approximately 4 minutes, generating a total of 10,900 requests. The average response time remained under 1 second, aligning with our performance target.
8. The successful test outcome is represented as follows. A total of 100 Purchase Orders were generated and successfully submitted to the PO Workflow without errors, currently awaiting approval. The result highlights an average rate of around 7 orders per second.
In Part 2 of this blog series, we explored how Apache JMeter is proficient in performing performance tests for Dynamics 365 finance and operations apps. By using JMeter and Azure Load Testing along with generated test script, you can conduct both single-user and multi-user tests with ease.
Sample script can be downloaded through link:
• Create Purchase Order and confirm PO. (Pending to be released)
• Create Purchase Order and submit for workflow approval.
• Create a Sales Order and confirmation.
