The Summary is the most important section of Apptim reports which contains data on each performance KPI measured by Apptim. The most relevant KPIs are shown on the top, including Average CPU Usage, Average Memory Usage and Average Power Usage, Maximum Data Downloaded and Data Uploaded and any Errors/Exceptions captured during the test session.
The Summary table contains a list of each metric evaluated against default thresholds set by Apptim, which are based on Google’s best practices (Android) and Apple’s best practices (iOS), along with other market benchmarks that consider device fragmentation.
Each threshold is specific for each metric and contains two limits: a Moderate Limit and a Warning Limit. These limits define the color shown: yellow if a Moderate Limit is reached, and red if a Warning Limit is reached. You can check the limits for each metric here.
How to use these thresholds/limits
When a Warning Limit has been reached in some metric listed here and it seems a bit high, the user can then click on the specific graph related to that metric or navigate to it in the left panel to understand:
- At what moment of the test that value was reached
- How the app UI was at that moment, looking also at the UI seconds before and after in the video
Also, it is important to cross-reference the information with device-specific behavior shown inside the "Environment” section.
The Session info contains details about the test session including the Test name, duration, start and end time of the tests, and the device used (model, OS).
Video and main performance KPI Graphs
When scrolling down, users will find the recorded Test Session Video and performance KPI graphs for CPU, Memory, Threads, Network, Render and Power Usage, and a table with Database top SQLs times.
Test Session Video
This video is generated at the end of the test session and it is accelerated for easier navigation. The real timestamp during the test is shown on the top left side of the video.
Navigation Panel (left)
Besides the summary, the report contains a navigation panel on the left with the following content:
- Environment: General information about the test environment: device information such as OS version, screen size, and hardware details. It also contains app-related information such as: version, compatibility, permissions requested, activities and libraries used*.
- Correctness*: Contains information and checks related to the app's activities, UI controls and layout drawing.
- Errors: Contains crash information and ANR exceptions (when the app is not responding).
- Bug Reports: This section is visible only when the user has reported a bug during the test session. It contains the time stamp when the bug occurred, a screenshot and a description of the issue with steps to reproduce it (provided by the tester).
- Logs: Contains all the device logs and downloadable artifacts collected during the test.
* Available for Android only