Android App Performance Test Scenarios, Page 2
To set up representative Android app performance testing scenarios, follow the instructions for each of the following settings.
- Script the necessary flows for different workload types using the load generation tool for the server.
- Thin client — Install the hybrid application in your application server. Record the necessary flows using JMeter.
- Thick client — Install the hybrid
apk
file in the emulator. Record the necessary flows using JMeter.
- Load the server with the different workload scenarios:
- 100% from the thin client
- 100% from the thick client
- Combination of both thick and thin client (For example, 40% load from the thin client and 60% load from the thick client)
- Identify the different network profiles for which you will test. Set up the network emulation tool for the following profiles.
- Airtel* Average 3G
- Airtel* Best 3G
- Airtel* Worst 3G
- Airtel* Average 2G
- Airtel* Best 2G
- Airtel* Worst 2G
Airtel is an Indian telecom operator. Note that the speed emulated for the profiles do not represent the actual values, but they should during actual tests. - Install the Android hybrid apk in both the high-end and low-end devices.
- High-end: Samsung Galaxy S
- Low-end: Samsung Galaxy Y
Workload Type
Workload Mix
Network types
Device types
During your actual performance tests, be sure to recreate all the scenarios applicable to the test application's context.
Android App Performance Test Execution
Here are the steps for actually executing the Android app performance test.
- Load the server: The server was loaded with the scenarios identified in the previous section using JMeter. The server was monitored for performance while loaded.
- Select a network type and a device: With the server loaded, the identified network profiles were recreated using the network emulator tool. In the tool, we edited the bandwidth, jitter, packet drop parameters based on our requirements such as 3G or 2G. The device was configured to use the emulated network. The POC MAC address of the device and the server were configured in the network emulator tool to filter the relevant information.
- Access the application on a real device: The hybrid application was installed on the device to be tested. The Android command for installing the application (
apk
) is:
Once installed, the transaction was carried out on the real device with server and network conditions emulated. In addition, the transaction was also carried out on the on-device browser.adb install <apk>
- Monitor the components involved: The components were monitored during the transaction execution on the device.
These steps were repeated to cover all the test scenarios mentioned earlier. The following data was captured on the device using adb
.
Server | Emulator | Device | |
---|---|---|---|
CPU % |
√
|
√
|
√
|
Memory Info |
√
|
√
|
√
|
Processor Info |
√
|
√
|
√
|
Network Info |
NA
|
√
|
√
|
Response Time |
NA
|
√
|
√
|
Android App Performance Test Results
Here are a few sample results that can be obtained by testing. Note that these results are only the tip of the iceberg of what you can do with end-to-end performance testing.
Device Performance
The graph below is the device result which is monitored by the adb
commands. We observed no huge difference in PSS, memory allocated, heap size or used memory with the increase in load.
Click here for larger image
Figure 3. Device Performance Results
Thick and Thin Client Performance Comparison
The graph below shows the difference between thick client and thin client. This allows us to understand how the rendering of responses affects performance from the end user's point of view.
Click here for larger image
Figure 4. Thick and Thin Client Performance Results
Network Performance Comparison
The graph below presents the network speed for 3G and 2G networks. Load is generated via the load runner tool. We observed that the 2G response time rose higher than that of the 3G as we increased the load.
Click here for larger image
Figure 5. Comparison of Network Performance Results
As you can see, extensive analysis can be performed with the end-to-end data available for the different components involved in mobile application delivery. Such analysis would not be possible if the device were tested in isolation with no control over server/network conditions.
Conclusion
This article presented a proof of concept for end-to-end mobile application performance testing using a sample Android app. It detailed every component to be tested, the method for testing as well as the challenges involved.
When adopting this approach, be sure your Android application performance tests are under these conditions:
- On multiple devices
- With native applications as well as browser-based applications
- Under production like delivery mode-based workload mix on server
- Under production like geography-based workload mix on server
- Under varying network profiles
- Under varying device states
Originally published on https://www.developer.com.
Page 2 of 2
This article was originally published on June 1, 2012