Android App Performance Test Scenarios, Page 2

June 1, 2012
By Developer.com Staff
Send Email »

To set up representative Android app performance testing scenarios, follow the instructions for each of the following settings.

Workload Type

Script the necessary flows for different workload types using the load generation tool for the server.
- Thin client — Install the hybrid application in your application server. Record the necessary flows using JMeter.
- Thick client — Install the hybrid apk file in the emulator. Record the necessary flows using JMeter.

Workload Mix

Load the server with the different workload scenarios:
- 100% from the thin client
- 100% from the thick client
- Combination of both thick and thin client (For example, 40% load from the thin client and 60% load from the thick client)

Network types

Identify the different network profiles for which you will test. Set up the network emulation tool for the following profiles.
- Airtel* Average 3G
- Airtel* Best 3G
- Airtel* Worst 3G
- Airtel* Average 2G
- Airtel* Best 2G
- Airtel* Worst 2G
Airtel is an Indian telecom operator. Note that the speed emulated for the profiles do not represent the actual values, but they should during actual tests.

Device types

Install the Android hybrid apk in both the high-end and low-end devices.
- High-end: Samsung Galaxy S
- Low-end: Samsung Galaxy Y

During your actual performance tests, be sure to recreate all the scenarios applicable to the test application's context.

Android App Performance Test Execution

Here are the steps for actually executing the Android app performance test.

Load the server: The server was loaded with the scenarios identified in the previous section using JMeter. The server was monitored for performance while loaded.
Select a network type and a device: With the server loaded, the identified network profiles were recreated using the network emulator tool. In the tool, we edited the bandwidth, jitter, packet drop parameters based on our requirements such as 3G or 2G. The device was configured to use the emulated network. The POC MAC address of the device and the server were configured in the network emulator tool to filter the relevant information.
Access the application on a real device: The hybrid application was installed on the device to be tested. The Android command for installing the application (apk) is:
```
adb install <apk>
```
Once installed, the transaction was carried out on the real device with server and network conditions emulated. In addition, the transaction was also carried out on the on-device browser.
Monitor the components involved: The components were monitored during the transaction execution on the device.

These steps were repeated to cover all the test scenarios mentioned earlier. The following data was captured on the device using adb.

	Server	Emulator	Device
CPU %	√	√	√
Memory Info	√	√	√
Processor Info	√	√	√
Network Info	NA	√	√
Response Time	NA	√	√

Android App Performance Test Results

Here are a few sample results that can be obtained by testing. Note that these results are only the tip of the iceberg of what you can do with end-to-end performance testing.

Device Performance

The graph below is the device result which is monitored by the adb commands. We observed no huge difference in PSS, memory allocated, heap size or used memory with the increase in load.

Click here for larger image
Figure 3. Device Performance Results

Thick and Thin Client Performance Comparison

The graph below shows the difference between thick client and thin client. This allows us to understand how the rendering of responses affects performance from the end user's point of view.

Click here for larger image
Figure 4. Thick and Thin Client Performance Results

Network Performance Comparison

The graph below presents the network speed for 3G and 2G networks. Load is generated via the load runner tool. We observed that the 2G response time rose higher than that of the 3G as we increased the load.

Click here for larger image
Figure 5. Comparison of Network Performance Results

As you can see, extensive analysis can be performed with the end-to-end data available for the different components involved in mobile application delivery. Such analysis would not be possible if the device were tested in isolation with no control over server/network conditions.

Conclusion

This article presented a proof of concept for end-to-end mobile application performance testing using a sample Android app. It detailed every component to be tested, the method for testing as well as the challenges involved.

When adopting this approach, be sure your Android application performance tests are under these conditions: