Android testing is a topic that is not discussed enough, in our opinions. So today we thought we’d start a dialogue about this much-overlooked topic: part of mobile app development is mobile app quality assurance (QA).
How Important Is QA to a Mobile App Development Project?
Mobile development teams are quite small compared to traditional software teams, so dedicated testing is often limited or non-existent. Mobile apps are still perceived as “simpler” and “more straightforward” than traditional large software packages. When you envision the stereotypical mobile software developer, isn’t he working in his garage, wearing his pajamas and making millions so he can play World of Warcraft in his spare time? Sure, these types exist, but the truth is most Android developers these days sit in cubicles like other engineers.
Unfortunately, these perceptions lead companies and individuals just jumping into mobile development to underestimate the myriad of resources they will need to truly launch a successful application. Smartphone computing power rivals that of desktop computers only a handful of years ago. Smartphone and device developers work in a resource-constrained environment, and need to use those powers and resources efficiently in order to develop responsive applications. Applications are highly networked and chock full of private user information that must be protected and secured. And, still, they run on little screens with more input methods than your average desktop application–different kinds of touch screens, hardware buttons, keyboards, trackballs, voice control, etc. They are generally being used not for your app, but rather for the features of the phone, such as, well, being a phone, SMS device, and so on. Your app will be interrupted frequently. Finally, just because those screens tend to be small and need to be simple doesn’t mean that good user interfaces are easy to design. In fact, a really elegant, high-performance user interface is a real challenge and not something to be taken lightly.
Then there are the users. They are so diverse! They speak different languages, live in different places–places with good and bad network coverage. They’re constantly moving around. They go through tunnels and fly on airplanes. They drop their phones into hot tubs and answer calls in the middle of playing games. They forget to charge their batteries. They don’t read license agreements and blindly dismiss dialogs. They make mistakes and do things that the developer would never dream anyone sane would try to do with their application. They cannot spell! They don’t grok your awesome, intuitive user interface! And worst of all, users are generally not patient, kind, or understanding when your app fails on them. No, it’s your fault as the developer. Your app gets uninstalled, eviscerated on the market, and who suffers? You do.
We think the answer to the question “How important is QA, really?” can be summed up by your answer to other, more pointed questions:
- “Who will make sure your app is the best it can be, before it is put in users’ hands?”
- “Who will make sure you haven’t missed some obvious, glaring detail that required a second (or third, fourth or tenth) pair of eyes to see?”
- “How important is it to you to make your users happy, or at least not angry?”
- “How important is it to you to have a thriving user community loyal to your app and therefore to you?”
- “How important is it to you that your app reviews are positive?”
- “Who thinks like, and advocates for, the users, as opposed to the developers?” (Yes, it’s annoying, but being challenged makes we developers create better apps-there’s no denying it.)
Who makes these questions a priority? That’s right, QA does.
How Do I Test a Mobile App?
Generally speaking, you test a mobile app just like you’d test any software application. You come up with a test plan based on the features of the application. You take into account the requirements of the platform. For Android, we’re talking about stability, responsiveness, accessibility, performance, elegance, security, and so on. Once you have your test plan, you execute it. Bugs are filed, fixed, verified. The process, the lingo… sure, it varies by company as it always has, but the fundamentals are the same. It requires a budget, personnel, hardware, and will.
But How Do I Test An Android App?
There are a number of ways to approach Android app testing. You can create automated tests to exercise your application systematically. You can tackle it from a white box perspective, diving into the guts of how the application works and debugging at the code level. You can approach it from a black box perspective, using the application exactly as a user might (e.g. randomly). (This is hard for the developers and designers to do – they tend to want to use it how it was designed.) Or you can combine these approaches as best to suit your team, your product, your time and your budget. After all, there’s no such thing as 100% test coverage. You just do the best you can with what you’ve got.
What’s a Basic Setup for Automated Testing an Android App?
You’ve got a couple of choices for automated testing. To start, there is the unit test framework built into the Android SDK. You can use it to develop robust test suites in Java to verify specific functionality available within your application. There is also a set of python APIs called monkeyrunner for automated testing of applications. You can write scripts that install and run applications, send specific keystrokes and take screenshots. There’s also the Exerciser Monkey tool that, while similarly named, is for much more random testing. You can think of it as a virtual monkey mashing at the screen and keys.
What’s a Basic Setup for White Box Testing an Android App?
To white box test an Android app, you’ll want a similar setup to the developer. You’ll want a computer with the Android SDK and tools installed. You’ll want access to the target devices so that you can debug via USB.
There are also a lot of tools you can use to test different aspects of your application. Many of these tools fall into the white box testing category:
- The Eclipse development environment and its debugger
- The Android SDK and related plugin for Eclipse
- The Android emulator
- Android devices enabled for development and debugging, with appropriate USB drivers as necessary
What’s a Basic Setup for Black Box Testing an Android App?
To black box test an Android app simply need access to the target devices to test on, as well as some mechanism for accessing the application package file and installing it on each device. This could be an internal website, or via USB, or what have you. Here are some more important Android tools that can help with black box testing:
- The Android Debug Bridge (ADB) tool can be used for communicating and interacting with devices and emulators. It has a shell interface, which can be useful for installing and uninstalling applications, copying files to and from devices, capturing log output without the hassle of having a development environment set up, inspecting app databases, etc.
- The http://developer.android.com/guide/developing/tools/monkey.html Exerciser Monkey stress testing tool can help hammer your application with input.
- The Hierarchy Viewer and
layoutopttools can help diagnose problems with user interfaces.
What if I Cannot Get a Hold of a Specific Android Target Device?
Sometimes you want to target devices that are not yet released or difficult to get a hold of. There are companies and developer programs (often through manufacturers or carriers) that provide device lab time for you to test your apps. You can also use the Android emulator to create and perform scenarios that are not easily reproduced in the real world, for one reason or another. Just remember that the emulator is only a software representation of the device, and will not behave exactly as a real device.
There are also a variety of programs from manufacturers, carriers, and third-party companies that provide remote access to devices. These usually allow you to load apps onto devices via a Web service or application, and then control the device remotely. The level of control depends on the service. So does the cost. One example is Samsung’s Lab.dev.
Any Other Testing Options I Should Know About?
There are also third-party testing companies out there that specialize in testing of mobile applications. They range from full testing teams to Web tools to test your app on a variety of devices. If you don’t have the time or resources to hire and manage an in-house test team, these options may work better for you.
Conclusion
Quality assurance is a much overlooked area of Android and should be considered essential part of the mobile application development process. The Android SDK provides many resources to support the different kinds of testing approaches out there, including automated testing, black box and white box testing, as well as performance and stress testing. Really, there’s little excuse not to test your applications thoroughly and it’s one of the most straightforward ways to ensure your reaches “killer app” status.
About the Authors
 |
Shane Conder and Lauren Darcey—Contributing Editors, Mobile Development–have coauthored two books on Android development: an in-depth programming book entitled Android Wireless Application Development (ISBN-13: 978-0-321-62709-4) and Sams Teach Yourself Android Application Development in 24 Hours (ISBN-13: 978-0-321-67335-0). When not writing, they spend their time developing mobile software at their company and providing consulting services. |  |
