The Surprising Truth about Exploratory Testing
By Vu Lam
Exploratory testing is an important tool in any experienced tester’s kit. In the modern age of Agile development, with software being updated and released faster than ever, exploratory testing is nothing short of essential. The trouble is that there seems to be some confusion over what the term actually means.
What is Exploratory Testing?
Real exploratory testing is not the same as ad hoc testing; it’s much more sophisticated and organized than that. The term was originally coined by Cem Caner in his book, Testing Computer Software , to describe an alternative approach to traditional formal planning. He implored testers to “Trust your instincts.” But he also warned that you should “always write down what you do and what happens when you run exploratory tests.”
Another author, James Bach, described exploratory testing as “scientific thinking in real-time.” In his 1999 paper, General Functionality and Stability Test Procedure , Bach expounded on that, explaining that “unlike traditional informal testing, this procedure consists of specific tasks, objectives, and deliverables that make it a systematic process…” and went on to say, “In operational terms, exploratory testing is an interactive process of concurrent product exploration, test design, and test execution. The outcome of an exploratory testing session is a set of notes about the product, failures found, and a concise record of how the product was tested. When practiced by trained testers, it yields consistently valuable and auditable results.”
More than a decade has passed since Bach wrote those lines, yet his wisdom is still lost on many people.
When to Employ it
It’s very common for testers to be under time pressure, and the shift to Agile methodology in software development has had a tangible impact on documentation. To put it simply: testers have very little time to test a new build and must often do so without access to detailed documentation.
Exploratory testing is ideal for this scenario, but should not be an exclusive approach. It complements scripted testing activities. Exploratory testing is simply part of an overall plan that must include new feature exploration, bug validation and regression testing.
If you’re not sure what to test next, or you want to dig deeper into a complex piece of software, then it’s time for some exploratory testing.
How to do it Right
What you’re really looking to do is design and execute tests at the same time. Although you won’t be using requirements to prepare detailed test cases before you begin, that doesn’t mean you should test randomly. There should be a focus on specific parts of the product, or an agreement to employ specific strategies during the testing. Combine your intuition with the knowledge you have amassed on the software from previous tests, and your perceptions about the needs of the end user. You must understand what the software is supposed to do. Then you can appreciate how developers tried to solve specific problems and decide whether they accomplished their objectives. By leveraging your insight and targeting your exploration, new issues will be discovered. This is not a random bug hunt.
Good testers will naturally develop the skills required for exploratory testing; the ability to think creatively and generate useful tests in real-time. A dogged, rigid approach to testing software, especially software that is rapidly evolving, does not make the best use of a tester’s intellect, and won’t result in the best final product possible.
You’ve established your boundaries and the scope of your testing session upfront. In order to get the maximum benefit from your discoveries you have to generate reports of your findings.
You should be recording your expectations and the actual results. Were you able to achieve your goal? Did it take longer than expected? Did you encounter weaknesses in the software? Remember that another tester, or a member of the development team, should be able to follow your thoughts and reasoning. Clarity is vital.
Because you have a frame of context the data you generate can really benefit the development team. Instead of the isolated defects, suggestions, and concerns that an ad hoc testing approach might generate, you get in-depth information on how well the software meets intended goals and how it will be perceived by the end user.
In addition to verifying software and discovering issues, exploratory testing can help identify useful test cases for future regression testing. An exploratory testing session can be the bridge between unscripted and scripted testing for your project.
The Right Approach
The surprising truth is that exploratory testing encourages the right approach to testing any software. You need to build a real insight into the product and think imaginatively about how it will be used. Combine that creativity with metrics and detailed reporting, and you can see the big picture. Unlike ad hoc testing, exploratory testing is an organized, well-structured approach that lets testers excel in the ever changing landscape of Agile development.
About the Author
Vu Lam is CEO of QASymphony , a leading developer of Quality Management solutions for software developers and QA testers. He was previously with First Consulting Group and was an early pioneer in Vietnam’s offshore IT services industry since 1995. He holds an MS degree in electrical engineering from Purdue University. You may reach him at firstname.lastname@example.org .