Some programmers do it as they code, and others wait until the end. Either
way, testing is a necessary part of any software development project. Without
it, one cannot determine that the software functions correctly. In this article,
I present the basics of software testing from a programmer’s perspective. In a
follow-up article, I will illustrate unit testing with JUnit, an open source
framework for testing Java applications.
Testing Levels
Software testing is usually done at several levels. They are commonly referred
to as unit testing, integration testing, and system testing.
Unit Testing
The objective of unit testing is to verify that individual units — the
smallest compilable components, such as Java classes — function correctly. Unit
testing is usually done by programmers.
Unit testing is also called component testing. However, component testing is
sometimes considered to be a level of testing above unit testing. This may be
the case with a system that contains individually testable components that are
composed of multiple units. For example, in a system that contains Java classes
and Enterprise Java Beans (EJBs), the Java classes could be tested as units, and
the EJBs, could be tested as components.
Alternatively, some people distinguish unit testing from component testing by
the degree to which components are isolated. In unit testing, called components
are replaced with stubs, simulators, or trusted components, and calling
components are replaced with drivers or trusted super components, so that the
component being tested is isolated. In component testing, all stubs and
simulators are replaced with real implementations. [1]
Integration Testing
The objective of integration testing is to test the integration of and
communication between components. Additionally, it may include testing the
integration of subsystems or communication with external systems. Integration
testing may be done by the programmer, but it may also be done by the build
captain, or the team lead, the project manager, or even a configuration
management group.
On some projects, integration testing may be divided into two levels:
assembly testing and system integration testing. During assembly testing, the
integration of the software components is tested. During system integration
testing, the communication with external systems is tested. For example, on a
project to develop a set of EJBs for use by external applications, assembly
testing could be done to test the integration of the EJBs and the components
from which they are built, and system integration could be done to test
communication between the EJBs and the external applications.
System Testing
The objectives of system testing are to find defects that are attributable to
the behavior of the system as a whole, rather than the behavior of individual
components, and to test that the software functions as a complete system. This
level of testing is different from integration testing in that the tests are
concerned with the entire system, not just the interactions between components.
Other than system functionality and behavior, system testing may include testing
configuration, throughput, security, resource utilization, and performance.
Test Design
Just as the components of a system must be designed, tests for a system must
be designed. Two common methods of test design are black box testing and white
box testing. After the tests are implemented, though, the test design method may
not be evident.
Black Box
Black box testing focuses on designing tests that view components as opaque.
The implementation details of the components are not known (or, at least, are
ignored), and only externally observable behavior or functionality is tested.
Other terms for black box testing are functional testing and behavioral
testing.
However, behavioral testing and black box testing are slightly different.
Even though both view components as black boxes, behavioral testing allows
knowledge of the implementation details of components to be used in test design
if necessary.
White Box
White box testing focuses on designing tests that view components as
transparent. The implementation details of the components are known, and that
knowledge is used in test design and creation of test data. Other terms for
white box testing are structural testing and clear box testing.
Testing Techniques
There are many techniques that can be used to test software. Some are better
than others, and some can be used in conjunction with others to get better test
coverage. Here is a summary of some common testing techniques:
- Manual testing – Tests are done by a human with est data that may be
predetermined but may also be determined per test. In some cases, manual
testing could be characterized as “banging away” at the software. - Automated Testing – Tests can be run by a tool or an unattended
process, such as a nightly build, and they can be re-run many times. Test
data is predetermined or generated. - Regression Testing – Tests, usually automated, are run to determine if
modifications or enhancements have negatively affected functionality that
passed previous testing. - Stress Testing – Tests are run to determine if the software can
withstand an unreasonable load with insufficient resources or extreme
usage. - Load Testing – Tests are run to determine if the software can handle a
typical or maximum load or to determine the maximum sustainable load. - Performance Testing – Tests are run to determine actual performance
as compared to predicted performance.
Testing Tools
As would be expected, testing tools can make testing more efficient. Here is
a short list of popular tools:
- JUnit – Open source, unit testing framework for Java. (Note: JUnit has
been ported to other programming languages and platforms, including .NET,
C#, Perl, Python, Delphi, and others.) - JTest, C++Test, .Test – Automated unit testing and static analysis for
error prevention in Java, C++, and .NET programs
(Parasoft). - WinRunner – GUI testing
(Mercury Interactive). - TeamTest – Automated functional and performance testing
(Rational). - SilkTest – Automated functional and regression testing
(Segue Software).
Resources
- comp.software.testing FAQ
- Test
for Fun and Profit, Todd Sundsted, JavaWorld, October 1999 - StickMinds.com Testing Zone
- JUnit
