Software Testing for Programmers, Part 1
Some programmers do it as they code, and others wait until the end. Either way, testing is a necessary part of any software development project. Without it, one cannot determine that the software functions correctly. In this article, I present the basics of software testing from a programmer's perspective. In a follow-up article, I will illustrate unit testing with JUnit, an open source framework for testing Java applications.
Software testing is usually done at several levels. They are commonly referred to as unit testing, integration testing, and system testing.
The objective of unit testing is to verify that individual units -- the smallest compilable components, such as Java classes -- function correctly. Unit testing is usually done by programmers.
Unit testing is also called component testing. However, component testing is sometimes considered to be a level of testing above unit testing. This may be the case with a system that contains individually testable components that are composed of multiple units. For example, in a system that contains Java classes and Enterprise Java Beans (EJBs), the Java classes could be tested as units, and the EJBs, could be tested as components.
Alternatively, some people distinguish unit testing from component testing by the degree to which components are isolated. In unit testing, called components are replaced with stubs, simulators, or trusted components, and calling components are replaced with drivers or trusted super components, so that the component being tested is isolated. In component testing, all stubs and simulators are replaced with real implementations. 
The objective of integration testing is to test the integration of and communication between components. Additionally, it may include testing the integration of subsystems or communication with external systems. Integration testing may be done by the programmer, but it may also be done by the build captain, or the team lead, the project manager, or even a configuration management group.
On some projects, integration testing may be divided into two levels: assembly testing and system integration testing. During assembly testing, the integration of the software components is tested. During system integration testing, the communication with external systems is tested. For example, on a project to develop a set of EJBs for use by external applications, assembly testing could be done to test the integration of the EJBs and the components from which they are built, and system integration could be done to test communication between the EJBs and the external applications.
The objectives of system testing are to find defects that are attributable to the behavior of the system as a whole, rather than the behavior of individual components, and to test that the software functions as a complete system. This level of testing is different from integration testing in that the tests are concerned with the entire system, not just the interactions between components. Other than system functionality and behavior, system testing may include testing configuration, throughput, security, resource utilization, and performance.
Just as the components of a system must be designed, tests for a system must be designed. Two common methods of test design are black box testing and white box testing. After the tests are implemented, though, the test design method may not be evident.
Black box testing focuses on designing tests that view components as opaque. The implementation details of the components are not known (or, at least, are ignored), and only externally observable behavior or functionality is tested. Other terms for black box testing are functional testing and behavioral testing.
However, behavioral testing and black box testing are slightly different. Even though both view components as black boxes, behavioral testing allows knowledge of the implementation details of components to be used in test design if necessary.
White box testing focuses on designing tests that view components as transparent. The implementation details of the components are known, and that knowledge is used in test design and creation of test data. Other terms for white box testing are structural testing and clear box testing.
There are many techniques that can be used to test software. Some are better than others, and some can be used in conjunction with others to get better test coverage. Here is a summary of some common testing techniques:
Manual testing - Tests are done by a human with est data that may be predetermined but may also be determined per test. In some cases, manual testing could be characterized as "banging away" at the software.
Automated Testing - Tests can be run by a tool or an unattended process, such as a nightly build, and they can be re-run many times. Test data is predetermined or generated.
Regression Testing - Tests, usually automated, are run to determine if modifications or enhancements have negatively affected functionality that passed previous testing.
Stress Testing - Tests are run to determine if the software can withstand an unreasonable load with insufficient resources or extreme usage.
Load Testing - Tests are run to determine if the software can handle a typical or maximum load or to determine the maximum sustainable load.
Performance Testing - Tests are run to determine actual performance as compared to predicted performance.
As would be expected, testing tools can make testing more efficient. Here is a short list of popular tools:
JUnit - Open source, unit testing framework for Java. (Note: JUnit has been ported to other programming languages and platforms, including .NET, C#, Perl, Python, Delphi, and others.)
JTest, C++Test, .Test - Automated unit testing and static analysis for error prevention in Java, C++, and .NET programs (Parasoft).
WinRunner - GUI testing (Mercury Interactive).
TeamTest - Automated functional and performance testing (Rational).
SilkTest - Automated functional and regression testing (Segue Software).
- comp.software.testing FAQ
- Test for Fun and Profit, Todd Sundsted, JavaWorld, October 1999
- StickMinds.com Testing Zone