Code Coverage for Free: How to Use EMMA
In this article you will learn how to configure Open Source framework EMMA to work with a web-based application.
Why Do You Need Code Coverage?
How do you know if the code you have written is being executed during regression or unit testing? One way is to use a debugger. Another way is to log every line of code. Yet another way is to compare the expected results of the test with the actual results. The latter usually suffices. All three methods are "somewhat" potent, but nonetheless, all three methods are not enough.
The first case will work in a small, non-repeatable test. The second method clutters your code and the verification process is labor intensive and ineffective. Only the third method is generally accepted by the IT and user community. The third method relies on the test author to cover ALL conditions and prepare test data in such a way that all test conditions are met. You put your trust into the test author's hands. Now, consider the following situation:
You are testing a method that accepts two parameters. It returns false if the first parameter is 1 or the second parameter is 1. Otherwise, the method returns true. The test author creates three test cases to cover three possible scenarios:
- A is 1; the expected result is false.
- B is 1; the expected result is false.
- Both A and B are not 1; the expected result is true.
For the test data, the test author creates three records corresponding to each case:
- A = 1, B = 0
- A = 0, B = 1
- A = 2, B = 2
The above test data in conjunction with three test cases covers all exit points of the returnSomething () method. Now, suppose the test author made a mistake and created the following test data:
- A = 1, B = 0
- A = 1, B = 1
- A = 2, B = 1
What happens now? The second test case return false, which matches our expected result, but Line 4 had never been executed. This is precisely why you need a code coverage tool.
I have chosen EMMA as a code coverage tool of choice because:
- It is open-source 100% Java.
- It has a large market share compared with the other open source coverage tools.
- It is easy to use (once you figure out how to use it).
- It has most of the features that one needs from the code coverage tool:
- It has multiple report type formats.
- It has class instrumentation that can be done offline and on the fly.
- It can be used in an enterprise environment.
- It has ANT support.
- It includes the ability to drill down to class, method, and lines of code.
- It is free for both open-source and commercial development.
Note: See other open source coverage tools here: http://java-source.net/open-source/code-coverage.
Emma's home page is http://emma.sourceforge.net/.
This overview explains how to configure EMMA for a web-based Java application. It entails:
- Recompilation of Java classes using the debug="on" option.
- Off-line instrumentation of application-specific Java classes by using the EMMA <instr> command.
- Extracting Java coverage dump from JVM by using the <ctl> ANT command.
- Building code coverage report by using the EMMA <report> command.
All development for this how-to is done within the IBM Rational Software Development Platform 6.0.1 and Websphere 5.1 test environment.
Get JAR Files
Emma jar files (emma.jar and emma_ant.jar) can be downloaded here: http://sourceforge.net/project/showfiles.php?group_id=108932&package_id=117768. You want to get version 2.1.5320 just to be on board with my example.
Import Emmarun.zip into Your Workspace
You can find a link to the download of EmmaRun.zip at the end of this article. The zip file contains one servlet (EmmaTestServlet.java), deployment descriptor (web.xml), one jsp file (input.jsp), and ANT files for running EMMA tools (1_emma_compile.xml, 2_emma-instument.xml, 3_emma_copyfiles.xml, 4_emma-extract-covarage.xml, and 5_emma_report.xml).
Go to FILE → IMPORT → ZIP FILE and follow further instructions on the screen.
Add JAR Files to Your Application Server Classpath
Add Emma.jar and emma_ant.jar to application server classpath Locate Application Server for your web application → Select Environment Tab → under Class Path, click on "Add External Jars", and add emma.jar and emma_ant.jar.
Recompile Classes for Code Coverage
Locate the 1_emma-compile.xml Ant script and execute it. The output should look like this:
This script compiles your EmmaTestServlet.java and places the class file into a emma_classes directory under WebContent/WEB-INF/. The only thing to note in this script is that you do compile with the debug="on" option.
Instrument Classes for Code Coverage
Locate the 2_emma-instument.xml Ant script and execute it. The output should look like this:
This script performs the following steps:
- It resets this demo project to a clean state.
- It turns on EMMA instrumentation.
- It runs the EMMA instrumentation <instr> by taking the class files produced by 1_emma-compile.xml ANT script and producing metadata.em instrumentation file and EMMA-instrumented class files.
Locate the 3_emma-copyfiles.xml Ant script and execute it. This script copies instrumented class files from the "emma_intr_classes" directory to the "classes" directory. This is the output of the script: