How Buggy are Java-Based Open Source Software Projects?

Enerjy, the developers of Enerjy code analysis software, scanned dozens of Java-based open source software projects and ranked them from best to worst in terms of code quality.

Scores

The code was scored on a scale of 0 (very likely to be fault-prone) to 10 (very unlikely to be fault-prone). The ten best, with scores ranging from 9.3 to 9.8, were:

• Commons Chain
• Apache Tapestry
• JadClipse
• PMD
• Struts 2
• JUnit
• JFreeReport
• Guice
• PicoContainer
• GanttProject

Popular projects with lower scores included Eclipse, Saxon, Jython, Apache Tomcat, MySQL Connector/J, Commons Logging, and others.

Methodology

Enerjy defines code quality in terms of "fault-proneness," which can be estimated based on the number of faults that had to be fixed in a source code file (as reported by the source code control system) over a given period. Historical analysis of a large body of source code reveals statistical relationships between certain code metrics and code quality. For example, small files are less likely than large files to be fault-prone, files with lots of inline comments are more likely to be fault-prone than those without, and so on. More specifics are available in the technical paper which accompanies the test results.