With the prevalence of distributed computing, software integration becomes a vital point of any software project these days. The challenge of any large software project is finding problems early and often. A continuous integration (CI) environment allows us to detect problems from build to unit test to basic deployment. If a change breaks the process, CI allows us to find out immediately and fix the problem before it simmers for days or even longer.
Many organizations have realized that the continuous integration paradigm is required to be agile and able to meet business demands. To that end, their development teams have created their own processes and tools/scripts to automate the build and test processes as much as possible. Such tools/scripts are at times difficult to build, costly and time consuming. Today, however, a number of free and open-source CI tools are available to help with the build and test aspects of a developer’s job.
In this article, I present the pain points that CI helps alleviate and then survey a pair of the best known open source CI tools that can help you implement CI in your organization.
Before Continuous Integration: The Integration Problem
Integration costs time, money and other resources. As software components become more complicated, the cost to integrate various pieces increases. Another important factor is that as the number of components increases, the number of integration points increases exponentially. Figure 1 further illustrates this challenge. In a large enterprise, this model is ever more complex as the number of software components tends to be in the hundreds, if not thousands.
In the 90s, messaging middleware and eventually Service Oriented Architecture (SOA) helped with this integration problem by inserting a common bus or interface for all the components to speak through. The Enterprise Service Bus (ESB) is still a major part of the infrastructure for many organizations, large and small. SOA further commoditized the concept of using a known interface and dynamic discovery of a “service” as a measure to ease integration (see Figure 2).
The “integration problem” solved, right? Not exactly; the SOA paradigm just moved the bottleneck to other parts of the software engineering process lifecycle. Let’s look at the software engineering lifecycle (SEL) at a very high-level (see Figure 3).
Figure 3 is not a depiction of what a real SEL is or should be. It merely presents a conceptual representation of the process:
What ESB, SOA and other such technologies did was to dramatically decrease the design and development cycles, but the side effect was an increase in the test and QA cycles! Many books and papers have been written on this topic (e.g. Object-Oriented Defect Management of Software by Houman Younessi, 2002), but decoupling essentially introduces bugs into the design that are harder to catch until the code is ready to be tested. This increases testing time, and some of those bugs go undetected until deployment.
A number of other factors have contributed to the increase in QA/testing cycles, some of which are business driven (demand for faster delivery) and some of which are simply human error (dispersed team members). The fact of the matter is that a change of approach was required to increase efficiency in the process.
Enter Continuous Integration
The introduction of a continuous integration paradigm into the aforementioned cycle fixes the integration problem. CI allows you to know almost immediately if the build process or QA or anything else failed due to a change in the code as a result of a bug. If a developer checks in a piece of code that breaks the build process, you want to know that as soon as possible. For instance, if your integration tests fail after a developer changes an interface, you want to know that right away.
In continuous integration, when code is checked in, the repository is built, basic (or even advanced) tests run, and a new drop is ready for further QA. Some open source tools can help bring this concept to your SEL.
Open Source Tools for Continuous Integration
Hudson (whose primary developers recently ported their own version of the project to GitHub and renamed it Jenkins) is one of many open-source tools (MIT license) available today that can assist with your CI efforts. Hudson targets the following steps in the overall software engineering lifecycle:
- Pre-build cleanup and environment setup
- Actual build process (compile and link)
- Post-build actions such as cleanup, unit test (jUnit commands, for example), integration testing, and deployment
The notification events included in Hudson can alert and/or notify the team about the status of the build.
Hudson can be installed on a remote server in the data center, which potentially is where you have your repository and test environment setup. Hudson can be configured through its Web interface so that various scripts or commands (cron, shell, etc.) can be executed at various points through the process.
Think of Hudson as the final step needed to complete your software engineering lifecycle. Hudson seamlessly integrates with CVS, Subversion, Git and Clearcase and can execute projects based on either Apache Ant or Apache Maven.
Another open source tool (BSD license) with similar functionality to Hudson is CruiseControl (cruisecontrol.sourceforge.net). However, CruiseControl has specific plug-ins for .NET and Ruby.
Either of these two products, along with many other open-source or commercial tools targeting continuous integration, should become part of your overall software engineering lifecycle — if they are not already!
About the Author
Art Sedighi is a freelance writer and IT consultant. Before that, Sedighi was the CTO and founder of SoftModule — now part of Univa UD — where he was the chief architect for SoftModule’s xFactor product. Sedighi received both his BS in Electrical Engineering and MS in Computer Science from Rensselaer Polytechnic Institute. He also holds an MS in Bioinformatics from The Johns Hopkins University.