eXtreme Programming eXperienced (Part 1)

This article describes the experiences of one development team that has successfully adopted many of the eXtreme Programming (XP) principles and values in their development methodologies.

Introduction to the Development Team

Cognitech Ltd. is a small software development company based in central London that develops software solutions for European financial markets. Cognitech consists of a team of eight developers, whose average age is 26; so they are a reasonably young company, even in the software industry. The company was formed in 1997 by three people and has always shown itself to be a highly dynamic environment, having been the incubator for two other companies: Web Precinct Ltd. and Code Factory Ltd.

Cognitech has always promoted learning and now considers itself as a learning organization. They have a substantial library of all the latest technical books and programming manuals, along with a healthy splattering of business-related books and high-tech fiction novels. The company has formed strong relationships with local universities and employs students on placements and a part-time basis. Another way in which learning is promoted is through the use of "morning glory" sessions, these are half-hour seminars that run every morning from 8:30 till 9:00. The members of the development team take turns in presenting topics, including C++ programming techniques, development methodologies, coding standards, and database technologies.

The company has always been quick to adopt new tools and technologies and is constantly looking for ways to improve the way it works. When XP popped up on the radar screen in the office in 1999 some of the development team were keen to embrace the principles and values it put forward. XP looked like it would fit in very well with the way the team worked, and some of the ideas were already being used, without attaching the name of XP to them. The team had already embraced many methods proposed by both Jim McCarthy in Dynamics of Software Development and Steve McConnell in Rapid Development. These methods seemed to be further reinforced by the XP model, and it seemed to make sense to try some of the concepts out.

The Project: Gateway

The project discussed here is a real-world project that the Cognitech development team was hired to work on at the end of November 2000. The software required was a real-time, multisession message interpreter in the form of a service that would run on an NT server machine. The behaviors required of the Gateway included:

1. Translating messages from a proprietary format to CTCI (computer to computer interface)
2. Translating messages from CTCI to a proprietary format
3. Buffering and queuing messages
4. Handling transactions in a timely manner
5. Connections to a control server
6. Supporting multiple monitor app connections.

Additionally, an extra set of functionality was required by a Control Server that could:

1. Connect to multiple gateways
2. Connect to multiple admin tools
3. Store gateway and session details in a database.

The schedule for the project was incredibly tight. Starting in December, with an Alpha delivery in mid-January and a Beta delivery in mid-February, the project was scheduled for completion by the beginning of March. With such tight deadlines, it was clear that the scope to be implemented would have to be chosen very carefully by the client. Cognitech put a team of four developers on the project.

This kind of project was screaming out for an XP approach; it needed very careful steering on a day-by-day basis. The project was started by identifying the core requirements without which the system could not operate. The most basic requirement was to have a system that could receive messages (on a socket) in one format and publish these messages (via a socket) in another format. The following set of components was identified as the basic building blocks for the system.

• Socket
• Translator
• Queue
• Transceiver
• Session
• Gateway

Each component would be able to remain very simple and loosely coupled from the others, allowing maximum flexibility for change without having to code a complex architecture or framework. Each component was defined with a simple COM interface to allow communication to the other components. Once this had been done, test harness code could be written to test the components individually. The test harness code was actually completed well before the component development. The test harnesses were all created in Excel spreadsheets, using VBA to connect through to the COM interfaces and make calls to the component methods.

By the end of the first week of development the components listed above had been skeleton-coded and the test harness code had been developed. The team had also developed an installer and a script for building all the components every night. The team actually had something it could ship; it didn't do much, but it was bug free, and that helps to increase the team morale. The team could then start adding features one at a time and have everything in place to test each feature. This is classic XP and helped the project get off to a good start.

It was considered important to build a close relationship with the client. In the first week of the project, the client phoned to ask if it would be okay if they could send one of their technical staff to sit with the development team for a few days a week. The Cognitech team was actually more than happy to agree to this, much to the surprise of the client. This helped to achieve a customer buy-in to the project very early on and help to steer the development in a direction that was of most value to the customer. It was also important to set up a steering group consisting of the key players in the project. This included the project manager from the client side and the lead developer and coach from the Cognitech team. This steering group met every week to discuss the project's progress and ensure the project was constantly on course. Once the project had reached a certain stage of completed core development, it became obvious that it would also be important to have regular technical reviews of the project with technical staff from the client.

The Development Process

The main XP practices that were applied to this project were:

• Early and continuous testing
• Iterative design
• Pair programming
• Always ready to ship.

The four key values -- communication, simplicity, feedback, and courage -- that Kent Beck discusses in his book were applied throughout this project and are discussed in turn.

Communication both between developers and the client was incredibly important to the success of this project. Without the constant input of direction and thoughts from the client, it would not have been possible to steer the project so closely. The communication between the developers was aided greatly by using the pair programming model. By constantly swapping pairs each developer had an opportunity to work on each part of the system and this greatly enhanced the overall understanding of the system.

Simplicity of the system was achieved by keeping the overall design loosely coupled and ensuring that the code obeyed the main rules of simplicity:

1. Communicates all that is needed
2. Contains no duplicate code
3. Fewest possible classes
4. Fewest possible methods.

This also worked well with enhancing the communication between the development team, as simple code is easy to discuss and understand by all the team members.

Feedback was achieved by building tests first and performing daily build and smoke tests. This constant level of testing highlighted any issues as early as possible. As soon as a test broke, it became the highest priority task on everyone's list to fix.

Courage was shown in two ways during the development process. Firstly, it was courageous to lead the client through the XP development process, the client had no previous knowledge of XP and had to be slowly infected by its ability to create solid, robust code quickly.

Secondly, courage was shown by taking some radical redesign work in hand half way through the project. There was a particular issue with threading and cross-thread synchronization. A decision was made to radically change the design by merging three of the components into one set of code. The transceiver, session, and gateway components were merged and the entire process took less than one morning. It was a bold step to have taken, but the simplicity of the code and the surrounding test harness code made the task far easier than we first imagined.

XP Principles

The major principles from XP were also applied to the project along with several of the other supporting principles. The team has not taken everything from XP as a silver bullet and has used what was found to work well.

By carrying out tests hourly and daily it was possible to achieve rapid feedback and always have knowledge of the exact state of the system. No more "it's nearly finished" either, the tests succeeded or failed.

The assumed level of simplicity was reinforced by ensuring that each component carried out one and only one task. Once the core components had been created as skeletons, the code was added to meet the requirements of the story being worked on. No more code than necessary was ever added to the system. This follows the model of incremental change, where everything was done in small steps or mini-milestones, one feature at a time.

As each feature was implemented, all the tests had to run before the next task could be started. This helped the team keep the quality of their work at the highest level and meant that after the implementation of each feature the system was ready to ship. The nightly build and smoke tests provided a day-by-day sanity check on the entire system.

By working together with the client, the team was able to develop the product together, rather than develop what was thought the client wanted. In order to form this close client relationship, the team involved the client directly in the development process. Technical staff from the client came to work for whole days with the Cognitech development team on the project. It was also important to formally identify the key parties from both the client and the development team. These included the technical lead and project manager from the client and the coach and lead programmer from Cognitech. These key players made up the project steering group, which met at least weekly to guide the project in the right direction for the following week. As well as the weekly steering group meetings, the project's progress was discussed daily by both phone and email, the project manager from the client was given overall project status reports, and the technical lead from the client was involved in a regular questioning of the system specifications.

After the Alpha release in mid-January, the team started to hold weekly technical meetings that involved the technical lead from the client, two technical staff from the client, and Cognitech's lead programmer and coach. These meetings were used to ensure that the technical details of the project were fully understood by the client. This was especially important for this project, as the client required ownership of the system developed. It was important that the client could maintain and support the system once the Cognitech team had developed it and handed it over. The client needed to control the source materials and rights of usage, and Cognitech recognized the importance of understanding this ownership issue. Helping the client to do this meant that Cognitech needed to encourage the client's staff to get involved in the whole development process.

Coding standards were, therefore, not an option but a requirement. It was important that everyone involved could easily pick up the code and make changes, not just the Cognitech development team but also the client's technical staff.

In Part 2 of this article, we'll follow through with our discussion of the project's planning and execution following the XP model.

References

1. Dynamics of Software Development by Jim McCarthy; Microsoft Press; ISBN: 1556158238.
2. Rapid Development: Taming Wild Software Schedules by Steve McConnell; Microsoft Press International; ISBN: 1556159005.
3. Extreme Programming Explained by Kent Beck; Longman Higher Education; ISBN: 0201616416.

About the Author

Dr. Neil Roodyn has been involved with software development for far too long. He did his Ph.D. at University College London in software architectures for distributed real-time systems. Neil currently mentors several London-based software houses and teaches around the world.