http://www.developer.com/

Back to article

Considering Test-After Development


September 19, 2007

In this article, I'll build a solution twice. First, I'll write the code for a "reverser" method, a simple bit of code that reverses the order of words in an input string. I'll then use the stricter technique known as test-driven development to re-build the solution.

Test-After Development

In test-after development (TAD), I use my skills as an experienced developer to write my code. I might sometimes write a test prior to development. Predominantly, however, I will first code a solution, and then look to refactor as necessary. I might code additional tests once my solution is in place, but that's my prerogative.

A first test, if I bother to code one, represents a shot at the whole ball of wax. For this example, my first test reverses a full sentence, a test case that will require the final, real logic to get it to pass. Or, at least what I think is the final required logic.

import static org.junit.Assert.*;
import org.junit.*;

public class ReverserTest {
   @Test
   public void reverse() {
      Reverser reverser = new Reverser();
      String text = "This is a short sentence";
      assertEquals("sentence short a is This",
                   reverser.reverse(text));
   }
}

The test takes me about 45 seconds to write. I have to think a bit about the reverse-order loop logic as I code a solution. My total time to get the test to pass is about 120 seconds.

public String reverse(String text) {
   StringBuilder buffer = new StringBuilder();
   String[] words = text.split(" ");
   for (int i = words.length - 1; i >= 0; i--) {
      buffer.append(words[i]);
      if (i > 0)
         buffer.append(' ');
   }
   return buffer.toString();
}

After coding, I reflect on my solution. I recognize that I should probably worry about the case where words are separated by more than a single space character. I code a solution without worrying about the tests. I'm confident that adding a call to the trim method, a small, simple change, will suffice.

public String reverse(String text) {
   StringBuilder buffer = new StringBuilder();
   String[] words = text.split(" ");
   for (int i = words.length - 1; i >= 0; i--) {
      buffer.append(words[i].trim());
      if (i > 0)
         buffer.append(' ');
   }
   return buffer.toString();
}

Ahh. Ready to ship, right? No. My experience tells me that something isn't quite right with the solution. My gut tells me to write an additional test. The test says that the string returned by the reverser method should have eliminated additional spaces.

import static org.junit.Assert.*;
import org.junit.*;

public class ReverserTest {
   private Reverser reverser;

   @Before
   public void initialize() {
      reverser = new Reverser();
   }

   @Test
   public void reverse() {
      String text = "This is a short sentence";
      assertEquals("sentence short a is This",
                   reverser.reverse(text));
   }

   @Test
   public void reverseWithExtraSpaces() {
      String text = "This is a  short sentence";
      assertEquals("sentence short a is This",
                   reverser.reverse(text));
   }
}

Time to write the test: about 30 seconds (including refactoring to a common setup method). The test fails. The actual response returns the extra space between "short" and "a."

Expected: sentence short a is This
Actual:   sentence short  a is This

Racking my brain for a few seconds, I remember that the split method takes a regular expression. I need to split on any grouping of one or more spaces, not just a single space:

public String reverse(String text) {
   StringBuilder buffer = new StringBuilder();
   String[] words = text.split("\\s+");    // <- change
   for (int i = words.length - 1; i >= 0; i--) {
      buffer.append(words[i].trim());
      if (i > 0)
         buffer.append(' ');
   }
   return buffer.toString();
}

Well, now that I think about it a bit harder, the call to trim is no longer necessary. I might not have noticed. And if I hadn't built a test, I probably wouldn't bother removing this redundancy even if I had noticed it.

Think time + solution time: 30 seconds.

Both tests pass. Ship it!

It was a good thing that I used my experience to remember to write a test for such a trivial change. Are there other tests that I need to write? Maybe I should look at the code a bit longer...

Test-Driven Development

With the more ad hoc TAD approach, I wrote a wider test first, one that allowed introduction of a larger chunk of code all at once. I used my skills and experience to drive the introduction of those two minutes of for-loop coding. That approach seemed to have worked well enough, but I did have to remember to go back and bolster things with an additional test.

The approach using test-driven development (TDD) is to start with the simplest possible case instead of the most typical case. What's the simplest case I can think of? Well, what if the string is null? I write my first test, which takes me about 30 seconds. (I have to get the test class in place to support the first test, like the TAD solution, but it's a simpler test.)

@Test
public void answersNullForNullText() {
   assertEquals(null, new Reverser().reverse(null));
}

That's easy! Within 10 seconds, I have a passing implementation in place:

public String reverse(String text) {
   return null;
}

Already, I have a test that I didn't have before, plus I have a more robust implementation. A novice programmer can easily remember to start from null as the simplest case.

Going back to the TAD solution, my experience failed me. Upon adding a new after-the-fact test, I discover that the reverse method throws a NullPointerException if passed a null input string. I'm sure I'd have thought to code for that, given enough time (or perhaps someone might have reminded me at our next code review session).

The next simplest step is the empty string:

import static org.junit.Assert.*;
import org.junit.*;

public class ReverserTest {
   private Reverser reverser;

   @Before
   public void initialize() {
      reverser = new Reverser();
   }

   @Test
   public void answersNullForNullText() {
      assertEquals(null, reverser.reverse(null));
   }

   @Test
   public void answersEmptyStringForEmptyString() {
      assertEquals("", reverser.reverse(""));
   }
}

Writing the test, plus refactoring to a common setup, takes about 30 seconds. The implementation again takes a few seconds (about 10):

public String reverse(String text) {
   return text;
}

For the second time, I document a case that I probably wouldn't have worried about when doing TAD. Fortunately, this is a test that does pass against the existing TAD solution. But what's better about the TDD solution is that the document—the test named answersEmptyStringForEmptyString—provides a piece of instantaneous information. In its absence, I would have had to spend time gleaning that information from careful code analysis of the TAD solution.

In about 15 seconds, I write the case for a single-word string, and realize that the implementation need not change (in other words, the tests already pass).

@Test
public void answersInputForSingleWord() {
   assertEquals("one", reverser.reverse("one"));
}

The next case, an input string that contains two words (15 seconds):

@Test
public void swapsTwoWords() {
   assertEquals("two one", reverser.reverse("one two"));
}

This failing test triggers the following implementation (about 30 seconds):

public String reverse(String text) {
   if (text == null)
      return null;
   String[] words = text.split(" ");
   if (words.length == 1)
      return words[0];
   return words[1] + " " + words[0];
}

My initial solution doesn't include the guard against text being null. That omission breaks the test answersNullForNullText, something I rectified rapidly.

For the next test, I could jump to the multi-word input that requires a loop in the solution. But, my interest in finding a next increment of solution in rapid time triggers the thought, "What if the two words are separated by more than one space?" I take 15 seconds to code this test:

@Test
public void eliminatesExtraSpacesBetweenWords() {
   assertEquals("two one", reverser.reverse("one  two"));
}

The implementation, very specific to the needs of the test:

public String reverse(String text) {
   if (text == null)
      return null;
   String[] words = text.split("\\s+");
   if (words.length == 1)
      return words[0];
   return words[1] + " " + words[0];
}

It took me 15 seconds to build this solution, but perhaps this is a legacy of having already solved the problem once. (Still, I sometimes find that the solution is more obvious in the smaller context that TDD often promotes.) But, for sake of a fairer comparison to the TAD solution, I'll say it took me the same 30 seconds to build it.

The final test, for now (20 seconds):

@Test
public void reversesManyWords() {
   assertEquals("three two one",
                reverser.reverse("one two three"));
}

Now, with all of the other special conditions and edge cases in place, I still need a loop, and much of the original TAD solution (90 seconds):

public String reverse(String text) {
   if (text == null)
      return null;
   String[] words = text.split("\\s+");
   StringBuilder builder = new StringBuilder();
   for (int i = words.length - 1; i >= 0; i--) {
      builder.append(words[i]);
      if (i > 0)
         builder.append(' ');
   }
   return builder.toString();
}

All six tests pass.

Conclusions

Did I spend more time building the TDD solution? You bet, but not by much. By using TAD, did I introduce a potentially devastating defect into the code? You bet. And if I had remembered to add the missing tests after the fact, and had fixed the problem in the TAD solution, the solution times would have been about the same between TDD and TAD.

Here are the times for building each solution, based on the step times that I tracked above:

TAD: 3 minutes 45 seconds + 1 defect (45+120+30+30)
TDD: 4 minutes 55 seconds + 0 defects (30+10+30+10+15+15+30+15+30+20+90)

Fascinating. Note the step times involved for each solution. Derivation of the TAD solution is more erratic, while the TDD solution demonstrates a more consistent, almost rhythmic, pace.

It took me a minute longer to build the TDD solution. My tradeoff: the ability to ship code with high confidence that it actually works. In a short-sighted world, where we look at only time to deliver, TAD works "just fine." But in the "real world," where we willingly ship software of poor quality, we pay for that one minute tens, hundreds, perhaps thousands of times over. I remind myself that it takes only a little more initial time to craft quality code.

It's theoretically possible to write as many tests using TAD. But often, developers don't think about all the tests that should be written, or they run out of time. The reality is that far fewer tests get written using TAD. Even expert proponents will admit that 70% coverage is about as high as is realistic for TAD code. In contrast, code coverage tools report coverages of 95% and up against the well-written TDD code I've seen.

One more thing that's almost impossible to factor into comparisons: The tests produced by TAD usually have little value in documenting the system. Worse, they usually don't provide the confidence required to refactor the code base at will—which means that the system will degrade in quality more rapidly.

In contrast, well-written TDD tests help an unfamiliar developer more rapidly understand the system. They also provide a much higher confidence for refactoring—which means that the system's quality will degrade less rapidly.

Ultimately, the amount of time saved here and there by TAD pales in comparison to the costs it generates in long-term maintenance and quality of the system. Not to mention that I haven't discussed at all how TDD can help significantly improve the design quality. [In my next article, I'll discuss where testability and design are aligned with one another and where they are not.]

My conclusion: test-after development is a "TAD" too late.

About the Author

Jeff Langr is a veteran software developer celebrating his 25th year of professional software development. He's authored two books and dozens of published articles on software development, including Agile Java: Crafting Code With Test-Driven Development (Prentice Hall) in 2005. You can find out more about Jeff at his site, http://langrsoft.com, or you can contact him via email at jeff at langrsoft dot com.

Sitemap | Contact Us

Thanks for your registration, follow us on our social networks to keep up-to-date