Using Randomization in Java Unit Testing

The amazing thing about life is that it is consistent and random simultaneously. If you live in the northeast of the United States, you expect the winter to be cold and the summer to be hot. While temperatures will vary randomly from day to day, you expect a "cold" or "hot" pattern to be in play during those seasons.

Thus, when testing a temperature-dependent product (anti-freeze, for example), it's more accurate to test against random temperatures within a range than to test against a single constant value.

The same can be said of software. Usually, software is meant to run under conditions that are variable. Thus, when you write your unit tests, you will do well to use data that is random within a given range. Yet, for the most part, when it comes to writing basic unit tests, coders tend to hard-code testing data. The reasoning is that when it comes to basic unit tests, hard coding data is much less time consuming and not that risky.

I am here to tell you differently. Using randomization in your unit tests is easy and not at all time consuming, and it will increase the value of your tests in the short and long term. All you need to do is use a Java-based tool that I wrote called the QaRandomizer, which enables you to emit random values for common patterns and data structures quickly and easily.

You can use the QaRandomizer to generate:

• A random zip code with its associated city and state.
• First names, last names, full names, or email address.
• Strings of random characters, each of random length.

The QaRandomizer also has a helper method that extracts a random item from a list of similar type objects.

In this article, I will show you how to use QaRandomizer to provide random data in basic unit testing scenarios, as well as unit tests that use mock objects. Although the concepts I discuss in the article are general, the examples are in Java and the provided sample code is a Maven parent project with child projects. Thus, if you plan to work with the example code, you should know something about object-oriented programming, Java, Maven, and the TestNG testing framework.

Understanding the postal-service Project

The project that accompanies this article is called postal-service. (Click here to get the Maven project source code for postal-service and QaRandomizer.) Postal-service contains objects that represent items and services you'd find in a postal scenario (see Figure 1):

• You create Letter objects that you use with a PostOffice object to stamp and send mail.
• You buy a Stamp object from the PostOffice object.
• You define the sender and recipient of the Letter using Address objects.
• In addition to providing a service to send a Letter, the Post Office provides an Email transmission service.
• The PostOffice object uses a PostalDispatcher interface to facilitate the movement of
• Letter and Email objects.

Figure 1. The Postal Service Object Model: Postal-service contains objects that represent items and services in a postal scenario.

Avoiding the Usual Unit Testing Pitfall

OK, let's get into using randomization in unit testing. A fundamental principle of test-driven development (TDD) is that you write your tests before you write your code. Thus, let's start with the Address object. A pretty basic test is the getter/setter test. Typically in such a test, you need to make sure that the values that you set in the object "stick" and can be accessed through the getter. Listing 1 below shows a unit test for Address setter and getters.

Although pretty trivial, the unit test in Listing 1 illustrates a common pitfall. As you can see in Listing 1 and the code below, the programmer assigned hard-coded values to the Address object and wrote production code that got the test to pass.

//Create some test data
String firstName="Bob";
String lastName="Reselman";
String address1="The Waldorff-Astoria Hotel";
String address2="301 Park Avenue";
String city="New York";
String state="NY";
String zip="10022";

//Apply the data to the Address object
Address address = new Address();
address.setFirstName(firstName);
address.setLastName(lastName);
address.setAddress1(address1);
address.setAddress2(address2);
address.setCity(city);
address.setState(state);
address.setZip(zip);

Yet, in the final analysis, the test passes using the assigned values and only those values. Thus, the only guarantee is that the implementation code written to pass this test is applicable only to that very small sample of hard-coded data: "Bob", "Reselman", "The Waldorf-Astoria Hotel" "301 Park Avenue", "New York", "NY", "10022".

The developer is assuming that this test is an accurate behavioral model for the code to be written—a questionable assumption. What happens if another developer comes along a year from now and decides in the most fantastic of manner that names that are longer than five characters will not be supported by the Address object's setFirstName(String name) method? What then? In terms of regression testing, "Bob" would pass just fine despite the fact that failing conditions exist.

Now this is not say that you can write a unit test that covers every situation and behavior that may occur for years to come. Clearly, such an expectation is unrealistic. But for the same amount of labor that is required to write tests using hard-coded test data, you can use randomization to cover your testing expectations.

Listing 2 below is an enhancement of the unit test demonstrated in Listing 1. In this case of Listing 2, you use the QaRandomizer to generate random data over multiple iterations.

The QaRandomizer provides the ability to generate random first and last name data. It also generates a RandomAddress object that describes a typical postal address and contains random address data. (You'll read more about the RandomAddress object later on when I discuss the methods that make up the QaRandomizer API.)

Notice that in Listing 2 not only is first name, last name, and address data generated randomly, the testing is run over a number of iterations (10, in fact). Thus, the code is saying that you have generated 10 sets of random data against which this test has run and passed 10 times. Probability is on your side!

Listing 3 below shows the data generated in the randomSetterGetterAddressTest() demonstrated in Listing 2.

Listing 3: A Sampling of Name and Address Data Generated by QaRandomizer
 
Akira Morrisroe, 38 Roosevelt St. Suite 52 SAINT PAUL, KS 66771
Kaoru Sole, 798 Olympic Ave. Floor 87 WASHTA, IA 51061
Irwin Gugliotti, 372 Cherry Dr. Apt. 31 SMITHFIELD, ME 04978
Cheryl Edsall, 865 Elm Dr. Apt. 73 SHANDON, CA 93461
Sharayn Zitello, 103 Olympic Blvd. Room 15 ZEPHYR, TX 76890
Aleck Hermes, 794 Main Dr. MailStop 39 VERONA, WI 53593
Ossie Morrow, 250 Pine St. Room 86 AMA, LA 70031
Milos Calderone, 591 Spruce St. Desk 34 SAINT PETERSBURG, FL 33701
Laurie Brocklehurst, 898 Orange Ct. Apt. 68 GREENWICH, NY 12834
Joleen Jarvis, 538 Washington Dr. Apt. 40 PHOENIX, AZ 85010

If you use the code in Listing 2 as the "Test First test to pass" when writing your trivial Address object, you can rest more assured that your production programming will produce the outcome that you desire. There is no hard coding in play. Each test iteration uses distinct data. And, if you want greater reliability that our object can handle any data, all you need to do is increase the number of iterations in which to run the test.

Now that I've shown you a small benefit of the QaRandomizer, let's look at the services that it offers so that you can use the tool in a variety of situations.

Working with the QaRandomizer

Table 1 below describes the methods that make up the QaRandomizer. Again, notice that there are methods that generate random first name, last name, full name, address structure, and email addresses. There are methods that provide lists of the internal data against which randomization is performed. Lastly, the QaRandomizer provides a method that allows you to retrieve a random item from a typed list.

The sections that follow describe the features of the QaRandomizer in detail.

Getting a Random Name Value

The QaRandomizer provides three methods that you can use to work with names on a random basis:

• getFirstName()
• getLastName()
• getFullName()

Listing 2 above uses these methods. They return a random first name, last name, or full name from a list of first and last names described in XML files that are embedded as resource files in the QaRandomizer project.

Figure 2 below illustrates the way that first name, last name, and zip code XML files are processed by the QaRandomizer code.

Figure 2. QaRandomizer XML-to-Java Conversion: The QaRandomizer uses dom4j to convert XML files into a DOM Java objects.

You can use the methods getFirstNamesList() and getLastNamesList() to retrieve the first and last names in force.

Working the Random Address Object

As mentioned previously, the QaRandomizer provides the ability to generate random address data within a RandomAddress object. Address data is constructed by randomly selecting a US zip code, with its related city and state, from an XML file embedded as a Java resource in the QaRandomizer binary. After the city, state, and zip are extracted from the XML file, a fictitious street address and a secondary address are constructed randomly.

Listing 4 shows you how to use the QaRandomizer.getInstance().getAddress() method to get a random address.

Notice that the QaRandomizer is a Singleton. Using the Singleton pattern allows all the XML data in the embedded resource files to load into the QaRandomizer internals once, upon module initialization.

Using Randomization with Mock Objects

The trick with unit testing is to test one unit of functionality. Usually that unit is a method. Testing the integrity of an entire class is typically relegated to integration testing. Many times when you are testing a method, you need to work with code that has yet to be written. In cases such as this, you use mock objects. (You can learn more about using mock object here.)

Testing the send() method of the PostServiceImpl object requires that you use a mock object. (PostOfficeImpl is an implementation of the PostOffice interface.) The send() method delegates dispatching letters and emails to an internal object that implements the PostalDispatch interface. However, the dispatch object that will implement PostalDispatch has yet to be written. Thus, you implement the PostDispatch interface into a mock object.

Listing 5 below demonstrates the use of a mock object to emulate unimplemented behavior in a testing scenario associated with the PostOfficeImpl object.

Listing 5: A Test That Uses a Mock Object to Verify Use of an Unimplemented Interface, PostalDispatcher
 
@Test 
public void sendEmailViaPostOfficeTest() throws QaRandomizerException {
// Set up the mock for the PostalDispatcher interface PostalDispatcher
mockDispatcher = createMock(PostalDispatcher.class);
// Set the mock Dispatcher to the Post Office object
PostOfficeImpl.getInstance().setDispatcher(mockDispatcher);
// Get an Email with data from the QaRandomizer Email
email = getRandomEmailWithBody(); 
// Tell the mock framework to expect a call to the 
// dispatch() method
mockDispatcher.dispatch(email); 
// Register the mock scenario
replay(mockDispatcher); 
// Call the behavior that we're testing for
PostOfficeImpl.getInstance().send(email); 
// Verify that the test passed
verify(mockDispatcher); 
}

The mock scenario in Listing 5 is looking for a call to the mockDispatcher.send() method, requires that an email address be provided. Now, you could have unwittingly used a hard-coded email address to satisfy that requirement. But you've thought ahead and brought the habit of using randomization to your unit test. Thus, using the code shown in Listing 6 below in conjunction with the code in Listing 5, you can leverage the QaRandomizer.getEmailAddress() method to easily create to a unit test the meets testing requirements under a very broad condition.

Listing 6: A Private Method That Uses a Random Email Addresses
 
private Email getRandomEmailWithBody() throws QaRandomizerException {
// Make the random data string data 
String body = QaRandomizer.getInstance().getComplexString(400);
//Create a random to email
address String toEmail = QaRandomizer.getInstance().getEmailAddress();
//Create a random from email address 
String fromEmail = QaRandomizer.getInstance().getEmailAddress(); 
// Create the Email object 
Email email = new Email(toEmail, fromEmail, body); 
// return it 
return email; 
}

Get a Random Item from a Typed List

The QaRandomizer is designed to provide basic randomization services that you can use in many situations. However, at some point you most likely will want to use randomization from a list of data that you designed. In such cases, you use the QaRandomizer.getRandomItem(Set<T>) method, which takes as a parameter a list of objects of the same type. Thus, you provide to the method a list of data-filled objects (Beans, for example). The QaRandomizer.getRandomItem(Set<T>) method will randomly choose an object to return from the list passed in as a parameter.

Listing 7, in conjunction with Listing 6 and Listing 8, shows you how to use getRandomItem(). The scenario that these code listings illustrate is creating a finite list of email addresses and then running a test that selects a random item from that finite list.

Listing 7: Getting a Random Object From a Strong Typed List in Conjunction With a Mock Object

public void sendEmailFromFiniteListTest() throws QaRandomizerException { 
// Get a list of 6 email addresses 
Set< Email>emails = getListOfEmails(6); 
//Set up the mock for the PostalDispatcher interface PostalDispatcher
mockDispatcher = createMock(PostalDispatcher.class); 
// Set the mock Dispatcher to the Post Office object
PostOfficeImpl.getInstance().setDispatcher(mockDispatcher);
// Get an Email with data from the finite list of email addresses 
Email email = QaRandomizer.getInstance().getRandomItem(emails);
// Tell the mock framework to expect a call to the 
// dispatch()
method mockDispatcher.dispatch(email);
// Register the mock scenario 
replay(mockDispatcher); 
// Call the behavior that we're testing against
PostOfficeImpl.getInstance().send(email);
// Verify that the test passed 
verify(mockDispatcher);
}



Listing 8: Creating a HashSet<Email> List
 
private Set< Email> getListOfEmails(int listCount) throws QaRandomizerException {
//Create the strong typed list
Set<Email> emails = new HashSet<Email>();
//Iterate through and add an email with body to the list
for (int i = 0; i < listCount; i++) {
Email email = getRandomEmailWithBody();
emails.add(email);
} 

return emails;
}

Granted, creating a list with a predefined number of email addresses is a bit trivial. However, you can easily expand the concept illustrated in Listings 6-8. Suppose that you had to write a test that required automobile data. You can create an object that describes an automobile and then populates a HashSet of such Automobile objects with data as you add each Automobile to the list (see Listing 9).

Then you can use getRandomItem() to return a random Automobile from that strong typed list as shown below in Listing 10.

Limitations and Gotchas

The QaRandomizer is not perfect; it's far from it. Many areas of the project will benefit from improvement. As such, there are some gotchas to consider when you use the tool:

• Iterations take time: Some tests are dedicated to performance: can your code run as expected within an expected time slice? Tests such as these are designed to run fast and furious. Thus, if you are writing tests that utilize randomization within an iterative scenario, particularly tests that iterate over code hundreds of times, you run the risk of polluting the validity of your other tests.

  To avoid this hazard and still allow your tests to have the time required to run their iterations, isolate tests that use iteration into a suite in a separate file. Doing so will allow more granular control over the entire testing scenario.

• Address data is real world for city, state, zip only: As mentioned previously, the QaRandomzer uses address data based on all the United States zip codes. The QaRandomizer gets a random zip code from this list and then, when the zip code is identified, looks up the corresponding city and state in the USPS list. However, address1 and address2 construction is fictitious. Thus, it is entirely possible to generate the following address:

  123 Elm Street
  Suite 400
  New York, NY 10001

  Be advised, there is no Elm Street in the zip code 10001. Thus, please no not use QaRandomizer.getAddress() with the expectation that you'll be able to deliver snail mail to that address.

• Randomization is useful, but there are times when you should use constant data: Using randomization in your unit tests is appropriate in most situations, but not in all situations. There will be very specific times when you should use constant data to make sure every base is covered. Then, by all means, do so.

Get Random for Your Code's Sake

Working with random data used to be more work than it was worth. However, using the QaRandomizer in conjunction with some of the practices that I've described in this article, you'll be able to bring the power of randomization to your testing practices easily. Hopefully, that will make your life easier and your code better.

So in the spirit of writing great code, go forward and get random!

Code Download