Apache Jakarta and Beyond: A Java Programmeramp;#039;s Introduction [Electronic resources] نسخه متنی

4.4. How and What to Test

JUnit itself is a fairly simple system, as the preceding sections have shown. In a sense JUnit is little more than a handful of utility classes to simplify the creation of tests and another set of classes to run tests and display the results. The secret to successful and useful testing lies not in an overly elaborate, overly complex testing framework but in how the framework is used. The question of how to test effectively is surprisingly complex, and there is not room for a full treatment here. Although it is certainly possible to run useful automated testing without subscribing to any design methodologies, such testing is often considered in the context of what is called extreme programming. There are many books and Web sites on extreme programming, each of which will address the issue of effective testing.

In lieu of a full analysis, here are general guidelines that should be useful.

Test every class.

Every class should have a corresponding test class that will typically live in a parallel directory. For example, tests for classes in the com.awl.toolbook. chapter04 package should live in tests.com.awl.toolbook.chapter04. This makes it very easy for Ant to omit these directories when bundling the system for deployment; just add excludes=" tests/*" to the appropriate target.

These test classes should have at least one test for every public method in the class they are meant to test. There are limits to this; conventional wisdom is that simple accessors do not need to be tested. Any method that may be considered part of the API exposed by that class should definitely be tested.

Write tests before writing code.

This may seem backward, but it makes a great deal of sense. Writing the tests first will require thinking through issues such as what methods should be public, what combinations of values each method should expect, how malformed or invalid input should be handled, and so on. It is certainly possible to think through these issues without writing tests, but the advantage of doing it as part of the testing process is that the tests become both a formal specification and a means to ensure that the specification has been satisfied.

Add a test whenever a bug is found.

Whenever a bug has been found the first step should be to create a new test that exhibits the bug. Then run the test suite and watch the JUnit progress bar turn red; this will confirm that the error condition has been captured. When the bug is fixed and the bar turns green, there is a level of assurance that that particular problem will never plague the system again.

Note that a bug may manifest at several layers of the code. At the outermost layer, an end user has probably provided some input and received an incorrect response, which may take the form of an incorrect value or an exception. This condition should be tested. In addition the bug will ultimately be the result of incorrect code in one or more methods of one or more classes. Each of these methods should have a new test or an extension of an existing test that exhibits the problem.

Adding tests when bugs are found is a good way to introduce automated testing to existing systems for which no tests were initially written. It can be a major hassle to go back and revisit large amounts of code to add tests (another reason to write tests before code). Adding tests for each bug will slowly build up a respectable test suite, and the parts of the system that are working don't pressingly need tests.

Testing saves time!

Often developers under a tight schedule feel they do not have time to write lots of extra test classes. However, even in the absence of automated testing tools the code still needs to be tested. Doing it manually every time a change is made cannot possibly be faster in the long run.

More importantly, it is very common in large systems that a fix for one bug will introduce another. This can result in a horrendous "one step forward, two steps back" mode of development, which always seems to happen just before a project is due. With automated testing there is much more of a guarantee that changes will not make anything worse.

Test early and often.

It is better to change one line of code and discover that one test has broken than to spend six hours writing code to discover that the new code has broken twenty tests. In the latter case it will be very difficult to back trace through the code to find the cause of each failure, whereas in the first case it is easy to track the influence of the one modified line.

Compilation and testing take time, and if development really had to pause to test after every line, nothing would get done. A balance between the overhead of compilation and testing and the need to keep development momentum going is needed, and it will come with experience.

Tests should be fast.

This is a corollary of the previous point. A test that takes one second will get run more than sixty times as often as one that takes a minute. Using the GUI version of JUnit can help here; by clicking "Reload classes every run" and leaving the GUI active, the time needed to restart JUnit is eliminated.

Select good sample values.

There is not much to be gained by testing Factorial on both 10 and 11. If one of these works, the other will as well due to the nature of the function. On the other hand, the correct behavior of Factorial for numbers greater than one is very different from correct behavior for numbers less than one. This is a specific instance of the more general question of choosing appropriate test values.

One extreme position, called white-box testing, advocates writing tests that are guaranteed to exercise every line of code. For example, if a conditional branches based on whether some variable is less than 10, then a white-box test would have one test where the variable is less than 10 and another where the variable is 10 or greater. This can hugely multiply the number of tests, ^[2] which can significantly slow down the testing process. This also breaks the rule about writing tests before code because it requires testing based on the implementation of the class rather than the specified behavior.

^[2] Roughly, there are likely to be 2

n tests, where

n is the number of conditionals.

Generally, although not always, it is reasonable to think in terms of representative values. Consider what kinds of values will give different kinds of results. Also consider what values are "pathological" and should be trapped, such as values that will result in a division by zero or indexing into an empty array.

Each test should leave a clean slate.

Recall that JUnit does not guarantee the order in which tests will be run. Consequently each test needs to be responsible for setting up any state that it needs to test against and should clean up after itself.

Testing database code requires special approaches.

The preceding requirement can be especially problematic when databases are involved, in which case the full state may consist of every row in every table. Problems may manifest only when there are exactly 100 rows of data or other equally specific conditions.

Such situations may be difficult to test for a number of reasons. If an entire development staff is sharing a database, it may be impossible to keep the database quiet long enough to run the tests. If a test tries to put itself into a clean state, it may wipe out large amounts of data needed by another developer.

There is no solution to this problem that will work in all cases. One possibility is to give each developer his or her own database to test against, possibly using a small simple database like Hsqld (see Chapter 10). Note, however, that testing against a database other than the one used in production may result in code that works in development but fails in production.

The problem of requiring specific sets of test data can be addressed by storing data files external to the test and having the test call out to a database utility to load data from a file.

Another possibility is to take the program as far as generating the SQL (Structured Query Language) that will be issued to the database and checking that SQL against expected strings without necessarily executing it.