CSE687 - Object Oriented Design

Design Note #6
Distinguish between Construction, Unit, and Regression Testing

Design Note #6:

• Construction Test:
  

  Construction tests are quick little tests you add to a package's test stub - its main function guarded by a preprocessor switch. Each time you add a new function to a package, you immediately add a test to the test stub, build, and exercise the test. It's purpose is to make a quick estimate of correctness. You don't leave the function until it works. The idea is that your code is always working unless you are currently adding a new function.

  Do not write lots of functions before starting tests. That is a recipe for poor quality code that takes a long time to get operational. Some of you will not heed this advice. I'll see you late at night in the computer labs looking very frustrated and I'll walk by, shaking my head.

• Unit Tests:
  

  Correctness is important for all code, but especially so for library and support code that a lot of other code calls. It is the purpose of unit testing to attempt to ensure correct operation. Unfortunately you can't guarantee correctness by testing. All you can demonstrate is its absence.

  Unit tests are very detailed tests, run on a single package, that walk through every line of the code, exploring every path with lots of test inputs to gain confidence that the code satisfies its specification - the only means for establishing correctness. We do that with a debugger and with programmed sequences laid out in a test driver. The test driver is a separate file that implements test functions and has a main function that supplies inputs.

  It is important to do unit testing bottoms-up, starting with packages that don't depend on any non-tested code. That way it is very likely that errors are the result of the tested code, not code it calls.

  Good unit tests have a statement of what is tested with a brief procedure, and description of expected results. If we don't get the expected results we know that either the tested code or the test driver is incorrect. But we have a good basis for finding out which and fixing that.

  I make it a practice to put test description, procedure, and expected results in comments in each test driver I use for Unit test. The description, procedure, and results prediction are put where we would put manual and maintenance information in a package.

  You won't do much unit testing in this course, but will get a chance to do that in detail in CSE784 - Software Studio.

• Regression Tests:
  

  Regression tests are mid-level tests, similar to construction tests, that run autonomously and flag and log detected errors. These tests attempt to exercise large parts, perhaps all, of an application, running in a test harness. Their purpose is to detect when just added code breaks an existing baseline. The tests often build several different code configurations and report build errors and run-time errors. Essentially, regression testing, when effectively executed, constitutes a process of continuous integration of the code from all developers on all teams.

  We discuss regression testing in CSE681 - Software Modeling and Analysis. In CSE784 - Software Studio, the class conducts a very large qualification test. Regression testing is very similar to a completely automated qualification test.

  It takes a lot of expertise and lots of effective software automation and testing tools to make Regression testing effective. That's a big investment, but the payoff is more than worth the cost. We're very likely to find problems as soon as they occur instead of months after they've been added to the baseline. If errors go undetected for long periods of time we may have to rewrite lots of code to fix the problem. If we find problems right away, that isn't likely to happen.

Conclusions for: Distinguish between Construction, Unit, and Regression Testing