Lecture #22 - Test Harness

Project4HelpF15

Mediator based Dispatcher: Class Diagram, Object Diagram
This flexible design for server functionality is more than needed for Project #4.

Simplified Test Harness (no Child AppDomain) - code

Test Harness Prototype - Block Diagram, vsdx
Test Harness - Class Diagram, vsdx
Test Harness - Structure Chart, vsdx

Test Harness Prototype - code
Look carefully at TestConfigure.cs

Testing:

• Construction Tests:
  • Purpose: Support building functioning packages
  • Method: Include guarded main function. As each method in the package is constructed one or more tests are created and run in the main function. This should lead to a style of development where every few lines of code are compiled and tested before adding more code. So there is a continuing sequence of adding a little code, building, and running to make sure the code is always functional. This should start very early in the coding phase.
  Unit Tests:
  • Purpose: Verify Correctness
  • Method: Detailed inspection and debugger walkthrough of every line of code, toggeling predicates, inspecting data, and exploring "corner" cases. This is a very expensive test and isn't applied to every package, but is applied to packages with many dependent parent packages.
  Integration Tests:
  • Purpose: To ensure that large software parts - subsystems - get assembled correctly.
  • Method: Assemble parts, usually in some form of test harness, build and run. Check that all data flows are as expected, that messages and notifications are issued at appropriate times, that exception handling works as expected. Essentially we run Integration Tests to ensure that the software cycles as it should.
  Regression Tests:
  • Purpose: Perform fairly quick test of an entire baseline.
  • Method: Build a detailed set of low, mid, and high-level tests that are run in sequence to give quick confirmation that code additions and changes in platform have not broken any of the parts.
  Stress Tests:
  • Purpose: Ensure that the system does not crash or behave erratically when tested against very large loads. The system may not be expected to meet specifications under huge loads, but it should recover and operate as specified when the load returns to the specified range.
  • Method: Design tests that are CPU intensive, read/write intensive, involve a lot of data transfer, may block waiting for resources, etc. Run multiple concurrent clients against server measuring throughput and latency.
  Qualification Tests:
  • Purpose: Verify that the development has met its obligations. This is a "delivery" test.
  • Method: Run a large suite of requirements tests - one or more for each requirement (usually hundreds). This is a very long process and needs to be automated. The customer and development team should examine each requirement, look at the code that implements the requirement, and examine recorded and live data that demonstrates the requirement is met.

C++ Regression tests of a FileSystem package

Attributed Programming Demo

Interception Presentation

Discussion of Managed Contexts

Synchronization Context

Serialization Demo

Review of Object Models