Supervised by Dr. James W. Fawcett | ||||
|
VERDICTS is a prototype tool to demonstrate a new approach to combining software
comprehension and testing by visualization and incremental discovery and addition of
requirements to existing software.
VERDICTS gathers live program execution data and interacts with the
target program to help developers analyze the aggregate behavior by viewing,
statistically analyzing, and testing multiple calls to each function traced.
As this approach is closer to execution tracing rather than debugging,
even though there are some parallels, there is no direct quantitative
comparison between VERDICTS and a standard IDE (Integrated
Development Environment).
Unfortunately this happens more often than we would like to admit, because cutting corners is efficient in the beginning, by skipping large-scale analysis, design and documentation and jumping straight into writing code. As there is no big-picture view of the system, new developers cannot easily comprehend software as a whole, due to:
Making a part of software reusable, ideally by compiling many related reusable functions into a library, requires much effort, especially to test these functions thoroughly to make them more generally applicable, as would be required of a reusable software component.
Opportunistic Software Reuse refers to reuse of parts of software that were not designed or documented for the purpose of reuse.
If you are working on an open-source project, you can reuse parts of other open source projects available on the internet, using public code search engines such as Koders and Google Code Search. For proprietary software development, you may have access to a large company code base, which may again be accessible through enterprise versions of code search engines.
As there is no universal specification language, these code search engines do text-search in source code, which often retrieves many false positive source code files.
Similar to exploratory testing ideas, VERDICTS allows exploratory discovery of how a software component operates. A VERDICTS session allows the developer to better understand the software component while generating a reusable specification for the component. Actually, evolving the specification is how the developer tests different ideas about the operation of the component. VERDICTS can generate unit tests based on the discovered specifications and trace data collected in the VERDICTS session. The unit tests (which contain specific component input data), and the more generally applicable specifications represent both human-readable documentation of component behavior as well as executable tests to verify component correctness.
Interactive live contract-setting allows new paradigms. The contract may represent:
For each tracing session, the user selects the methods to be traced, starts the session (starts recording), interacts with the target program (without having to ever stop the program), and stops the session. Then, all the method calls, their arguments (as well as return type and contract-specific user-defined observables), and whether they passed or failed the expectations set by the contracts can be viewed as seen here.
Aggregate behavior of methods can be observed by looking at correlations between the observables (arguments, return type and user defined variables that are of primitive type) as well as plots between pairs of observables. The patterns observed suggest new contracts that can be declared either to tighten specifications or to create hypotheses to better understand actual program behavior.