Supervised by Dr. James W. Fawcett
Kanat Bolazar VERDICTS Mutation Testing (MT)

Welcome to Kanat Bolazar's Research Page

Research Interests

• software comprehension
• requirements discovery, requirements maintenance
• unit testing and verification

I am also working on questions of assessing specification quality. When specifications are tightly coupled with test cases, specification adequacy is highly correlated with test set adequacy. Test set adequacy can generally be answered with test coverage metrics such as branch coverage, path coverage and mutation testing metrics.

VERDICTS

VERDICTS is a prototype tool to demonstrate a new approach to combining software comprehension and testing by visualization and incremental discovery and addition of requirements to existing software.

• Automated: Requirements are used to automate erroneous behavior detection.
• Visual: Visualization is used to aid comprehension by providing abundant information in a form that makes it easier to detect common patterns.
• Live: In one session, new requirements can be discovered, injected into the software, verified (checked against the behavior of the program), and modified without recompilation or restarting the program under test.

VERDICTS gathers live program execution data and interacts with the target program to help developers analyze the aggregate behavior by viewing, statistically analyzing, and testing multiple calls to each function traced. As this approach is closer to execution tracing rather than the debugging, there is no direct quantitative comparison between VERDICTS and a standard IDE (Integrated Development Environment).

Publications

We introduce a methodology and its prototype implementation for visual exploratory software analysis. VERDICTS combines exploratory testing, tracing, visualization, dynamic discovery and injection of requirements specifications into a live quick-feedback cycle, without recompilation or restart of the system under test. This supports discovery and verification of software dynamic behavior, software comprehension, testing, and locating the defect origin. Our prototype implementation uses aspect-oriented method call interception, functional specification of requirements using dynamically injected Design by Contract, statistical analysis and various forms of visualization. We report on our personal experience.

S. Kanat Bolazar, James Fawcett, "Debugging with Software Visualization and Contract Discovery" in Proceedings of 15th International Conference on Software Engineering and Data Engineering, SEDE 2006, 47-50. (local copy)

Despite advances, debugging remains a drudgery, especially for software without proper documentation. Both top-down visualization-based approaches and bottom-up programmatic automation can improve this process. For a given system under test (SUT), we propose a combined method of:

• Program trace visualization, with capability to zoom in to method call details (inputs and outputs) Interactive declaration of observables for the internal state of the system
• Statistical and visual analysis of the collected observable data
• Programmatic declaration of expected behavior of the system defined through invariant relationships and satisfaction of contracts (Design by Contract) on these observables
• The ability to modify and repeat these steps without restarting the SUT

We report that instrumentation necessary to collect trace data is feasible; large amounts of data can be gathered without significant performance penalty while the visualizer remains responsive to tester interaction. Our personal experience is that the system is very quick to set up, faults are discovered quickly, and inefficient algorithms (which may produce correct results) become obvious through the visualizer. We are setting up human- interaction experiments to support the claim that our approach improves the efficiency of discovery of fault origin for a given system failure, compared to using a state-of-the-art debugger for Java (Eclipse).

Steven Morphet, James Fawcett, Kanat Bolazar and Murat Gungor, "Neural Net Analysis of the Propensity for Change in Large Software Systems" in Proceedings of 2006 International Joint Conference on Neural Networks, IJCNN 2006, part of IEEE World Congress on Computational Intelligence, WCCI 2006, 2606-2610. (local copy, IEEE explore link)

A novel approach for analyzing the relationship between code metrics and change count histories is presented. Specifically, neural networks are employed to determine a mapping between metrics and change count. While these neural networks can be trained to a high degree of accuracy, their internal workings remain opaque to the user. As such, a fuzzy modeling approach is additionally employed to generate the rules governing the neural computation. These rules are linguistic in nature and are thus more easily interpreted by software project managers. Application of this method to Mozilla change data reveals the importance of fan-out, total lines of code and maximum cyclomatic complexity metrics in predicting amount of change per file.

James Fawcett, Murat Gungor, Arun Iyer, Kanat Bolazar, "Relationship Between Code Metrics And Change History" in Proceedings of 20th International Conference on Computers and Their Applications, CATA 2005, 354-360. (local copy)

This paper studies relationships between code metrics and change count histories for a large open-source project, Mozilla. We examine several structural and code property metrics and construct statistically significant relationships with code base changes. Our results provide one step toward tool support for software managers to detect problems in large development projects.

DOPL at Syracuse University

DOPL is part of the:

• Department of Electrical Engineering and Computer Science (EECS)
• at the L.C. Smith College of Engineering and Computer Science (ECS),
• at Syracuse University (S.U.).

We are located at:

Center for Science and Technology 4-116
Syracuse University
Syracuse, NY 13244