Software Modeling & Analysis - Fall 2016

Project #4 - Remote Test Harness

Version 2.3,
Due Date: Wednesday, November 16th

Purpose:

In this project we will develop an implementation for accessing and using a remote Test Harness server from multiple concurrent clients. The Test Harness will retrieve test code from a Repository server1.

An important part of this project is to assess the performance of the testing and communication as the number of concurrent reader and writer clients increases.

Requirements:

Your Remote Remote Test Harness project:
  1. (1) Shall be implemented in C# using the facilities of the .Net Framework Class Library and Visual Studio 2015, as provided in the ECS clusters.
  2. (2) Shall implement a Test Harness Server that accepts one or more Test Requests, each in the form of an a message with XML body that specifies the test developer's identity and the names of a set of one or more test libraries to be tested. Each test driver and the code it will be testing is implemented as a dynamic link library (DLL) and sent by the client to the Repository server2 before sending the Test Request to the Test Harness. The Test Request XML body names one or more of these test DLLs to execute.
  3. (1) If a Test Request specifies test DLLs not available from the Repository, the Test Harness server will send back an error message to the client.
  4. (2) The Test Harness shall enqueue Test Requests from multiple concurrent clients and execute them by creating, for each Test Request, an AppDomain, running on its own thread. Once a child AppDomain is constructed, the Test Harness shall start the child processing the dequeued Test Request. The result is that Test Requests can be processed concurrently.
  5. (1) Each test driver derives from an ITest interface that declares a method test() that takes no arguments and returns the test pass status, e.g., a boolean true or false value.
  6. (2) Test libraries and Test Requests are sent to the Repository and Test Harness server, respectively, and results sent back to a requesting client, using an asynchronous message-passing communication channel3. The Test Harness receives test libraries from the Repository using the same communication processing. File transfer shall use streams or a chunking file transfer that does not depend on enqueuing messages4.
  7. (2) At the end of test execution the Test Harness shall store the test results and logs in the Repository5 and send test results to the requesting client. It then shall unload the child AppDomain responsible for that testing.
  8. (1) The Test Harness shall, in cooperation with the Repository, store test results and logs for all of the test executions using a key that combines the test developer identity and the current date-time.
  9. (1) The Repository shall support client queries about Test Results from the Repository storage.
  10. (2) All communication between Clients, the Test Harness, and the Repository, shall be implemented using Windows Communication Foundation (WCF) channels, passing messages that contain requests, results, and notifications.
  11. (2) Clients shall provide a Graphical User Interface constructed with Windows Presentation Foundation (WPF). In order to demonstrate that you meet requirements you will probably need an interface for the Test Harness and for the Repository. These may be Console interfaces if you wish.
  12. (1) Shall include means to time test executions and communication latency.
  13. (1) Shall be accompanied by a test executive that clearly demonstrates you've met all the functional requirements #2-#10, above. If you do not demonstrate a requirement you will not get credit for it even if you have it correctly implemented.
  14. (1) Shall contain a brief text document that compares this implementation with the concept developed in Project #3. Does this project fully implement its concept? Was the original concept practical? Were there things you learned during the implementation that made the origninal concept less relevant or incomplete?

  1. The Repository simply manages storage of code and logs. It is not expected to contain dependency information, provide for check-in and check-out, nor is it expected to provide version management, other than to use DateTime stamps and author name to retrieve code and logs. Note that a full implementation of a Repository would need to provide these functionalities. Note also that it would make a lot of sense for the Repository to store TestRequests as well, since they will be run again later when the code that they depend on changes to fix latent errors, add new functionality, or fix performance problems.
  2. The client can send the test code to the repository at any time prior to sending a Test Request, that cites that code, to the Repository.
  3. You will implement an asynchronous communication channel that supports Test Harness callbacks when test results are ready, so the client will simply make a request and the server will communicate back the result when it is ready, without the client blocking, awaiting the result.
  4. This means that your communication service will provide message-passing for sending requests and results, and file streaming or file chunking for sending binary files.
  5. Note that the test harness calls test() and stores the test result and test log in the Repository. It then composes a response message with the test results and sends that to the requesting client.
  6. You may, but are not required to, use an instance of MongoDB to manage stored information in the Repository.

What you need to know:

In order to successfully meet these requirements you will need to:
  1. Write C# code and use advanced facilities of the .Net Framework.
  2. Understand .Net threading docs and Threading Examples.
  3. Use WCF example code and the WCF Docs.
  4. Use WPF example code and the WPF Docs.