Difference between revisions of "Testing Tool (Architecture)"

(Scheme of different components needed for test creation/execution)
m (Moved missplaced title)
Line 9: Line 9:
 
* in CDD: test system is implicit target (does not have to be in ecf) which inherits from development target, test target simply defines new root class/feature
 
* in CDD: test system is implicit target (does not have to be in ecf) which inherits from development target, test target simply defines new root class/feature
 
* re-use development EIFGEN (copy) so test system does not need to compile from scratch?
 
* re-use development EIFGEN (copy) so test system does not need to compile from scratch?
 
== Communication between tool and test executor ==
 
  
 
<pre>
 
<pre>
Line 42: Line 40:
  
 
</pre>
 
</pre>
 +
 +
== Communication between tool and test executor ==
  
 
=== Protocol ===
 
=== Protocol ===

Revision as of 15:57, 12 June 2008


Test system architecture

  • separate system for executing tests
  • system is only responsible for running tests, without determine whether a test fails or not -> compiling and launching system is done by tool, which is also oracle
  • tests are compiled up to degree 3 in development system, so user can edit tests like normal classes, but do not affect resulting binary
  • test referenced by root class of development system will be compiled anyway, this way we can also run tests in debugger
  • in CDD: test system is implicit target (does not have to be in ecf) which inherits from development target, test target simply defines new root class/feature
  • re-use development EIFGEN (copy) so test system does not need to compile from scratch?

EiffelStudio  <----------------------------->  Testing tool  <----------------------------->  Test executor

* Show tests in system                         * can be part of EiffelStudio or               * execute test in safe environment
* Show test result                               compiled as separate tool to be                (executor is allowed to crash)
* Provide test creation wizards                  used e.g. through console
* Interface for CDD, Auto tests,               * compile test executor
  creating manual tests, running               * distribute test executors to
* provide ESF service for                        different machines
  testing/test results                         * schedule test execution
                                               * provide test results
                                               * find all tests for a given ecf file
                                               * write root class for test executor

CDD
* implemented partially in
  debugger/executable
* should be part of any Eiffel
  application, that way test can be
  created for bug submitting
* extraction can be initiated
  through debugger, breakpoints,
  failure window, etc.


Auto Test
* separate tool, interface in EiffelStudio

Communication between tool and test executor

Protocol

From tool to executor

  • name(s) of test to execute
  • quit

From executor to tool

  • test result
  • text output produced by test
  • exception details (type, tag, feature, class? occurred during set up, test, tear down?)
  • call stack for exception

Open questions

  • executor per machine/processor?
  • text based/object base communication?

See also