Difference between revisions of "Testing Tool (Architecture)"

Revision as of 14:08, 12 June 2008

Test system architecture

• separate system for executing tests
• system is only responsible for running tests, without determine whether a test fails or not -> compiling and launching system is done by tool, which is also oracle
• tests are compiled up to degree 3 in development system, so user can edit tests like normal classes, but do not affect resulting binary
• test referenced by root class of development system will be compiled anyway, this way we can also run tests in debugger
• in CDD: test system is implicit target (does not have to be in ecf) which inherits from development target, test target simply defines new root class/feature
• re-use development EIFGEN (copy) so test system does not need to compile from scratch?

Communication between tool and test executor

EiffelStudio  <----------------------------->  Testing tool  <----------------------------->  Test executor

* Show tests in system                         * can be part of EiffelStudio or               * execute test in safe environment
* Show test result                               compiled as separate tool to be                (executor is allowed to crash)
* Provide test creation wizards                  used e.g. through console
* Interface for CDD, Auto tests,               * compile test executor
  creating manual tests, running               * distribute test executors to
* provide ESF service for                        different machines
  testing/test results                         * schedule test execution
                                               * provide test results
                                               * find all tests for a given ecf file
                                               * write root class for test executor

CDD
* implemented partially in
  debugger/executable
* should be part of any Eiffel
  application, that way test can be
  created for bug submitting
* extraction can be initiated
  through debugger, breakpoints,
  failure window, etc.


Auto Test
* separate tool, interface in EiffelStudio

Protocol

From tool to executor

• name(s) of test to execute
• quit

From executor to tool

• test result
• text output produced by test
• exception details (type, tag, feature, class? occurred during set up, test, tear down?)
• call stack for exception

Open questions

• executor per machine/processor?
• text based/object base communication?

See also

• Testing Tool (Specification)