Testing Tool (Architecture)
Contents
Test system architecture
- separate system for executing tests
- system is only responsible for running tests, without determine whether a test fails or not -> compiling and launching system is done by tool, which is also oracle
- tests are compiled up to degree 3 in development system, so user can edit tests like normal classes, but do not affect resulting binary
- test referenced by root class of development system will be compiled anyway, this way we can also run tests in debugger
- in CDD: test system is implicit target (does not have to be in ecf) which inherits from development target, test target simply defines new root class/feature
- re-use development EIFGEN (copy) so test system does not need to compile from scratch?
EiffelStudio <-----------------------------> Testing tool <-----------------------------> Test executor
* Show tests in system * can be part of EiffelStudio or * execute test in safe environment
* Show test result compiled as separate tool to be (executor is allowed to crash)
* Provide test creation wizards used e.g. through console
* Interface for CDD, Auto tests, * compile test executor
creating manual tests, running * distribute test executors to
* provide ESF service for different machines
testing/test results * schedule test execution
* provide test results
* find all tests for a given ecf file
* write root class for test executor
CDD
* implemented partially in
debugger/executable
* should be part of any Eiffel
application, that way test can be
created for bug submitting
* extraction can be initiated
through debugger, breakpoints,
failure window, etc.
Auto Test
* separate tool, interface in EiffelStudio
Provide testing as a service in EiffelStudio
Communication between tool and test executor
Protocol
From tool to executor
- name(s) of test to execute
- quit
From executor to tool
- test result
- text output produced by test
- exception details (type, tag, feature, class? occurred during set up, test, tear down?)
- call stack for exception
Open questions
- executor per machine/processor?
- text based/object base communication?
