Difference between revisions of "GUI Replay"
m (→Overview) |
m (→Windows messaging system) |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 12: | Line 12: | ||
===Checking assertions=== | ===Checking assertions=== | ||
| − | To check assertions the test has to access the current state of the AUT. In Eiffel this is a problem since no mechanism for this exist. | + | To check assertions the test has to access the current state of the AUT. In Eiffel this is a problem since no mechanism for this exist. Multiple solutions to check assertions exist: |
| − | + | ====Use Debugger==== | |
| + | |||
| + | By using the same method as the debugger, the test could read the current state of the AUT. The test could issue commands on the AUT and call queries to check assertions. For this to work the AUT has to be compiled in workbench mode, but it can be compiled separate from the test. | ||
| + | |||
| + | ====Compile test into AUT==== | ||
| + | |||
| + | Another way would be to compile the whole test case - event sequence and assertions - into the AUT. When the test is run it starts the AUT and replays the event sequence and checks the assertions. But for every test case or group of test cases the AUT has to be compiled anew. | ||
| + | |||
| + | ====Interprocess communication==== | ||
| + | |||
| + | If some means of interprocess communication exist, one could think of a protocol to get the current state of the AUT. The test could use this to check the assertions. | ||
| + | |||
| + | ====Windows messaging system==== | ||
| + | |||
| + | On Windows one can retreive data from another window by sending it a message (e.g. WM_GETTEXT to an edit field). Through this mechanism you could retreive the data from single widgets. Unfortunatly only native Windows widgets are supported by this. For example the EiffelStudio editor is natively a drawing panel and thus no meaningful data can be retreived from it. | ||
| + | |||
| + | ====Screenshot comparison==== | ||
| + | |||
| + | Another approach of test assertion is to create a screenshot when capturing the test case and then compare it to the result after replaying the events. | ||
| + | |||
| + | ====Comparison of methods==== | ||
{| border="1" cellpadding="5" cellspacing="0" align="center" | {| border="1" cellpadding="5" cellspacing="0" align="center" | ||
| Line 21: | Line 41: | ||
! Method !! Pro !! Contra | ! Method !! Pro !! Contra | ||
|- | |- | ||
| − | | Use debugger | + | | Use debugger |
| Testprogram and AUT are completly separate | | Testprogram and AUT are completly separate | ||
| Can only debug non-finalized applications | | Can only debug non-finalized applications | ||
|- | |- | ||
| − | | Compile test | + | | Compile test into AUT |
| Easy to access state of AUT | | Easy to access state of AUT | ||
| Each test (or a group of tests) has to be compiled together with AUT | | Each test (or a group of tests) has to be compiled together with AUT | ||
|- | |- | ||
| − | | | + | | Interprocess communication |
| Testprogram and AUT are completly separate | | Testprogram and AUT are completly separate | ||
| Unclear how to provide interprocess communication in Eiffel | | Unclear how to provide interprocess communication in Eiffel | ||
| + | |- | ||
| + | | Windows messaging system | ||
| + | | Testprogram and AUT are completly separate | ||
| + | | Unable to retreive data from all Vision2 widgets | ||
| + | |- | ||
| + | | Screenshot comparison | ||
| + | | Testprogram and AUT are completly separate | ||
| + | | Very vulnerable to layout changes | ||
|} | |} | ||
Latest revision as of 09:32, 2 November 2006
Contents
Overview
In the replay phase a recorded sequence of events is replayed and the assertions are checked. The desired behaviour is as follows:
- The test cases can be compiled separate from the AUT.
- The AUT can be in finalized mode.
- Changes to the layout in the AUT don't have an impact on many tests, or at least is adapted easily or automatic.
Problems
Checking assertions
To check assertions the test has to access the current state of the AUT. In Eiffel this is a problem since no mechanism for this exist. Multiple solutions to check assertions exist:
Use Debugger
By using the same method as the debugger, the test could read the current state of the AUT. The test could issue commands on the AUT and call queries to check assertions. For this to work the AUT has to be compiled in workbench mode, but it can be compiled separate from the test.
Compile test into AUT
Another way would be to compile the whole test case - event sequence and assertions - into the AUT. When the test is run it starts the AUT and replays the event sequence and checks the assertions. But for every test case or group of test cases the AUT has to be compiled anew.
Interprocess communication
If some means of interprocess communication exist, one could think of a protocol to get the current state of the AUT. The test could use this to check the assertions.
Windows messaging system
On Windows one can retreive data from another window by sending it a message (e.g. WM_GETTEXT to an edit field). Through this mechanism you could retreive the data from single widgets. Unfortunatly only native Windows widgets are supported by this. For example the EiffelStudio editor is natively a drawing panel and thus no meaningful data can be retreived from it.
Screenshot comparison
Another approach of test assertion is to create a screenshot when capturing the test case and then compare it to the result after replaying the events.
Comparison of methods
| Method | Pro | Contra |
|---|---|---|
| Use debugger | Testprogram and AUT are completly separate | Can only debug non-finalized applications |
| Compile test into AUT | Easy to access state of AUT | Each test (or a group of tests) has to be compiled together with AUT |
| Interprocess communication | Testprogram and AUT are completly separate | Unclear how to provide interprocess communication in Eiffel |
| Windows messaging system | Testprogram and AUT are completly separate | Unable to retreive data from all Vision2 widgets |
| Screenshot comparison | Testprogram and AUT are completly separate | Very vulnerable to layout changes |
Layout changes
To be able to react on layout changes in the AUT, a test case should not consist of a series of events on certain positions, but rather as a series of events on certain widgets. If a widget can be identified by its name (or underlying command) the change of position would not affect a test case. For that to work, all widgets or commands should have a name (which does not necessarily has to be unique).
