GUI Replay


Overview

In the replay phase a recorded sequence of events is replayed and the assertions are checked. The desired behaviour is as follows:

  • The test cases can be compiled separate from the AUT.
  • The AUT can be in finalized mode.
  • Changes to the layout in the AUT don't have an impact on many tests, or at least is adapted easily or automatic.

Problems

Checking assertions

To check assertions the test has to access the current state of the AUT. In Eiffel this is a problem since no mechanism for this exist. Multiple solutions to check assertions exist:

Use Debugger

By using the same method as the debugger, the test could read the current state of the AUT. The test could issue commands on the AUT and call queries to check assertions. For this to work the AUT has to be compiled in workbench mode, but it can be compiled separate from the test.

Compile test into AUT

Another way would be to compile the whole test case - event sequence and assertions - into the AUT. When the test is run it starts the AUT and replays the event sequence and checks the assertions. But for every test case or group of test cases the AUT has to be compiled anew.

Interprocess communication

If some means of interprocess communication exist, one could think of a protocol to get the current state of the AUT. The test could use this to check the assertions.

Windows messaging system

On Windows one can retreive data from another window by sending it a message (e.g. WM_GETTEXT to an edit field). Through this mechanism you could retreive the data from single widgets. Unfortunatly only native Windows widgets are supported by this. For example the EiffelStudio editor is natively a drawing panel and thus no meaningful data can be retreived from it.

Screenshot comparison

Another approach of test assertion is to create a screenshot when capturing the test case and then compare it to the result after replaying the events.

Comparison of methods

Assertion checking methods
Method Pro Contra
Use debugger Testprogram and AUT are completly separate Can only debug non-finalized applications
Compile test into AUT Easy to access state of AUT Each test (or a group of tests) has to be compiled together with AUT
Interprocess communication Testprogram and AUT are completly separate Unclear how to provide interprocess communication in Eiffel
Windows messaging system Testprogram and AUT are completly separate Unable to retreive data from all Vision2 widgets
Screenshot comparison Testprogram and AUT are completly separate Very vulnerable to layout changes

Layout changes

To be able to react on layout changes in the AUT, a test case should not consist of a series of events on certain positions, but rather as a series of events on certain widgets. If a widget can be identified by its name (or underlying command) the change of position would not affect a test case. For that to work, all widgets or commands should have a name (which does not necessarily has to be unique).