GUI Testing/Framework


The framework should facilitate retrieval of widgets and sending of events to a known GUI. This can be used to create a GUI test by hand.

Since it is not yet clear how to do the replay of an event sequence, the framework should use an abstraction layer to later change the way this is handled. The first implementation will consist of the simplest way where the executing test is compiled and run together with the application.

Vision2 Changes

In order to find widgets by name, widgets first need to have a name. Vision2 has to be extended accordingly by adding a name feature to the widgets. Proposed as follows:

      -- Parent of object, if any
  name: STRING
      -- Name of the widget
      -- If no specific name is set, the class type will be returned.
      result_not_void: Result /= Void
      result_not_empty: not Result.is_empty
  full_path: STRING
      -- Full name of widget by prepending names of parent widgets
      -- Uses '.' as a separator.
      result_not_void: Result /= Void
      result_correct: parent = Void implies Result.is_equal (name)
      result_correct: parent /= Void implies Result.is_equal ( + "." + name)
  set_name (a_name: STRING)
      -- Set `name' to `a_name'.
      name_not_void: name /= Void
      name_not_empty: not name.is_empty
      no_period_in_name: not a_name.has ('.')
      no_special_regexp_characters_in_name: -- TODO
      name_set: name = a_name

This will become an ancestor of EV_CONTAINABLE.

To find widgets when having their name the container class can be extended by searching functionality:

  find_first_widget_by_name (a_name: STRING): EV_WIDGET
      -- Find a widget in container with `a_name'.
      -- If no widget is found, Void is returned.
  find_first_widget_by_name_recursive (a_name: STRING): EV_WIDGET
      -- Find a widget in container with `a_name' and search recursive.
      -- Recursive search is a breadth-first.
      -- If no widget is found, Void is returned.
  find_first_widget_by_path (a_path: STRING): EV_WIDGET
      -- Find a widget which corresponds to `a_path'.
      -- The path will be split on periods and looked up accordingly.
  find_all_widgets_by_name (a_name: STRING): LIST [EV_WIDGET]
      -- TODO
      -- If no widget is found, an empty list is returned.
  find_all_widgets_by_name_recursive (a_name: STRING): LIST [EV_WIDGET]
      -- TODO
      -- If no widget is found, an empty list is returned.
  find_all_widgets_by_path (a_path: STRING): LIST [EV_WIDGET]
      -- TODO
      -- If no widget is found, an empty list is returned.

All strings can be regular expressions and will be matched accordingly.

wel and gtk

Both implementations of Vision2 - wel and gtk - have a support for names on the widget level. Thus it should not be a problem to add this functionality to Vision2.


Since the framework covers the replay part whose implementation is not yet defined, it should use an abstraction layer so tests can easily be adapted if necessary.

Common Functionality

  • Launch the application
    • If test and AUT are separate this needs to invoke a program by name
    • If test and AUT are compiled together, this needs to instantiate the original root class and call its creation feature. Threads could make a problem since the normal Vision2 application uses the thread for the event loop. So the test either has to create a thread for its actions or somehow be called from the Vision2 event loop.

Widget Retrieval

  • Get a widget
    • By name or path in the whole application
      this includes searching for windows and dialogs
    • By name or path in a specific widget
      this includes searching in a specific window or dialog
    • By class
    • By name or path and class
    • Maybe: By specifying other attributes
      A search for icon, size, position, activation status or other attributes can be imagined
    • Maybe: By specifying a prototype
      A widget is created and the attributes like size and icon are set. Then a widget with these properties is searched for.
  • Get a list of widgets
    • (Same as above but always return all widgets which conform to the query)
  • Get current focus
  • Get menu (easy access on menubar)

Event Execution

  • Invoke event
    • On the whole application
      mouse events with coordinates relative to application
      keyboard commands issued on current focus
    • On specific widget
      mouse events with coordinates relative to widget
      keyboard commands issued on specific widget (maybe focus should change automatically to better simulate behaviour)
  • Keyboard events
    • Issue a list of keyboard events by string (no need to issue every single key stroke)
    • Implement a way of describing which modifers are pressed
  • Mouse events
    • Clicks
    • Movement
    • Dragging
  • Mode where some events can be infered
    When two mouse clicks happen on different coordinates, the framework can issue mouse movement events between the positions to better simulate the user behaviour. This can be made as an option which can be activated or deactivated.

Class layout

Single Test

The class text of a single test should look similar to:

class Test
  application_root: APPLICATION_ROOT_CLASS
      -- Root class of application under test
  make is
      -- Execute test.
        -- launch application in a separate thread
        -- get EV_APPLICATION object
      set_ev_application (application_root)
        -- Just for illustration. Default values should alredy be 'good'
      mouse.set_click_delay (100)
      keyboard.set_typing_delay (50)
        -- run test procedure surrounded by rescue clause
      safe_run (agent run_test)
  launch_application is
      -- Launch application under test.
      create application_root.make
      wait (1000)
  run_test is
      -- Run test case.
      widget: EV_WIDGET
      widget := gui.widget_by_name ("my widget")
      mouse.left_click_on (widget)
      keyboard.type ("some text") (keys.Enter)
        -- close application
      mouse.left_click_menu ("File.Exit")
        -- other way to close application
      keyboard.press_modified (modifiers.Control, keys.Enter)



Helper class for mouse events. Dragging support still needs to be included.

class MOUSE
feature -- Button clicking
  click (button, click_count, x, y)
    -- Generic click. Used by other features.
  click_on (button, click_count, widget)
    -- Generic click. Used by other features.
  left_click_on_position (x, y)
  right_click_on_position (x, y)
  middle_click_on_position (x, y)
  left_click_on (widget)
  right_click_on (widget)
  middle_click_on (widget)
  left_multi_click_on_position (x, y, click_count)
  right_multi_click_on_position (x, y, click_count)
  middle_multi_click_on_position (x, y, click_count)
  left_multi_click_on (widget, click_count)
  right_multi_click_on (widget, click_count)
  middle_multi_click_on (widget, click_count)
feature -- Button pressing
  button_down (button, x, y)
    -- Generic button down. Used by other features.
  button_up (button, x, y)
    -- Generic button up. Used by other features.
  left_button_down_on_position (x, y)
  left_button_down_on_widget (widget)
  left_button_up_on_position (x, y)
  left_button_up_on_widget (widget)
feature -- Moving
  move_to_position (x, y)
  move_to (widget)
feature -- Scrolling

Helper class for keyboard input.

  type (string)
  type_modified (modifiers, string)
  type_into (widget, string)
  type_modified_into (widget, modifiers, string)
  press_key (key)
  release_key (key)
  press_modifier (modifier)
  release_modifier (modifier)

Helper class to retreive widgets. The features which retreive widgets will have contracts to always return a non-void result. This is because if you have a test case and specify to get a widget with a certain name it has to be there or else the test case will fail. The error should be presented nicely so the test case can be adapted at the erroneous position. It would be good if in the case of an unknown widget the whole GUI is searched for the name to supply some suggestions as to how correct the test case.

class GUI
  widget_by_name (name): EV_WIDGET
  widgets_by_name (name): LIST [EV_WIDGET]
  ... (see Vision2 changes above - the same features will be present here)
Test Case

Helper class for test case.

  ev_application: EV_APPLICATION
    -- Application under test
  set_ev_application (an_app)
    -- Set `ev_application' to `an_app'.
  gui: GUI
    -- Helper class to access Vision2 widgets
  mouse: MOUSE
    -- Mouse interface
  keyboard: KEYBOARD
    -- Keyboard interface
  keys: KEYS
    -- Helper class to keys enumeration
  modifiers: MODIFIERS
    -- Helper class to keyboard modifiers
  buttons: BUTTONS
    -- Helper class to mouse buttons enumeration
GUI lookup

In order to retreive a widget, a special notation is used:

Find a widget anywhere in the GUI-tree:

 gui.widget ("name")

Find a widget with a specific direct parent anywhere in the GUI-tree:

 gui.widget ("")

Find a widget with a specific direct or indirect parent anywhere in the GUI-tree:

 gui.widget ("")

Find a widget with a specific type:

 gui.widget ("{TYPE}")

Find a widget with a specific type and name:

 gui.widget ("{TYPE}name")

Find a widget on a specifi window:

 gui.widget ("window:name")

Combination of all of the above

 gui.widget ("{WINDOW_TYPE}window:indirect_parent..{TYPE}")