GUI Capture (Developement)
Contents
Overall structure
Platform independence
For the output to remain platform independent from the input, the engine passes the events in a uniform format to the output which can use the process_xy_event features inherited from the CAPTURE_EVENT_VISITOR class to generate the output. The input is generated by the engine implementation which is specific for the platform used.
Event Hierarchy
To allow different output formats, the events types are equiped with a process feature which implements the visitor pattern together with the CAPTURE_EVENT_VISITOR class.
Event Data
The event has to store the necessary data to replay it (raw data) but should also store additional information like the widget it happend on. This could be done by giving all widgets (or just the important ones) a name and storing that name along with the event data.
The widget which was accessed has to be accessed through Vision2 which poses some problems for the capturing process on how to communicate with the Vision2 application.
All Events:
- Vision2 widget name (if applicable)
All Mouse Events:
- x, y coordinate
Mouse Click Event:
- Button state change
All Keyboard Events:
- Key Code
- Key state change
Windows Implementation
Event Hook
To get the events from another window, a hook can be installed. The hook is installed through a custom DLL. All messages which are sent to the window are then stored by the dll and can be polled by the capturing application.
How to get the widget name for an event is not yet clear. Since the name is only known in Vision2 the capturing application has to somehow call a feature on the window. This could probably be done by sending a special message to the window (through the windows message queue) which is then catched by Vision2. The name is then sent back by sending a message back through the windows message queue to the capturing application.
