Revision as of 15:28, 7 May 2007

Summary

CDD is an extension to EiffelStudio that provides support for both automatic test case extraction and test execution. It is undisruptive to the development process of the user. The tool is in its early stages. Feedback and bug reports are very much appreciated. Please send them to mailto:andreas.leitner@inf.ethz.ch and or use the discussion page for this article.

Installation

Requirements

• ISE Eiffel 5.7 (below version is recommended, other versions may or may not work)
  • Linux-x86: http://se.ethz.ch/people/leitner/cdd/Eiffel57_gpl_64493-linux-x86.tar.gz
  • Win32: http://se.ethz.ch/people/leitner/cdd/Eiffel57_gpl_64493-windows.msi

Install CDD-Extension for EiffelStudio

• Linux
  • Delete the directory $HOME/.es
  • Download http://se.ethz.ch/people/leitner/cdd/cdd_linux_x86_0_2_0.tgz
  • Unpack it over your existing EiffelStudio installation. (You have done it right when you are overwriting a few files in the process.)

• Windows
  • Start the registry editor (regedit.exe) and delete the key "HKEY_CURRENT_USER\Software\ISE\Eiffel57"
  • Download http://se.ethz.ch/people/leitner/cdd/cdd_win32_0_2_1.zip
  • Unpack it over your existing EiffelStudio installation. (You have done it right when you are overwriting a few files in the process.)

How to Use

Make sure you also have assertion checking enabled. This can be done easily in: "Project->Project Settings..." select assertions and select "True" for all of them (the more the better cdd will work).

Step 1: Show Testing Window

If not already visible, you need to make sure the testing windows are all properly displayed. You can make them appear by following the menu "View->Tools->Testing":

Step 2: Enable CDD-Testing

To enable CDD-testing for your project, open your project and then enable CDD-testing, by clicking on the half-black / half-white button in the testing window. You can disable testing again at any time by pressing this button again.

Now continue developing your project as usual. You will notice that for every exception that occurs a new test case shows up in the testing window. Test cases are automatically executed on each compilation to see whether you fixed that particular bug or not. A test case can be in one of three states:

• "F" for failing - last time the test case was executed the original problem was reproduced
• "OK" for passing - last time the test case was executed the original problem appeared fixed
• "INV" for invalid - the test case is out of date (it can no longer run your program in the expected way)

You can now delete test cases with the delete button and debug test cases with the debug button as shown below:

"bank_account" example walkthrough

TODO: ...

How Test Cases Are Extracted

TODO: ...

How to modify test cases by hand

TODO: ...

Re-extracting test cases

TODO: ...

Release Notes

• CDD can be quite fragile, if there you encounter problems you can do the following:
  • Look into the testing-log-window (bottom half of screen) and see what went wrong
  • Remove the directory EIFGENs/cdd_tester from your projects directory.
  • Disable precompiles if having problems
  • Have a look at the test cases (you can just pick&drop them into the editor)
  • Do enable assertion checking!
  • Make sure you don't open your project with the 'cdd_tester' target, this target is only used internaly.

Source Code

The source of the last stable version can be found on the cdd branch of the EiffelStudio svn repository (https://origo.ethz.ch/eiffelsoftware/es/branches/cdd/Eiffel_57). After compiling it you need to do the following to create a delivery

• Take an existing delivery (5.7 final should be fine)
• Copy the resulting executable ec.exe resp. ec to $ISE_EIFFEL/studio/spec/$ISE_PLATFORM/bin/
• Copy the the file Delivery/eifinit/studio/default.xml to $ISE_EIFFEL/eifinit/studio/default.xml
• Copy the directory Src/library/cdd to $ISE_EIFFEL/library/cdd

If you would like to have a look at the latest source use https://origo.ethz.ch/eiffelsoftware/es/branches/cdd/Eiffel_60. Attention: this code is still under developement. Most functionality is not implemented yet.

NOTE: To compile the Eiffel_57 version of cdd, you will need the latest 5.7 release of estudio. Same for the Eiffel_60 version, there you will need the latest 6.0 release of estudio.

Testing

• CddRegressionTesting

Development (Todo's):

CDD_MANAGER class

• Improve interaction between estudio and the current test suite (violation notification, etc...)
• Improve logging and status handling

Capturing

• Use cdd reflection and test case creation classes from Eiffel_57 branch
• Save context data in test case class as string (XML format?)
• Create test case for each routine in current call stack (attach test cases through a linked list)
  • Only reflect the state of routines which are exported to ANY
• Add option for extracting context of non failing routines
• Handle postconditions/invariants within creation procedures
• POINTERS?
• Restore once values

GUI (Tool)

• Improve Model/View abstraction for displaying test suite (especially for the GRID)
• New views (through tabs):
  • Cluster view (with option for only showing failing test cases)
  • Call stack view

Execution (Background)

• Parse context string in class file for rebuilding state
• check invariants of all objects in context before running fut
• run test cases depending on processor load (run relevant test cases first)
• compile tester after setting up cdd in a new project (speeds up first test execution)

Execution (Debugger)

• Set breakpoint at feature beginning before executing (so user can actually debug the feature)

GUI (Context)

• Clickable Compiler/Tester output

GUI (Refinements)

• New icons!

Things we need from estudio

• Invariants should be checked during debugging equally to pre- and post conditions (they could also be visualised in the flat view the same way like pre- and post conditions are)
• The information whether some call is a creation call or a normal routine call (Not sure if this is really necessary, what if we assume every call to some creation procedure is always a creation call?)
• Support for multiple open targets