Revision as of 03:35, 4 January 2008

Summary

CDD is an extension to EiffelStudio that provides support for both automatic test case extraction and test execution. It is undisruptive to the development process of the user. The tool is in its early stages. Feedback and bug reports are very much appreciated. Please send them to mailto:andreas.leitner@inf.ethz.ch and or use the discussion page for this article.

Installation (old version, new version not ready yet)

Requirements

• ISE Eiffel 5.7 (below version is recommended, other versions may or may not work)
  • Linux-x86: http://se.ethz.ch/people/leitner/cdd/Eiffel57_gpl_64493-linux-x86.tar.gz
  • Win32: http://se.ethz.ch/people/leitner/cdd/Eiffel57_gpl_64493-windows.msi

Install CDD-Extension for EiffelStudio

• Linux
  • Delete the directory $HOME/.es
  • Download http://se.ethz.ch/people/leitner/cdd/cdd_linux_x86_0_2_0.tgz
  • Unpack it over your existing EiffelStudio installation. (You have done it right when you are overwriting a few files in the process.)

• Windows
  • Start the registry editor (regedit.exe) and delete the key "HKEY_CURRENT_USER\Software\ISE\Eiffel57"
  • Download http://se.ethz.ch/people/leitner/cdd/cdd_win32_0_2_1.zip
  • Unpack it over your existing EiffelStudio installation. (You have done it right when you are overwriting a few files in the process.)

How to Use

Make sure you also have assertion checking enabled. This can be done easily in: "Project->Project Settings..." select assertions and select "True" for all of them (the more the better cdd will work).

Step 1: Show Testing Window

If not already visible, you need to make sure the testing windows are all properly displayed. You can make them appear by following the menu "View->Tools->Testing":

Step 2: Enable CDD-Testing

To enable CDD-testing for your project, open your project and then enable CDD-testing, by clicking on the half-black / half-white button in the testing window. You can disable testing again at any time by pressing this button again.

Now continue developing your project as usual. You will notice that for every exception that occurs a new test case shows up in the testing window. Test cases are automatically executed on each compilation to see whether you fixed that particular bug or not. A test case can be in one of three states:

• "F" for failing - last time the test case was executed the original problem was reproduced
• "OK" for passing - last time the test case was executed the original problem appeared fixed
• "INV" for invalid - the test case is out of date (it can no longer run your program in the expected way)

You can now delete test cases with the delete button and debug test cases with the debug button as shown below:

"bank_account" example walkthrough

TODO: ...

How Test Cases Are Extracted

TODO: ...

How to modify test cases by hand

TODO: ...

Re-extracting test cases

TODO: ...

Release Notes

• CDD can be quite fragile, if there you encounter problems you can do the following:
  • Look into the testing-log-window (bottom half of screen) and see what went wrong
  • Remove the directory EIFGENs/cdd_tester from your projects directory.
  • Disable precompiles if having problems
  • Have a look at the test cases (you can just pick&drop them into the editor)
  • Do enable assertion checking!
  • Make sure you don't open your project with the 'cdd_tester' target, this target is only used internaly.

Source Code

The source of the last stable version can be found on the cdd branch of the EiffelStudio svn repository (https://origo.ethz.ch/eiffelsoftware/es/branches/cdd/Eiffel_57). After compiling it you need to do the following to create a delivery

• Take an existing delivery (5.7 final should be fine)
• Copy the resulting executable ec.exe resp. ec to $ISE_EIFFEL/studio/spec/$ISE_PLATFORM/bin/
• Copy the the file Delivery/eifinit/studio/default.xml to $ISE_EIFFEL/eifinit/studio/default.xml
• Copy the directory Src/library/cdd to $ISE_EIFFEL/library/cdd

If you would like to have a look at the latest source use https://origo.ethz.ch/eiffelsoftware/es/branches/cdd/Eiffel_60. Attention: this code is still under developement. Most functionality is not implemented yet.

NOTE: To compile the Eiffel_57 version of cdd, you will need the latest 5.7 release of estudio. Same for the Eiffel_60 version, there you will need the latest 6.0 release of estudio.

Testing

• CddRegressionTesting

Development (Todo's):

Schedule

• 04.01.2008: Final experiment definition (questions to ask, how to conduct experiment)
• 08.01.2008: Finalized list of features to go in release (including logging and log submission)
• 25.01.2008: Beta 1 (feature complete version online)
• 04.02.2008: Beta 2 (designated testers test release, # > 3)
• 18.02.2008: Final CDD release for experiment online
• 20.05.2008: Having all data
• 06.06.2008: Finished analysis

Stefans Master Plan

• MA Start ca 17.12.2007
• MA End ca 17.6.2008

• Testing the tester
  • System level test for CDD (incl. framework)
  • Recreating existing unit test suite with CDD
  • Large scale validation of CDD
    • Info 4 and/or Software Engineering
    • Questions
      • Does testing (manual/extracted) increase developer productivity?
      • How many tests do ppl end up with (manual/extracted)?
      • ...

Meeting Minutes

CddMeeting01042008

Specifications

• CddTreeViewSpec

Things we need from estudio

• Invariants should be checked during debugging equally to pre- and post conditions (they could also be visualised in the flat view the same way like pre- and post conditions are)
• The information whether some call is a creation call or a normal routine call (Not sure if this is really necessary, what if we assume every call to some creation procedure is always a creation call?)
• Support for multiple open targets