Difference between revisions of "CddBranch"

(→‎Meeting Minutes)
m (Reverted edits by Jacksonsophia (Talk) to last version by Aleitner)
 
(198 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 
[[Category:Testing]]
 
[[Category:Testing]]
 
[[Category:EiffelDebugger]]
 
[[Category:EiffelDebugger]]
 +
[[Category:CDD]]
  
−
== Summary ==
 
−
CDD is an extension to EiffelStudio that provides support for both automatic test case extraction and test execution.
 
−
It is undisruptive to the development process of the user. The tool is in its early stages. Feedback and bug reports are
 
−
very much appreciated. Please send them to mailto:andreas.leitner@inf.ethz.ch and or use the  discussion page for this article.
 
  
 +
== What is the CDD EiffelStudio? ==
  
−
[[Image:video_still.png|center]]
+
[[Image:cdd_logo.png|center]]
−
 
+
 
<center>
 
<center>
−
<h1>
+
[http://se.ethz.ch/people/leitner/cdd/video/ '''Play CDD Video''', click here]
−
[http://se.ethz.ch/people/leitner/cdd/video Play Video!]
+
−
</h1>
+
 
</center>
 
</center>
 +
CDD (short for Contract Driven Development) is a project developed at [http://se.ethz.ch ETH Zurich]. It adds advanced support for unit testing to EiffelStudio. With CDD EiffelStudio you can
 +
 +
* Write test cases
 +
* Manage test cases (using tags)
 +
* Run test cases
 +
* View test outcomes
 +
* '''Automatically extract test cases'''
 +
 +
If you have questions, feedback, or would like to report a bug please visit the [http://eiffelstudio.origo.ethz.ch/forum/20 CDD forum].
 +
 +
CDD EiffelStudio adds the following panel to regular EiffelStudio:
 +
 +
[[Image:cdd_panel.png|center]]
 +
 +
 +
== Publications ==
 +
 +
Leitner, A., Ciupa, I., Oriol, M., Meyer, B., Fiva, A., "Contract Driven Development = Test Driven Development - Writing Test Cases", in Proceedings of ESEC/FSE'07: European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering 2007, (Dubrovnik, Croatia), September 2007 [http://se.ethz.ch/people/leitner/publications/cdd_leitner_esec_fse_2007.pdf pdf] [http://se.ethz.ch/people/leitner/publications/ESEC_FSE2007.bib.txt bibtex]
 +
 +
Leitner, A., Pretschner A., Mori S., Meyer B., Oriol M., "On the Effectiveness of Test Extraction without Overhead", under submission
 +
 +
== News ==
 +
14.May 2008: Added patch for Windows and Linux that fixes a bug that can cause a crash during extraction.
 +
 +
10.April 2008: If installation exits with cryptic error codes, please have a look at the [[CDD Common Problems|Common Problems]] page.
 +
 +
08.April 2008: CDD EiffelStudio final 7 available
 +
* Important bug fix: EiffelStudio no longer freezes when opening projects with many test cases
  
−
== Installation (old version, new version not ready yet) ==
+
02.April 2008: CDD EiffelStudio Final 6 available
 +
* Less mess: Redundant or duplicate test cases are no longer extracted
 +
* Smoother upgrade: Cleaning a project will automatically clean the test suite
 +
* Fixed bug in invarant checking
 +
* Fixed bug that caused EiffelStudio to freeze in some situations
 +
* Improved logging
 +
* Installer ''might'' work with mingw on Windows now. (Not yet tested, reports welcome)
  
−
=== Requirements ===
+
== Download CDD ==
−
* ISE Eiffel 5.7 (below version is recommended, other versions may or may not work)
+
−
** Linux-x86: http://se.ethz.ch/people/leitner/cdd/Eiffel57_gpl_64493-linux-x86.tar.gz
+
−
** Win32: http://se.ethz.ch/people/leitner/cdd/Eiffel57_gpl_64493-windows.msi
+
  
−
=== Install CDD-Extension for EiffelStudio ===
+
The following packages contain the full EiffelStudio 6.1 plus the CDD extension. You do not need to have EiffelStudio installed already in order to install below packages. On Windows you do have to have either the Platform SDK or Visual C++ installed. Do not use EiffelStudio with the gcc/mingw or the .Net backend.
  
−
* Linux
+
=== Linux ===
−
** Delete the directory $HOME/.es
+
* Full Linux version: http://se.ethz.ch/people/leitner/cdd/Eiffel61_cdd_final_7.tar.bz2
−
** Download http://se.ethz.ch/people/leitner/cdd/cdd_linux_x86_0_2_0.tgz
+
* Full Linux version (for Debian stable): http://se.ethz.ch/people/leitner/cdd/Eiffel61_cdd_final_7_debian_stable.tar.bz2
−
** Unpack it over your existing EiffelStudio installation. (You have done it right when you are overwriting a few files in the process.)
+
* Installation instructions: http://docs.eiffel.com/eiffelstudio/installation/studio/060_linux.html
 +
Notes:
 +
* If you are '''upgrading''' from a previous version, make sure you delete the old version first (rm -rf)
  
−
* Windows
+
Download the above file and install it just like you would install a EiffelStudio tar ball. Afterwards proceed to section "Using CDD". Make sure you set/update the environment variables PATH, ISE_EIFFEL, and ISE_PLATFORM according to the installation instructions.
−
** Start the registry editor (regedit.exe) and delete the key "HKEY_CURRENT_USER\Software\ISE\Eiffel57"
+
−
** Download http://se.ethz.ch/people/leitner/cdd/cdd_win32_0_2_1.zip
+
−
** Unpack it over your existing EiffelStudio installation. (You have done it right when you are overwriting a few files in the process.)
+
  
−
== How to Use ==
+
=== Linux Patch ===
−
Make sure you also have assertion checking enabled.
+
* Patch for CDD EiffelStudio Final 7: Download http://se.ethz.ch/people/leitner/cdd/ec and replace it with the '$ISE_EIFFEL/studio/spec/linux-x86/bin/ec' from Final 7. This patch fixes a crash that sometimes occurs during extraction of test cases. You may need to delete the precompile and EIFGENs of your project after applying the patch.
−
This can be done easily in: "Project->Project Settings..." select assertions and select "True" for all of them (the more the better cdd will work).
+
  
−
=== Step 1: Show Testing Window ===
+
=== Windows ===
  
−
If not already visible, you need to make sure the testing windows are all properly displayed. You can make them appear by following the menu "View->Tools->Testing":
+
* Full Windows version (with installer): http://n.ethz.ch/~moris/download/Eiffel61_cdd_final_7-windows.msi
 +
Notes:
 +
* Installation is independent of installations of official EiffelStudio 6.1 (neither overwrites nor invalidates nor is influenced by those)
 +
* If you are '''upgrading''' from a previous version, you first have to uninstall the old version then delete the existing precompilations (= delete EIFGENs in subdirectories of <INSTALL_DIRECTORY>/precomp/spec/windows/) and then install the new version.
 +
* Which C compiler to install?
 +
** Use the Microsoft C compiler either from Visual Studio or the Windows SDK:
 +
*** Visual Studio (up to Visual Studio 2005, but no later, and only the non-express version)
 +
*** Windows SDK (up to version 6.0, but no later)
 +
** Do not use the gcc/mingw.
 +
** Do not use the .NET compiler backend.
  
−
[[Image:show_testing.png]]
+
Have a look at http://dev.eiffel.com/Installing_Microsoft_C_compiler_6.1_and_older to learn how to install either Visual C++ or the Windows SDK for use with EiffelStudio.  
  
−
=== Step 2: Enable CDD-Testing ===
 
−
To enable CDD-testing for your project, open your project and then enable CDD-testing, by clicking on the half-black / half-white button in the testing window. You can disable testing again at any time by pressing this button again.
 
  
−
[[Image:enable_testing.png]]
+
==== Windows Patch ====
  
−
Now continue developing your project as usual.  
+
* Patch for CDD EiffelStudio Final 7: Download http://se.ethz.ch/people/leitner/cdd/ec.exe and replace it with the '$ISE_EIFFEL/studio/spec/windows/bin/ec.exe' from Final 7. This patch fixes a crash that sometimes occurs during extraction of test cases. You may need to delete the precompile and EIFGENs of your project after applying the patch.
−
You will notice that for every exception that occurs a new test case shows up in the testing window. Test cases are automatically executed on each compilation to see whether you fixed that particular bug or not. A test case can be in one of three states:
+
  
−
* "F" for failing - last time the test case was executed the original problem was reproduced
+
== Documentation ==
−
* "OK" for passing - last time the test case was executed the original problem appeared fixed
+
−
* "INV" for invalid - the test case is out of date (it can no longer run your program in the expected way)
+
  
−
[[Image:test_case_states.png]]
+
=== Manually Written Test Cases ===
 +
CDD Eiffelstudio allows you to create a new “empty” test case. These test cases are similar to jUnit test cases. A manually written test class must start with the word “TEST” and all test routines also have to start with the word “test”. It also has to inherit from class CDD_TEST_CASE.
  
−
You can now delete test cases with the delete button and debug test cases with the debug button as shown below:
+
=== Extracted Test Cases ===
 +
CDD EiffelStudio automatically extracts test cases whenever you run your program and an exception is triggered. This feature is novel and not yet part of any other testing environment. You will be the first to try it out.
  
−
[[Image:testing_buttons.png]]
+
=== Test outcomes ===
 +
A test case checks whether your program contains a particular bug. A test cases can fail indicating that the bug is present in your program, or pass indicating your program does not contain this bug. Sometimes test cases will be unresolved, in which case the testing framework was unable to find out whether the test case passed or failed. A test case can be unresolved for several reasons.
  
−
=== "bank_account" example walkthrough ===
 
  
−
TODO: ...
+
[[Image:cdd_buttons.png|center]]
  
−
=== How Test Cases Are Extracted ===
+
=== Debug Test Case ===  
 +
Select a test case and press this button to run a test case in the debugger.
  
−
TODO: ...
+
=== Enable/Disable Execution of Test Cases in Background ===
 +
If enabled, all test cases are retested every time you press compile. If disabled no test cases are executed.
  
−
=== How to modify test cases by hand ===
+
=== Enable/Disable automatic extraction of Test Cases ===  
 +
If enabled every time an exception is triggered a set of test cases that try to reproduce this exception is extracted. If disabled no test cases are extracted.
  
−
TODO: ...
+
=== Clean up/Delete ===
 +
You can use the “Clean Up/Delete” button in two different ways. By simply pressing it you will delete all unresolved test cases. By pick and dropping a test case to the “Clean up/Delete” button (right click on test case, move mouse to button and right click again) you can delete a test case. By the way, test cases are just regular classes. So you can use all existing tools that apply to classes in EiffelStudio too.
  
−
=== Re-extracting test cases ===
+
Update: To remove duplicate test cases (until the next cdd update), please use the command-line tool from the [http://clean-cdd.origo.ethz.ch/ clean-cdd project].
  
−
TODO: ...
+
=== Create new manual test class ===
 +
Press this button to create a empty test class. You can then edit the class to add manually written test cases. This is how a manually written test case can look like:
  
−
== Release Notes ==
+
<eiffel>
 +
class TEST_BANK_ACCOUNT
 +
inherit CDD_TEST_CASE
 +
feature
 +
  test_deposit
 +
    local
 +
      ba: BANK_ACCOUNT
 +
    do
 +
      create ba.make_with_balance (0)
 +
      ba.deposit (100)
 +
      check
 +
        money_depisited: ba.balance = 100
 +
      end
 +
    end
 +
end
 +
</eiffel>
  
−
* CDD can be quite fragile, if there you encounter problems you can do the following:
+
Redefine routine `set_up' or `tear_down' if you want something to be executed before resp. after every test routine of a class.
−
** Look into the testing-log-window (bottom half of screen) and see what went wrong
+
−
** Remove the directory EIFGENs/cdd_tester from your projects directory.
+
−
** Disable precompiles if having problems
+
−
** Have a look at the test cases (you can just pick&drop them into the editor)
+
−
** Do enable assertion checking!
+
−
** Make sure you don't open your project with the 'cdd_tester' target, this target is only used internaly.
+
  
−
== Source Code ==
+
[[Image:cdd_buttons_2.png|center]]
  
−
The source of the last stable version can be found on the cdd branch of the EiffelStudio svn repository (https://origo.ethz.ch/eiffelsoftware/es/branches/cdd/Eiffel_57).
+
=== Search for tags ===
−
After compiling it you need to do the following to create a delivery
+
Enter keywords to search for paritcular test cases. Some tags are automatically set for you like the name of the test case. You can also easily add your own tags by adding an indexing item "tag" to your test class or routine:
  
−
* Take an existing delivery (5.7 final should be fine)
+
<eiffel>
−
* Copy the resulting executable ec.exe resp. ec to $ISE_EIFFEL/studio/spec/$ISE_PLATFORM/bin/
+
indexing
−
* Copy the the file Delivery/eifinit/studio/default.xml to $ISE_EIFFEL/eifinit/studio/default.xml
+
  tag: "fixme"
−
* Copy the directory Src/library/cdd to $ISE_EIFFEL/library/cdd
+
class TEST_BANK_ACCOUNT
 +
inherit CDD_TEST_CASE
 +
feature
 +
  ...
 +
end
 +
</eiffel>
  
−
If you would like to have a look at the latest source use https://origo.ethz.ch/eiffelsoftware/es/branches/cdd/Eiffel_60. Attention: this code is still under developement. Most functionality is not implemented yet.
+
Tags can be adde to all test routines and classes. Whether they are extracted or manually written does not matter.
  
−
NOTE: To compile the Eiffel_57 version of cdd, you will need the latest 5.7 release of estudio. Same for the Eiffel_60 version, there you will need the latest 6.0 release of estudio.
+
=== Restrict execution of test cases ===
  
−
== Testing ==
+
Once you have many test cases, you will run into situation where you don't want to execute all of them. The restrict button will help you to achieve this. As long as the "Restrict" button is pushed test cases that don't show up in the test case view will not be tested. Execution is restricted to those test cases that do show up.
  
−
* [[CddRegressionTesting]]
+
=== Change test case view ===
  
−
== Development (Todo's): ==
+
Select one of several predefined test case views. For example you can group test cases by their outcome to quickly see only failing test cases.
  
−
== Meeting Minutes ==
+
== Further Documentation and Common Problems ==
 +
Please visit [[Using CDD]] for further documentation or look at
 +
[[CDD Common Problems|Common Problems]] if you run into problems.
  
−
CddMeeting01042008
+
== Related Publications ==
  
−
== Specifications ==
+
* Sunghun Kim, Shay Artzi, and Michael D. Ernst, "reCrash: Making Crash Reproducible" MIT Computer Science and Artificial Intelligence Laboratory technical report MIT-CSAIL-TR-2007-054, (Cambridge, MA), November 20, 2007. [http://recrash.googlecode.com/files/MIT-CSAIL-TR-2007-054.pdf (pdf)]
  
−
* [[CddTreeViewSpec]]
+
== Project Internal Stuff ==
  
−
=== Things we need from estudio ===
+
[[CddBranchInternal]]
−
* Invariants should be checked during debugging equally to pre- and post conditions (they could also be visualised in the flat view the same way like pre- and post conditions are)
+
−
* The information whether some call is a creation call or a normal routine call (Not sure if this is really necessary, what if we assume every call to some creation procedure is always a creation call?)
+
−
* Support for multiple open targets
+

Latest revision as of 06:23, 31 May 2010


What is the CDD EiffelStudio?

Cdd logo.png

Play CDD Video, click here

CDD (short for Contract Driven Development) is a project developed at ETH Zurich. It adds advanced support for unit testing to EiffelStudio. With CDD EiffelStudio you can

  • Write test cases
  • Manage test cases (using tags)
  • Run test cases
  • View test outcomes
  • Automatically extract test cases

If you have questions, feedback, or would like to report a bug please visit the CDD forum.

CDD EiffelStudio adds the following panel to regular EiffelStudio:

Cdd panel.png


Publications

Leitner, A., Ciupa, I., Oriol, M., Meyer, B., Fiva, A., "Contract Driven Development = Test Driven Development - Writing Test Cases", in Proceedings of ESEC/FSE'07: European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering 2007, (Dubrovnik, Croatia), September 2007 pdf bibtex

Leitner, A., Pretschner A., Mori S., Meyer B., Oriol M., "On the Effectiveness of Test Extraction without Overhead", under submission

News

14.May 2008: Added patch for Windows and Linux that fixes a bug that can cause a crash during extraction.

10.April 2008: If installation exits with cryptic error codes, please have a look at the Common Problems page.

08.April 2008: CDD EiffelStudio final 7 available

  • Important bug fix: EiffelStudio no longer freezes when opening projects with many test cases

02.April 2008: CDD EiffelStudio Final 6 available

  • Less mess: Redundant or duplicate test cases are no longer extracted
  • Smoother upgrade: Cleaning a project will automatically clean the test suite
  • Fixed bug in invarant checking
  • Fixed bug that caused EiffelStudio to freeze in some situations
  • Improved logging
  • Installer might work with mingw on Windows now. (Not yet tested, reports welcome)

Download CDD

The following packages contain the full EiffelStudio 6.1 plus the CDD extension. You do not need to have EiffelStudio installed already in order to install below packages. On Windows you do have to have either the Platform SDK or Visual C++ installed. Do not use EiffelStudio with the gcc/mingw or the .Net backend.

Linux

Notes:

  • If you are upgrading from a previous version, make sure you delete the old version first (rm -rf)

Download the above file and install it just like you would install a EiffelStudio tar ball. Afterwards proceed to section "Using CDD". Make sure you set/update the environment variables PATH, ISE_EIFFEL, and ISE_PLATFORM according to the installation instructions.

Linux Patch

  • Patch for CDD EiffelStudio Final 7: Download http://se.ethz.ch/people/leitner/cdd/ec and replace it with the '$ISE_EIFFEL/studio/spec/linux-x86/bin/ec' from Final 7. This patch fixes a crash that sometimes occurs during extraction of test cases. You may need to delete the precompile and EIFGENs of your project after applying the patch.

Windows

Notes:

  • Installation is independent of installations of official EiffelStudio 6.1 (neither overwrites nor invalidates nor is influenced by those)
  • If you are upgrading from a previous version, you first have to uninstall the old version then delete the existing precompilations (= delete EIFGENs in subdirectories of <INSTALL_DIRECTORY>/precomp/spec/windows/) and then install the new version.
  • Which C compiler to install?
    • Use the Microsoft C compiler either from Visual Studio or the Windows SDK:
      • Visual Studio (up to Visual Studio 2005, but no later, and only the non-express version)
      • Windows SDK (up to version 6.0, but no later)
    • Do not use the gcc/mingw.
    • Do not use the .NET compiler backend.

Have a look at http://dev.eiffel.com/Installing_Microsoft_C_compiler_6.1_and_older to learn how to install either Visual C++ or the Windows SDK for use with EiffelStudio.


Windows Patch

  • Patch for CDD EiffelStudio Final 7: Download http://se.ethz.ch/people/leitner/cdd/ec.exe and replace it with the '$ISE_EIFFEL/studio/spec/windows/bin/ec.exe' from Final 7. This patch fixes a crash that sometimes occurs during extraction of test cases. You may need to delete the precompile and EIFGENs of your project after applying the patch.

Documentation

Manually Written Test Cases

CDD Eiffelstudio allows you to create a new “empty” test case. These test cases are similar to jUnit test cases. A manually written test class must start with the word “TEST” and all test routines also have to start with the word “test”. It also has to inherit from class CDD_TEST_CASE.

Extracted Test Cases

CDD EiffelStudio automatically extracts test cases whenever you run your program and an exception is triggered. This feature is novel and not yet part of any other testing environment. You will be the first to try it out.

Test outcomes

A test case checks whether your program contains a particular bug. A test cases can fail indicating that the bug is present in your program, or pass indicating your program does not contain this bug. Sometimes test cases will be unresolved, in which case the testing framework was unable to find out whether the test case passed or failed. A test case can be unresolved for several reasons.


Cdd buttons.png

Debug Test Case

Select a test case and press this button to run a test case in the debugger.

Enable/Disable Execution of Test Cases in Background

If enabled, all test cases are retested every time you press compile. If disabled no test cases are executed.

Enable/Disable automatic extraction of Test Cases

If enabled every time an exception is triggered a set of test cases that try to reproduce this exception is extracted. If disabled no test cases are extracted.

Clean up/Delete

You can use the “Clean Up/Delete” button in two different ways. By simply pressing it you will delete all unresolved test cases. By pick and dropping a test case to the “Clean up/Delete” button (right click on test case, move mouse to button and right click again) you can delete a test case. By the way, test cases are just regular classes. So you can use all existing tools that apply to classes in EiffelStudio too.

Update: To remove duplicate test cases (until the next cdd update), please use the command-line tool from the clean-cdd project.

Create new manual test class

Press this button to create a empty test class. You can then edit the class to add manually written test cases. This is how a manually written test case can look like: 

class TEST_BANK_ACCOUNT
inherit CDD_TEST_CASE
feature
  test_deposit
    local
      ba: BANK_ACCOUNT
    do
      create ba.make_with_balance (0)
      ba.deposit (100)
      check
         money_depisited: ba.balance = 100
      end
    end
end

Redefine routine `set_up' or `tear_down' if you want something to be executed before resp. after every test routine of a class.

Cdd buttons 2.png

Search for tags

Enter keywords to search for paritcular test cases. Some tags are automatically set for you like the name of the test case. You can also easily add your own tags by adding an indexing item "tag" to your test class or routine: 

indexing
  tag: "fixme"
class TEST_BANK_ACCOUNT
inherit CDD_TEST_CASE
feature
   ...
end

Tags can be adde to all test routines and classes. Whether they are extracted or manually written does not matter.

Restrict execution of test cases

Once you have many test cases, you will run into situation where you don't want to execute all of them. The restrict button will help you to achieve this. As long as the "Restrict" button is pushed test cases that don't show up in the test case view will not be tested. Execution is restricted to those test cases that do show up.

Change test case view

Select one of several predefined test case views. For example you can group test cases by their outcome to quickly see only failing test cases.

Further Documentation and Common Problems

Please visit Using CDD for further documentation or look at Common Problems if you run into problems.

Related Publications

  • Sunghun Kim, Shay Artzi, and Michael D. Ernst, "reCrash: Making Crash Reproducible" MIT Computer Science and Artificial Intelligence Laboratory technical report MIT-CSAIL-TR-2007-054, (Cambridge, MA), November 20, 2007. (pdf)

Project Internal Stuff

CddBranchInternal

Retrieved from "https://dev.eiffel.com/index.php?title=CddBranch&oldid=13869"