Difference between revisions of "CddBranch"

(How to Use)
m (Reverted edits by Jacksonsophia (Talk) to last version by Aleitner)
 
(285 intermediate revisions by 8 users not shown)
Line 1: Line 1:
== Summary ==
+
[[Category:Testing]]
CDD is an extension to EiffelStudio that provides support for both automatic test case extraction and test execution.
+
[[Category:EiffelDebugger]]
It is undistruptive to the development process of the user. The tool is in its earlys stages. Feedback and bug reports are
+
[[Category:CDD]]
very much appreciated. Please send them to mailto:andreas.leitner@inf.ethz.ch .
+
  
== Installation ==
 
  
=== Requirements ===
+
== What is the CDD EiffelStudio? ==
* ISE Eiffel 5.7 (below version is recommended, other versions may or may not work)
+
** Linux-x86: http://se.ethz.ch/people/leitner/cdd/Eiffel57_gpl_64493-linux-x86.tar.gz
+
** Win32: http://se.ethz.ch/people/leitner/cdd/Eiffel57_gpl_64493-windows.msi
+
  
=== CDD-Extension ===
+
[[Image:cdd_logo.png|center]]
 +
<center>
 +
[http://se.ethz.ch/people/leitner/cdd/video/ '''Play CDD Video''', click here]
 +
</center>
 +
CDD (short for Contract Driven Development) is a project developed at [http://se.ethz.ch ETH Zurich]. It adds advanced support for unit testing to EiffelStudio. With CDD EiffelStudio you can
  
* Linux: http://se.ethz.ch/people/leitner/cdd/cdd_linux_x86_0_1_0.tgz
+
* Write test cases
* Windows: http://se.ethz.ch/people/leitner/cdd/cdd_win32_0_1_0.tgz
+
* Manage test cases (using tags)
 +
* Run test cases
 +
* View test outcomes
 +
* '''Automatically extract test cases'''
  
* Installation instructions
+
If you have questions, feedback, or would like to report a bug please visit the [http://eiffelstudio.origo.ethz.ch/forum/20 CDD forum].  
** Download the zip if you use Windows, the tgz if you use Linux
+
** Extract it over your installation of EiffelStudio. (You have done it right when you are overwriting a few files in the process.)
+
** You are ready to go. Just start EiffelStudio the way you are used to. You will notice a two new windows, one on the left and one on the bottom. Both are titeled "Testing". If they do not appear on your installation, make sure Testing is enabled via the menu path "View->Tool->Testing"
+
  
== How to Use ==
+
CDD EiffelStudio adds the following panel to regular EiffelStudio:
  
=== Step 1: Show Testing Window ===
+
[[Image:cdd_panel.png|center]]
  
First you need to make sure the testing windows are all properly displayed. You can make them appear by following the menu "View->Tools->Testing":
 
  
[[Image:show_testing.png]]
+
== Publications ==
  
=== Step 1: Enable CDD-Testing ===
+
Leitner, A., Ciupa, I., Oriol, M., Meyer, B., Fiva, A., "Contract Driven Development = Test Driven Development - Writing Test Cases", in Proceedings of ESEC/FSE'07: European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering 2007, (Dubrovnik, Croatia), September 2007 [http://se.ethz.ch/people/leitner/publications/cdd_leitner_esec_fse_2007.pdf pdf] [http://se.ethz.ch/people/leitner/publications/ESEC_FSE2007.bib.txt bibtex]
To enable CDD-testing for your project, open your project and then enable CDD-testing, by clicking on the half-black / half-white button in the testing window. You can disable testing again at any time by pressing this button again.
+
  
 +
Leitner, A., Pretschner A., Mori S., Meyer B., Oriol M., "On the Effectiveness of Test Extraction without Overhead", under submission
  
Make sure you also have assertion checking enabled (the more the better cdd will work). Now continue developing your project as usual.
+
== News ==
You will notice that for every exception that occurs a new test case shows up in the testing window. Test cases are automatically executed on each compilation to see whether you fixed that particular bug or not. A test case can be in one of three states:
+
14.May 2008: Added patch for Windows and Linux that fixes a bug that can cause a crash during extraction.
  
* "F"ailing - last time the test case was executed the original problem was reproduced
+
10.April 2008: If installation exits with cryptic error codes, please have a look at the [[CDD Common Problems|Common Problems]] page.
* "P"assing - last time the test case was executed the original problem appeared fixed
+
* "INV"alid - the test case is out of date (it can no longer run your program in the expected way)
+
  
[[Image:test_case_states.png]]
+
08.April 2008: CDD EiffelStudio final 7 available
 +
* Important bug fix: EiffelStudio no longer freezes when opening projects with many test cases
  
You can now delete test cases with the delete button and debug test cases with the debug button as shown below:
+
02.April 2008: CDD EiffelStudio Final 6 available
 +
* Less mess: Redundant or duplicate test cases are no longer extracted
 +
* Smoother upgrade: Cleaning a project will automatically clean the test suite
 +
* Fixed bug in invarant checking
 +
* Fixed bug that caused EiffelStudio to freeze in some situations
 +
* Improved logging
 +
* Installer ''might'' work with mingw on Windows now. (Not yet tested, reports welcome)
  
[[Image:testing_buttons.png]]
+
== Download CDD ==
  
=== How Test Cases Are Extracted ===
+
The following packages contain the full EiffelStudio 6.1 plus the CDD extension. You do not need to have EiffelStudio installed already in order to install below packages. On Windows you do have to have either the Platform SDK or Visual C++ installed. Do not use EiffelStudio with the gcc/mingw or the .Net backend.
  
=== How to modify test cases by hand ===
+
=== Linux ===
 +
* Full Linux version: http://se.ethz.ch/people/leitner/cdd/Eiffel61_cdd_final_7.tar.bz2
 +
* Full Linux version (for Debian stable): http://se.ethz.ch/people/leitner/cdd/Eiffel61_cdd_final_7_debian_stable.tar.bz2
 +
* Installation instructions: http://docs.eiffel.com/eiffelstudio/installation/studio/060_linux.html
 +
Notes:
 +
* If you are '''upgrading''' from a previous version, make sure you delete the old version first (rm -rf)
  
=== Re-extracting test cases ===
+
Download the above file and install it just like you would install a EiffelStudio tar ball. Afterwards proceed to section "Using CDD". Make sure you set/update the environment variables PATH, ISE_EIFFEL, and ISE_PLATFORM according to the installation instructions.
  
== Release Notes ==
+
=== Linux Patch ===
 +
* Patch for CDD EiffelStudio Final 7: Download http://se.ethz.ch/people/leitner/cdd/ec and replace it with the '$ISE_EIFFEL/studio/spec/linux-x86/bin/ec' from Final 7. This patch fixes a crash that sometimes occurs during extraction of test cases. You may need to delete the precompile and EIFGENs of your project after applying the patch.
  
* CDD can be quite fragile, if there you encounter problems you can do the following:
+
=== Windows ===
** Look into the testing-log-window (bottom half of screen) and see what went wrong
+
** Remove the directory EIFGENs/cdd_tester from your projects directory.
+
** Disable precompiles if having problems
+
** Have a look at the test cases (you can just pick&drop them into the editor)
+
** Make sure your root cluster is not recursive.
+
** Do enable assertion checking!
+
  
 +
* Full Windows version (with installer): http://n.ethz.ch/~moris/download/Eiffel61_cdd_final_7-windows.msi
 +
Notes:
 +
* Installation is independent of installations of official EiffelStudio 6.1 (neither overwrites nor invalidates nor is influenced by those)
 +
* If you are '''upgrading''' from a previous version, you first have to uninstall the old version then delete the existing precompilations (= delete EIFGENs in subdirectories of <INSTALL_DIRECTORY>/precomp/spec/windows/) and then install the new version.
 +
* Which C compiler to install?
 +
** Use the Microsoft C compiler either from Visual Studio or the Windows SDK:
 +
*** Visual Studio (up to Visual Studio 2005, but no later, and only the non-express version)
 +
*** Windows SDK (up to version 6.0, but no later)
 +
** Do not use the gcc/mingw.
 +
** Do not use the .NET compiler backend.
  
 +
Have a look at http://dev.eiffel.com/Installing_Microsoft_C_compiler_6.1_and_older to learn how to install either Visual C++ or the Windows SDK for use with EiffelStudio.
  
== Source Code ==
 
  
The source can be found on the cdd branch of the EiffelStudio svn repository (https://origo.ethz.ch/eiffelsoftware/es/branches/cdd/es).
+
==== Windows Patch ====
After compiling it you need to do the following to create a delivery
+
  
* Take an existing delivery (5.7 final should be fine)
+
* Patch for CDD EiffelStudio Final 7: Download http://se.ethz.ch/people/leitner/cdd/ec.exe and replace it with the '$ISE_EIFFEL/studio/spec/windows/bin/ec.exe' from Final 7. This patch fixes a crash that sometimes occurs during extraction of test cases. You may need to delete the precompile and EIFGENs of your project after applying the patch.
* Copy the resulting executable ec.exe resp. ec to $ISE_EIFFEL/studio/spec/$ISE_PLATFORM/bin/
+
 
* Copy the the file Delivery/eifinit/studio/default.xml to $ISE_EIFFEL/eifinit/studio/default.xml
+
== Documentation ==
* Copy the directory Src/library/cdd to $ISE_EIFFEL/library/cdd
+
 
 +
=== Manually Written Test Cases ===
 +
CDD Eiffelstudio allows you to create a new “empty” test case. These test cases are similar to jUnit test cases. A manually written test class must start with the word “TEST” and all test routines also have to start with the word “test”. It also has to inherit from class CDD_TEST_CASE.
 +
 
 +
=== Extracted Test Cases ===
 +
CDD EiffelStudio automatically extracts test cases whenever you run your program and an exception is triggered. This feature is novel and not yet part of any other testing environment. You will be the first to try it out.
 +
 
 +
=== Test outcomes ===
 +
A test case checks whether your program contains a particular bug. A test cases can fail indicating that the bug is present in your program, or pass indicating your program does not contain this bug. Sometimes test cases will be unresolved, in which case the testing framework was unable to find out whether the test case passed or failed. A test case can be unresolved for several reasons.
 +
 
 +
 
 +
[[Image:cdd_buttons.png|center]]
 +
 
 +
=== Debug Test Case ===
 +
Select a test case and press this button to run a test case in the debugger.
 +
 
 +
=== Enable/Disable Execution of Test Cases in Background ===
 +
If enabled, all test cases are retested every time you press compile. If disabled no test cases are executed.
 +
 
 +
=== Enable/Disable automatic extraction of Test Cases ===
 +
If enabled every time an exception is triggered a set of test cases that try to reproduce this exception is extracted. If disabled no test cases are extracted.
 +
 
 +
=== Clean up/Delete ===
 +
You can use the “Clean Up/Delete” button in two different ways. By simply pressing it you will delete all unresolved test cases. By pick and dropping a test case to the “Clean up/Delete” button (right click on test case, move mouse to button and right click again) you can delete a test case. By the way, test cases are just regular classes. So you can use all existing tools that apply to classes in EiffelStudio too.
 +
 
 +
Update: To remove duplicate test cases (until the next cdd update), please use the command-line tool from the [http://clean-cdd.origo.ethz.ch/ clean-cdd project].
 +
 
 +
=== Create new manual test class ===
 +
Press this button to create a empty test class. You can then edit the class to add manually written test cases. This is how a manually written test case can look like:
 +
 
 +
<eiffel>
 +
class TEST_BANK_ACCOUNT
 +
inherit CDD_TEST_CASE
 +
feature
 +
  test_deposit
 +
    local
 +
      ba: BANK_ACCOUNT
 +
    do
 +
      create ba.make_with_balance (0)
 +
      ba.deposit (100)
 +
      check
 +
        money_depisited: ba.balance = 100
 +
      end
 +
    end
 +
end
 +
</eiffel>
 +
 
 +
Redefine routine `set_up' or `tear_down' if you want something to be executed before resp. after every test routine of a class.
 +
 
 +
[[Image:cdd_buttons_2.png|center]]
 +
 
 +
=== Search for tags ===
 +
Enter keywords to search for paritcular test cases. Some tags are automatically set for you like the name of the test case. You can also easily add your own tags by adding an indexing item "tag" to your test class or routine:
 +
 
 +
<eiffel>
 +
indexing
 +
  tag: "fixme"
 +
class TEST_BANK_ACCOUNT
 +
inherit CDD_TEST_CASE
 +
feature
 +
  ...
 +
end
 +
</eiffel>
 +
 
 +
Tags can be adde to all test routines and classes. Whether they are extracted or manually written does not matter.
 +
 
 +
=== Restrict execution of test cases ===
 +
 
 +
Once you have many test cases, you will run into situation where you don't want to execute all of them. The restrict button will help you to achieve this. As long as the "Restrict" button is pushed test cases that don't show up in the test case view will not be tested. Execution is restricted to those test cases that do show up.
 +
 
 +
=== Change test case view ===
 +
 
 +
Select one of several predefined test case views. For example you can group test cases by their outcome to quickly see only failing test cases.
 +
 
 +
== Further Documentation and Common Problems ==
 +
Please visit [[Using CDD]] for further documentation or look at
 +
[[CDD Common Problems|Common Problems]] if you run into problems.
 +
 
 +
== Related Publications ==
 +
 
 +
* Sunghun Kim, Shay Artzi, and Michael D. Ernst, "reCrash: Making Crash Reproducible" MIT Computer Science and Artificial Intelligence Laboratory technical report MIT-CSAIL-TR-2007-054, (Cambridge, MA), November 20, 2007. [http://recrash.googlecode.com/files/MIT-CSAIL-TR-2007-054.pdf (pdf)]
 +
 
 +
== Project Internal Stuff ==
 +
 
 +
[[CddBranchInternal]]

Latest revision as of 07:23, 31 May 2010


What is the CDD EiffelStudio?

Cdd logo.png

Play CDD Video, click here

CDD (short for Contract Driven Development) is a project developed at ETH Zurich. It adds advanced support for unit testing to EiffelStudio. With CDD EiffelStudio you can

  • Write test cases
  • Manage test cases (using tags)
  • Run test cases
  • View test outcomes
  • Automatically extract test cases

If you have questions, feedback, or would like to report a bug please visit the CDD forum.

CDD EiffelStudio adds the following panel to regular EiffelStudio:

Cdd panel.png


Publications

Leitner, A., Ciupa, I., Oriol, M., Meyer, B., Fiva, A., "Contract Driven Development = Test Driven Development - Writing Test Cases", in Proceedings of ESEC/FSE'07: European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering 2007, (Dubrovnik, Croatia), September 2007 pdf bibtex

Leitner, A., Pretschner A., Mori S., Meyer B., Oriol M., "On the Effectiveness of Test Extraction without Overhead", under submission

News

14.May 2008: Added patch for Windows and Linux that fixes a bug that can cause a crash during extraction.

10.April 2008: If installation exits with cryptic error codes, please have a look at the Common Problems page.

08.April 2008: CDD EiffelStudio final 7 available

  • Important bug fix: EiffelStudio no longer freezes when opening projects with many test cases

02.April 2008: CDD EiffelStudio Final 6 available

  • Less mess: Redundant or duplicate test cases are no longer extracted
  • Smoother upgrade: Cleaning a project will automatically clean the test suite
  • Fixed bug in invarant checking
  • Fixed bug that caused EiffelStudio to freeze in some situations
  • Improved logging
  • Installer might work with mingw on Windows now. (Not yet tested, reports welcome)

Download CDD

The following packages contain the full EiffelStudio 6.1 plus the CDD extension. You do not need to have EiffelStudio installed already in order to install below packages. On Windows you do have to have either the Platform SDK or Visual C++ installed. Do not use EiffelStudio with the gcc/mingw or the .Net backend.

Linux

Notes:

  • If you are upgrading from a previous version, make sure you delete the old version first (rm -rf)

Download the above file and install it just like you would install a EiffelStudio tar ball. Afterwards proceed to section "Using CDD". Make sure you set/update the environment variables PATH, ISE_EIFFEL, and ISE_PLATFORM according to the installation instructions.

Linux Patch

  • Patch for CDD EiffelStudio Final 7: Download http://se.ethz.ch/people/leitner/cdd/ec and replace it with the '$ISE_EIFFEL/studio/spec/linux-x86/bin/ec' from Final 7. This patch fixes a crash that sometimes occurs during extraction of test cases. You may need to delete the precompile and EIFGENs of your project after applying the patch.

Windows

Notes:

  • Installation is independent of installations of official EiffelStudio 6.1 (neither overwrites nor invalidates nor is influenced by those)
  • If you are upgrading from a previous version, you first have to uninstall the old version then delete the existing precompilations (= delete EIFGENs in subdirectories of <INSTALL_DIRECTORY>/precomp/spec/windows/) and then install the new version.
  • Which C compiler to install?
    • Use the Microsoft C compiler either from Visual Studio or the Windows SDK:
      • Visual Studio (up to Visual Studio 2005, but no later, and only the non-express version)
      • Windows SDK (up to version 6.0, but no later)
    • Do not use the gcc/mingw.
    • Do not use the .NET compiler backend.

Have a look at http://dev.eiffel.com/Installing_Microsoft_C_compiler_6.1_and_older to learn how to install either Visual C++ or the Windows SDK for use with EiffelStudio.


Windows Patch

  • Patch for CDD EiffelStudio Final 7: Download http://se.ethz.ch/people/leitner/cdd/ec.exe and replace it with the '$ISE_EIFFEL/studio/spec/windows/bin/ec.exe' from Final 7. This patch fixes a crash that sometimes occurs during extraction of test cases. You may need to delete the precompile and EIFGENs of your project after applying the patch.

Documentation

Manually Written Test Cases

CDD Eiffelstudio allows you to create a new “empty” test case. These test cases are similar to jUnit test cases. A manually written test class must start with the word “TEST” and all test routines also have to start with the word “test”. It also has to inherit from class CDD_TEST_CASE.

Extracted Test Cases

CDD EiffelStudio automatically extracts test cases whenever you run your program and an exception is triggered. This feature is novel and not yet part of any other testing environment. You will be the first to try it out.

Test outcomes

A test case checks whether your program contains a particular bug. A test cases can fail indicating that the bug is present in your program, or pass indicating your program does not contain this bug. Sometimes test cases will be unresolved, in which case the testing framework was unable to find out whether the test case passed or failed. A test case can be unresolved for several reasons.


Cdd buttons.png

Debug Test Case

Select a test case and press this button to run a test case in the debugger.

Enable/Disable Execution of Test Cases in Background

If enabled, all test cases are retested every time you press compile. If disabled no test cases are executed.

Enable/Disable automatic extraction of Test Cases

If enabled every time an exception is triggered a set of test cases that try to reproduce this exception is extracted. If disabled no test cases are extracted.

Clean up/Delete

You can use the “Clean Up/Delete” button in two different ways. By simply pressing it you will delete all unresolved test cases. By pick and dropping a test case to the “Clean up/Delete” button (right click on test case, move mouse to button and right click again) you can delete a test case. By the way, test cases are just regular classes. So you can use all existing tools that apply to classes in EiffelStudio too.

Update: To remove duplicate test cases (until the next cdd update), please use the command-line tool from the clean-cdd project.

Create new manual test class

Press this button to create a empty test class. You can then edit the class to add manually written test cases. This is how a manually written test case can look like:

class TEST_BANK_ACCOUNT
inherit CDD_TEST_CASE
feature
  test_deposit
    local
      ba: BANK_ACCOUNT
    do
      create ba.make_with_balance (0)
      ba.deposit (100)
      check
         money_depisited: ba.balance = 100
      end
    end
end

Redefine routine `set_up' or `tear_down' if you want something to be executed before resp. after every test routine of a class.

Cdd buttons 2.png

Search for tags

Enter keywords to search for paritcular test cases. Some tags are automatically set for you like the name of the test case. You can also easily add your own tags by adding an indexing item "tag" to your test class or routine:

indexing
  tag: "fixme"
class TEST_BANK_ACCOUNT
inherit CDD_TEST_CASE
feature
   ...
end

Tags can be adde to all test routines and classes. Whether they are extracted or manually written does not matter.

Restrict execution of test cases

Once you have many test cases, you will run into situation where you don't want to execute all of them. The restrict button will help you to achieve this. As long as the "Restrict" button is pushed test cases that don't show up in the test case view will not be tested. Execution is restricted to those test cases that do show up.

Change test case view

Select one of several predefined test case views. For example you can group test cases by their outcome to quickly see only failing test cases.

Further Documentation and Common Problems

Please visit Using CDD for further documentation or look at Common Problems if you run into problems.

Related Publications

  • Sunghun Kim, Shay Artzi, and Michael D. Ernst, "reCrash: Making Crash Reproducible" MIT Computer Science and Artificial Intelligence Laboratory technical report MIT-CSAIL-TR-2007-054, (Cambridge, MA), November 20, 2007. (pdf)

Project Internal Stuff

CddBranchInternal