Eiffel Mutation Testing Tool - Revision history

Kohlers: /* Manual */

2007-05-21T21:00:14Z

‎Manual

Kohlers: /* Problems */

2007-05-21T20:56:52Z

‎Problems

Kohlers: /* Design */

2007-05-21T20:53:49Z

‎Design

Kohlers: /* Design */

2007-05-21T20:53:14Z

‎Design

Kohlers at 20:52, 21 May 2007

2007-05-21T20:52:31Z

Kohlers at 20:50, 21 May 2007

2007-05-21T20:50:41Z

Kohlers at 20:49, 21 May 2007

2007-05-21T20:49:49Z

Kohlers at 15:37, 5 February 2007

2007-02-05T15:37:34Z

Philippe Masson at 14:20, 1 December 2006

2006-12-01T14:20:04Z

Liuli at 10:15, 23 November 2006

2006-11-23T10:15:34Z

← Older revision		Revision as of 21:00, 21 May 2007
Line 79:		Line 79:

	After locating the test classes and branching from normal execution of auto test, we had to call the mutated test case using all possible values of the mutant_id. The figure below gives an overview of the execution of the mutated manual test cases.		After locating the test classes and branching from normal execution of auto test, we had to call the mutated test case using all possible values of the mutant_id. The figure below gives an overview of the execution of the mutated manual test cases.
		+	[[Image:Manual.JPG]]

	=====Random=====		=====Random=====

@@ Line 89: / Line 89: @@
 One problem is that manually written test cases do not need to fulfill any requirements
 that restrict the instantiation of a class under test. Some test cases declare the class
-under test as a global variable, others as a local one for every feature. Therefore, there are many di�erent cases
+under test as a global variable, others as a local one for every feature. Therefore, there are many different cases
 to handle.
 One more, not yet solved problem is the comparison of the results the interpreter

@@ Line 65: / Line 65: @@
 The two ways that auto-test uses to test classes (manual and random) had to be treated separately.
-Manual:
+=====Manual=====
 These are the steps needed in general to test the manual test cases:
@@ Line 80: / Line 80: @@
 After locating the test classes and branching from normal execution of auto test, we had to call the mutated test case using all possible values of the mutant_id. The figure below gives an overview of the execution of the mutated manual test cases.
-=====Random:=====
+=====Random=====
 The random case was much easier to implement. Nevertheless it was harder to understand the functionality of this part of auto test.

@@ Line 80: / Line 80: @@
 After locating the test classes and branching from normal execution of auto test, we had to call the mutated test case using all possible values of the mutant_id. The figure below gives an overview of the execution of the mutated manual test cases.
-Random:
+=====Random:=====
 The random case was much easier to implement. Nevertheless it was harder to understand the functionality of this part of auto test.
 As there are no (manually written) test cases, only the mutated classes under test are needed. As in the manual case, an instance of a class under test can be changed to different mutants by changing their mutant_id.
 ====Problems====

@@ Line 64: / Line 64: @@
 ====Design====
 The two ways that auto-test uses to test classes (manual and random) had to be treated separately.
 Manual:
 These are the steps needed in general to test the manual test cases:
-*1. First run the original test case to obtain the results of the original, not mutated
+*1. First run the original test case to obtain the results of the original, not mutated class.
 class.
 *2. Change the attribute identifying the class under test to an instantiated object of the mutated class.
 *3. Set the mutant_id value of this class to 1.
@@ Line 74: / Line 75: @@
 *6. Repeat steps 3 to 5 using the other possible values for the mutant_id.
 As it was not possible to manipulate the test cases on run time, we decided to instrument not only the classes under test, but also the test cases.
 First, the class under test should be changed in the source code to the mutated type of the class under test. Second, the test case should provide a feature to change the value of the mutant_id attribute. To this end, also the mutant of the class under test should provide a feature to change this value.
 After locating the test classes and branching from normal execution of auto test, we had to call the mutated test case using all possible values of the mutant_id. The figure below gives an overview of the execution of the mutated manual test cases.
 Random:
 The random case was much easier to implement. Nevertheless it was harder to understand the functionality of this part of auto test.
 As there are no (manually written) test cases, only the mutated classes under test are needed. As in the manual case, an instance of a class under test can be changed to different mutants by changing their mutant_id.
 ====Problems====

← Older revision		Revision as of 20:49, 21 May 2007
Line 58:		Line 58:

	=== Execution ===		=== Execution ===
		+
		+	====Overview====
		+	The purpose of the „Execution“-Step of the mutation testing tool is to run auto-test with the mutated objects, and to compare the output of these mutants to the output of an „original“ object.
		+
		+	====Design====
		+	The two ways that auto-test uses to test classes (manual and random) had to be treated separately.
		+	Manual:
		+	These are the steps needed in general to test the manual test cases:
		+	1. First run the original test case to obtain the results of the original, not mutated
		+	class.
		+	2. Change the attribute identifying the class under test to an instantiated object of the mutated class.
		+	3. Set the mutant_id value of this class to 1.
		+	4. Run this version of the test case to obtain the „mutated“ results.
		+	5. Compare these results to the one gained from the first run to check whether they differ and therefore can be detected using the actual test case.
		+	6. Repeat steps 3 to 5 using the other possible values for the mutant_id.
		+	As it was not possible to manipulate the test cases on run time, we decided to instrument not only the classes under test, but also the test cases.
		+	First, the class under test should be changed in the source code to the mutated type of the class under test. Second, the test case should provide a feature to change the value of the mutant_id attribute. To this end, also the mutant of the class under test should provide a feature to change this value.
		+	After locating the test classes and branching from normal execution of auto test, we had to call the mutated test case using all possible values of the mutant_id. The figure below gives an overview of the execution of the mutated manual test cases.
		+	Random:
		+	The random case was much easier to implement. Nevertheless it was harder to understand the functionality of this part of auto test.
		+	As there are no (manually written) test cases, only the mutated classes under test are needed. As in the manual case, an instance of a class under test can be changed to different mutants by changing their mutant_id.
		+
		+	====Problems====
		+	One problem is that manually written test cases do not need to fulfill any requirements
		+	that restrict the instantiation of a class under test. Some test cases declare the class
		+	under test as a global variable, others as a local one for every feature. Therefore, there are many di�erent cases
		+	to handle.
		+	One more, not yet solved problem is the comparison of the results the interpreter
		+	returns, especially when there are exceptions.

	== Milestones ==		== Milestones ==

← Older revision		Revision as of 15:37, 5 February 2007
Line 57:		Line 57:
	Since, in our (Silvios and mine) opinion, reliability, extendibility and flexibility is more important than speed (although speed is important of course) we will focus on the first approach which will give us two almost completely independant components which can be easily exchanged and extended.		Since, in our (Silvios and mine) opinion, reliability, extendibility and flexibility is more important than speed (although speed is important of course) we will focus on the first approach which will give us two almost completely independant components which can be easily exchanged and extended.

		+	=== Execution ===

	== Milestones ==		== Milestones ==

@@ Line 15: / Line 15: @@
 * Instrument Eiffel source code to generate mutant schema file
 * Create mutant objects from mutant schema file and execute test cases on mutants to calculate the mutation score of test cases
 == Milestones ==

@@ Line 9: / Line 9: @@
 == Goals ==
-We aim to implement a completly automatic Eiffel mutation testing tool.
+We aim to implement a completely automatic Eiffel mutation testing tool.
 == Steps ==
@@ Line 24: / Line 24: @@
 == Team ==
 Philippe Masson
 Silvio Kohler