[[Category:Releases]]

During the [[:Category:EiffelStudio|EiffelStudio]] [[EiffelStudio 6.4 Releases|6.4]] development cycle Eiffel Software and any willing third-party contributors are updating the Eiffel stock [[:Category:Library|libraries]] to be Void-Safe. The libraries will still compile in non-Void-Safe contexts so your code will not be broken. The status reflects work completed so you may start migrating your own code to ensure Void-safety.

Make sure to follow the general rules given below, and ask the community for guidance if you run into any problems or uncertainties.

== Completion Status ==

To better hightlight the usefulness of the void-safety mechanism, we have put together a [[Void-Safe_Library_Results|non-exhaustive list]] of bugs found during the conversion process.

{| class="wikitable"
|-
! width="200"|Library Name
! width="100"|Status
! width="200"|Credits
|-
| EiffelBase
| Done
| Eiffel Software
|-
| EiffelBase extension
| Done
| Eiffel Software (Larry, Jocelyn)
|-
| EiffelTime
| Done
| Eiffel Software (Ted, Ian)
|-
| EiffelThread
| Done (classic)
| Eiffel Software (Arno)
|-
| EiffelUUID
| Done
| Eiffel Software ([[User:paulb|Paul]])
|-
| Eiffel2Java
| Done
| Eiffel Software (Ted, Ian)
|-
| WEL
| Done
| Eiffel Software (Manu)
|-
| EiffelVision2
|
|
|-
| EiffelVision2 extension
|
|
|-
| EiffelProcess
| Done (classic)
| Eiffel Software (Arno)
|-
| Argument parser
| Done
| Eiffel Software ([[User:paulb|Paul]])
|-
| EiffelLex
| Done
| Eiffel Software ([[User:paulb|Paul]])
|-
| EiffelParse
| In progress
| Eiffel Software (Ted, Ian)
|-
| EiffelNet
| Done
| Eiffel Software (Manu)
|-
| EiffelNet IPv6
| Done
| Eiffel Software (Manu)
|-
| EiffelCurl
| Done
| Eiffel Software (Manu)
|-
| Encoding
| Done
| Eiffel Software (Ted, Ian)
|-
| EiffelCOM
|
|
|-
| EiffelStore
|
|
|-
| EiffelTesting
| Done
| Eiffel Software (Arno)
|-
| EiffelWeb
| Done
| Eiffel Software (Manu)
|-
| Gobo
| In progress
| Eiffel Software (Jocelyn,Larry)
|-
| Docking
|
|
|-
| Gobo extension
|
|
|-
| EiffelGraph
|
|
|-
| Memory Analyzer
|
|
|-
| EiffelPreferences
|
|
|-
| Diff
| Done
| Eiffel Software ([[User:paulb|Paul]])
|}

== Contributing ==
EiffelStudio is open source and welcomes the Eiffel community contributions to speed up the adaptation process. If you are interested in participating please put a comment on the discussion board with your contact details.

==Rules to be applied ==

Please observe the following guidelines carefully to guarantee a quality result.

=== Examples ===
For examples of libraries already adapted, see UUID (for a small example) and EiffelBase (for a larger one).

=== Overall process ===

* First compile with the `full_class_checking' option on. Then enable the void-safe option.
* Compile libraries on all of Windows/.NET/Unix to ensure it is sound.

* Minimize modifications; types should be attached by default if it makes sense, otherwise it has to be detachable by default.
* Use the convention library-safe.ecf for naming void-safe libraries for now. All library references should be using the -safe.ecf variants.
* Use the same UUIDs for void-safe and non-void-safe libraries.
* Before any modifications add a library.lic and library-safe.lic (replace library with the name of the ECF minus the .ecf extension) next to the ECFs of the same name containing only the single line reference:forum2.
* Update all samples to use the void-safe ecfs and update them.

=== Rules ===
* DO NOT USE '''!''' (attached mark).
* MINIMIZE USE OF OBJECT TEST; ideally, don't use object test unless there was an assignment attempt in the original library.
* When a precondition expects a Void argument, use '''?''' if attached by default.
* When a precondition expects a non-Void argument, use '''!''' if detachable by default.
* Libraries should compile in both void-safe and non-void-safe mode.
* Only use the '''attribute''' keyword when it is impossible to initialize an attribute in the creation procedure. Never use it for lazy evaluation.
* You may include preconditions x /= Void, but it will have to be removed in the end (helped by a compiler warning that says this is not needed for attached x).

=== General cleanup ===
The void-safe adaptation process should be accompanied by a general upgrade to ISO/ECMA Eiffel:

* Remove uses of is_equal and equal to compare objects. (They can cause catcalls.) Replace them with the tilde operator, i.e. a ~ b instead of equal (a, b) or a.is_equal (b). Be careful to preserve the semantics (~ always returns false in the case of non-identical types).
* Replace the '''indexing''' keyword with '''note'''.
* Remove the '''is''' keyword in routines. Use the Replace tool with the regex '''\ is[ \t]*$'''. (Be careful not to use replace all, because comments and multi-line strings may have "is" text!)
* Replace the '''is''' keyword in constants with '''='''.

=== Test authoring ===
1. Create a cluster called 'tests' in the library root folder. E.g., for the UUID library the 'tests' folder exists at '$ISE_LIBRARY/uuid/tests'.

2. In the library ECFs, exclude the 'tests' cluster because it contains testing code and not library code.

3. Add a testing 'tests.ecf' in the 'tests' folder. (See the UUID library for an example ECF.) Be sure to create a library ECF and change the UUID. The library should also use the void-safe options found in the associated library's ECF.

4. Create test class names using the library name along with TEST as a prefix:
EiffelBase = BASE_TEST_
EiffelThread = THREAD_TEST_
EiffelVision2 = EV_TEST_ or VISION2_TEST_

=== Improving this page ===

As you encounter problems and devise your solutions, please include the results of your experience here.

Void-Safe Library Status

2009-01-22T23:20:53Z

Arnofiva: /* Completion Status */

[[Category:Releases]]

During the [[:Category:EiffelStudio|EiffelStudio]] [[EiffelStudio 6.4 Releases|6.4]] development cycle Eiffel Software and any willing third-party contributors are updating the Eiffel stock [[:Category:Library|libraries]] to be Void-Safe. The libraries will still compile in non-Void-Safe contexts so your code will not be broken. The status reflects work completed so you may start migrating your own code to ensure Void-safety.

Make sure to follow the general rules given below, and ask the community for guidance if you run into any problems or uncertainties.

== Completion Status ==

Here is a [[Void-Safe_Library_Results|non-exhaustive list]] of bugs found during this conversion process.

{| class="wikitable"
|-
! width="200"|Library Name
! width="100"|Status
! width="200"|Credits
|-
| EiffelBase
| Done
| Eiffel Software
|-
| EiffelBase extension
| In progress
| Eiffel Software (Larry, Jocelyn)
|-
| EiffelTime
| Done
| Eiffel Software (Ted, Ian)
|-
| EiffelThread
| Done (classic)
| Eiffel Software (Arno)
|-
| EiffelUUID
| Done
| Eiffel Software
|-
| Eiffel2Java
|
|
|-
| WEL
| In Progress
| Eiffel Software (Manu)
|-
| EiffelVision2
|
|
|-
| EiffelVision2 extension
|
|
|-
| EiffelProcess
| Done (classic)
| Eiffel Software (Arno)
|-
| Argument parser
| Done
| Eiffel Software
|-
| EiffelLex
|
|
|-
| EiffelParse
|
|
|-
| EiffelNet
| In Progress
| Eiffel Software (Manu)
|-
| EiffelNet IPv6
|
|
|-
| EiffelCurl
|
|
|-
| Encoding
| Done
| Eiffel Software (Ted, Ian)
|-
| EiffelCOM
|
|
|-
| EiffelStore
|
|
|-
| EiffelTesting
|
|
|-
| EiffelWeb
|
|
|-
| Gobo
| In progress
| Eiffel Software (Jocelyn,Larry)
|-
| Docking
|
|
|-
| Gobo extension
|
|
|-
| EiffelGraph
|
|
|-
| Memory Analyzer
|
|
|-
| EiffelPreferences
|
|
|}

== Contributing ==
EiffelStudio is open source and welcomes the Eiffel community contributions to speed up the adaptation process. If you are interested in participating please put a comment on the discussion board with your contact details.

==Rules to be applied ==

Please observe the following guidelines carefully to guarantee a quality result.

=== Examples ===
For examples of libraries already adapted, see UUID (for a small example) and EiffelBase (for a larger one).

=== Overall process ===

* First compile with the `full_class_checking' option on. Then enable the void-safe option.
* Compile libraries on all of Windows/.NET/Unix to ensure it is sound.

* Minimize modifications; types should be attached by default if it makes sense, otherwise it has to be detachable by default.
* Use the convention library-safe.ecf for naming void-safe libraries for now. All library references should be using the -safe.ecf variants.
* Use the same UUIDs for void-safe and non-void-safe libraries.
* Before any modifications add a library.lic and library-safe.lic (replace library with the name of the ECF minus the .ecf extension) next to the ECFs of the same name containing only the single line reference:forum2.
* Update all samples to use the void-safe ecfs and update them.

=== Rules ===
* DO NOT USE '''!''' (attached mark).
* MINIMIZE USE OF OBJECT TEST; ideally, don't use object test unless there was an assignment attempt in the original library.
* When a precondition expects a Void argument, use '''?''' if attached by default.
* When a precondition expects a non-Void argument, use '''!''' if detachable by default.
* Libraries should compile in both void-safe and non-void-safe mode.
* Only use the '''attribute''' keyword when it is impossible to initialize an attribute in the creation procedure. Never use it for lazy evaluation.
* You may include preconditions x /= Void, but it will have to be removed in the end (helped by a compiler warning that says this is not needed for attached x).

=== General cleanup ===
The void-safe adaptation process should be accompanied by a general upgrade to ISO/ECMA Eiffel:

* Remove uses of is_equal and equal to compare objects. (They can cause catcalls.) Replace them with the tilde operator, i.e. a ~ b instead of equal (a, b) or a.is_equal (b). Be careful to preserve the semantics (~ always returns false in the case of non-identical types).
* Replace the '''indexing''' keyword with '''note'''.
* Remove the '''is''' keyword in routines. Use the Replace tool with the regex '''\ is[ \t]*$'''. (Be careful not to use replace all, because comments and multi-line strings may have "is" text!)
* Replace the '''is''' keyword in constants with '''='''.

=== Test authoring ===
1. Create a cluster called 'tests' in the library root folder. E.g., for the UUID library the 'tests' folder exists at '$ISE_LIBRARY/uuid/tests'.

2. In the library ECFs, exclude the 'tests' cluster because it contains testing code and not library code.

3. Add a testing 'tests.ecf' in the 'tests' folder. (See the UUID library for an example ECF.) Be sure to create a library ECF and change the UUID. The library should also use the void-safe options found in the associated library's ECF.

4. Create test class names using the library name along with TEST as a prefix:
EiffelBase = BASE_TEST_
EiffelThread = THREAD_TEST_
EiffelVision2 = EV_TEST_ or VISION2_TEST_

=== Improving this page ===

As you encounter problems and devise your solutions, please include the results of your experience here.

2008-06-20T22:03:38Z

Arnofiva: Description for current service interfaces

[[Category:Testing]]

== Test system architecture ==

* separate system for executing tests
* system is only responsible for running tests, without determine whether a test fails or not -> compiling and launching system is done by tool, which is also oracle
* tests are compiled up to degree 3 in development system, so user can edit tests like normal classes, but do not affect resulting binary
* test referenced by root class of development system will be compiled anyway, this way we can also run tests in debugger
* in CDD: test system is implicit target (does not have to be in ecf) which inherits from development target, test target simply defines new root class/feature
* re-use development EIFGEN (copy) so test system does not need to compile from scratch?

<pre>

EiffelStudio <-----------------------------> Testing tool <-----------------------------> Test executor

* Show tests in system * can be part of EiffelStudio or * execute test in safe environment
* Show test result compiled as separate tool to be (executor is allowed to crash)
* Provide test creation wizards used e.g. through console
* Interface for CDD, Auto tests, * compile test executor
creating manual tests, running * distribute test executors to
* provide ESF service for different machines
testing/test results * schedule test execution
* provide test results
* find all tests for a given ecf file
* write root class for test executor

CDD
* implemented partially in
debugger/executable
* should be part of any Eiffel
application, that way test can be
created for bug submitting
* extraction can be initiated
through debugger, breakpoints,
failure window, etc.

Auto Test
* separate tool, interface in EiffelStudio

</pre>

== Provide testing as a service in EiffelStudio ==

Using ESS, we can provide all testing functionality as a service within EiffelStudio. That way other tools can make use of these testing functionalities. Also the tool won't have to access the implementation directly. This is a short description of interfaces so far created.

So far the service consists of tree major parts: the test suite storing all tests, test execution and test creation. The service already includes more than 20 interface classes, so it will be important to find a good abstraction. Another aspect is that some parts of the service should be extendible. Clients should be able to define new types of tests, executors or factories.

=== TEST_SUITE_S ===

The test suite is the first instance of the service. It has the list of all tests in the system and controls all execution of tests. Right now the service has the restriction that only one executor can run at a time. Although there might not be a reason against having to executors running in parallel, it will make observing the execution of tests much simpler. Whereas factories can be launched by anyone and so run in parallel. In that case clients are usually interested when a new test is created, for which events already exist in the test suite (see below).

Changes in the test suite can be observed, so if tests are added, removed or modified clients can be notified. There are also events for activating or deactivating an executor in the test suite.

The test suite also provides two registrars where new executors or factories can be registered. Later clients can query whether a certain executor/factory is available and use it if so. More on executors and factories later.

=== EIFFEL_TEST_I ===

EIFFEL_TEST_I will be the common test representation for now. It inherits from a more general interface TEST_I which enables users to introduce new types of tests (not necessarily written in Eiffel). TEST_I inherits from a class TAGABLE, which means that all tests have a list of tags represented by string ([[Testing_Tool_(Specification)#Tags|Tags section of specifications]]). This allows us to have common used functionality in the service itself, like filtering (see FILTERED_COLLECTION_I). Also it enables the user to introduce his own attributes for tests.

EIFFEL_TEST_I points to the abstract syntax representation of its routine and class the routine is located. This is useful to the implementation but could be also to clients. However implementation wise all relevant information should be accessible (such as feature name and tags in the indexing clause).

All tests have a list of outcomes from previous execution sessions. More on that is explained in the next section.

=== TEST_EXECUTOR_I ===

This is a general interface for executing tests. It takes a list of tests and executes each of them. One restriction it imposes to its implementers is that the execution is non blocking. This means that `'''run'''' will return immediately and all tests are executed asynchronously. This again makes it simpler for clients to use (especially graphical UIs).

All state changes of TEST_EXECUTOR_I can be observed by inheriting TEST_EXECUTOR_OBSERVER and connecting to the executor.

As mentioned above, TEST_I keeps a list of outcomes produced by TEST_EXECUTOR_I. In the case of EIFFEL_TEST_I the list contains items of type EIFFEL_TEST_OUTCOME_I. Each outcome points to a EIFFEL_TEST_ROUTINE_INVOCATION_RESPONSE_I which describes one stage of a test execution. The tree stages are namely setup, test and tear down. Test just means calling the actual testing routine. Based on the responses of each stage, EIFFEL_TEST_OUTCOME_I determines whether a test passes or fails. In cases where it cannot be determined because the execution ran unexpected, an outcome is flagged unresolved. In that case the test need to be inspected which is expressed as `'''is_maintenance_required'''' in EIFFEL_TEST_OUTCOME_I.

=== TEST_FACTORY_I ===

Factories are similar to executors since they are registered in the test suite and once triggered run asynchronously. A test factory takes a TEST_CONFIGURATION_I, which describes properties of a new test. There is a specialized version EIFFEL_TEST_CONFIGURATION_I for Eiffel tests (including class names, location, features and classes being tested by the new test).
So far the notification is kept simple by providing a call back function to the '''run''' routine of the factory. This is because clients will be notified anyway when a new test is added to the system through the test suite.

This pattern should also be valid for test generation and extraction (Auto Test/CDD). Where the factory might not create a single test but multiple ones.

== Communication between tool and test executor ==

=== Protocol ===

'''From tool to executor'''

* name(s) of test to execute
* quit

'''From executor to tool'''

* test result
* text output produced by test
* exception details (type, tag, feature, class? occurred during set up, test, tear down?)
* call stack for exception

=== Open questions ===

* executor per machine/processor?
* text based/object base communication?

== See also ==

* [[Testing Tool (Specification)]]

Testing Tool (Architecture)

2008-06-20T20:38:16Z

Arnofiva: Added service section

Tutorial: Creating a Service

2008-06-16T23:14:45Z

Arnofiva: typo

[[Category:Extending EiffelStudio]]
[[Category:EiffelStudio Services]]
{{UnderConstruction}}
In this tutorial I'll demonstrate the process for integrating third-party [[EiffelStudio_Service|services]] inside [[EiffelStudio]] and hooking up internal parts of [[EiffelStudio]] to use the new service.

Before we begin you should have a fundamental understanding of what a service is and a clear understanding of the guidelines for writing service

{{Note|This tutorial is followed up by another tutorial for creating an EiffelStudio tool for displaying information published by the service.}}

== Getting Started ==
When extending EiffelStudio, it is a good idea to separate your code from the EiffelStudio code. The [[Customizing the EiffelStudio Project]] page describes the process of doing this.

== Overview ==
This tutorial will showing you how to create a service logger service, used to log messages. The logger service will actually be a simplied version of the logger service already available in [[EiffelStudio]], <e>LOGGER_S</e>.

The tutorial will cover creating and implementing a service, registering a service, adding eventing and finally consuming the service and using an service observer.

Although the service contains a simple interface, it's actually quite complete in that the service itself will make use of the [[Event List Service]] as a demonstration how reusing services in [[EiffelStudio]] can make development strategies quicker and easier.

== Creating a Service Interface ==
The very first step in creating a service is to define a service interface. A service interface should contain only deferred routines or deferred routines with effective routines that only reference the service interface directly or other interfaces in the [[EiffelStudio]] [[EiffelStudio Ecosystem|ecosystem]].

{{Note|It is important that the interface abstraction allows for complete freedom to be given to the implementation of the service. Implementation details are not public and should remain that way. No consumer of the service should ever attempt to reverse assign a retrieve service to the implementation class but to the interface class. Consumer of the service should not have to rely on the implementation details of a service and doing so will potentially break code in the future or if a different implementation is returned than expected when querying to a specific service.}}

This tutorial is creating a logger service so it makes senses we should create a service interface class call <eiffel>LOGGER_SERVICE_S</eiffel>. Create a deferred class <eiffel>LOGGER_SERVICE_S</eiffel> in your extension project cluster.

{{Note|All service interfaces by convention are suffixed <eiffel>_S</eiffel>. This makes it clear to a consumer that they are using a service interface. All other related interfaces for the service should be suffixed <eiffel>_I</eiffel> to indicate an interface.}}

The first step is to define <eiffel>SIMPLE_LOGGER_S</eiffel> as an actually service interface. In order to achieve this <eiffel>SIMPLE_LOGGER_S</eiffel> must inherit a service base interface <eiffel>SERVICE_I</eiffel>. As of [[EiffelStudio]] [[EiffelStudio 6.1 Releases|6.1]] <eiffel>SERVICE_I</eiffel> does not contain any effective or deferred routines, it is merely a place holder for future additions and a method of classification. It does however inherit another service class <e>SITE</e>, which will be talked about it later.

So now you should have something looking like this:

<e>
deferred class
SIMPLE_LOGGER_S

inherit
SERVICE_I

end
</e>

Of course this doesn't do all that much, in fact it does nothing! We need to add a way to log messages. For this we'll add <e>put</e> routines; <e>put_message</e> and <e>put_message_with_severity</e>.

A logger shouldn't just simply log a message, it's just not powerful enough. So the put routines for the service should permit categorization and even indicate a level of severity in case a logger service consumer deems that a particular entry deserves more or less attention. Fortunately ESS offers built in support for categorization and a basic priority level, which will serve quite nicely as a translation for a log item severity level.

=== Categories and Priorities ===
<eiffel>ENVIRONMENT_CATEGORIES</eiffel> is a class consisting of constants defining EiffelStudio environment region categories. There are constants for the compiler, debugger the editor and so forth. As extenders you are free to add your own categories and utilize them. Any class can access a single instance of <eiffel>ENVIRONMENT_CATEGORIES</eiffel> through <eiffel>SHARED_ENVIRONMENT_CATEGORIES.categories</eiffel>.

<eiffel>PRIORITY_LEVELS</eiffel> is another class containing constants for basic priority levels; high, normal and low. Any class can access a single instance of <eiffel>PRIORITY_LEVELS</eiffel> through <eiffel>SHARED_PRIORITY_LEVELS.priorities</eiffel>.

We want to make use of both categories and priorites in the logger service so <eiffel>LOGGER_SERVICE_S</eiffel> should inherit both <eiffel>SHARED_ENVIRONMENT_CATEGORIES</eiffel> and <eiffel>SHARED_PRIORITY_LEVELS</eiffel>.

{{Warning|Inheriting the shared classes should not affect the service interface so be sure to set the export status when inheriting those shared classes!}}

<eiffel>ENVIRONMENT_CATEGORIES</eiffel> and <eiffel>PRIORITY_LEVELS</eiffel> in addition to being constant definition classes, also contain validation functions to ensure a category identifier is a known identifiers, as it true for a priorty identifier. In the practice of [[Design by Contract]] our service routines are going to be passed a category and severity (priority) level, which require validation. Given <eiffel>SHARED_ENVIRONMENT_CATEGORIES.categories</eiffel> and <eiffel>SHARED_PRIORITY_LEVELS.priorities</eiffel> are not exported members of the interface we'll need to create proxy query functions, which is actually good design. These proxy function can then be used an service routine preconditions and can also be used by a service consumer client when making the call to one of the service routines.

Below is the complete code for adding categories and severity levels to the logger service interface. The proxy function <eiffel>is_valid_category</eiffel> has been added for category validation and <eiffel>is_valid_severity_level</eiffel> added for severity level validation.

<eiffel>
deferred class
LOGGER_SERVICE_I

inherit
SERVICE_I

SHARED_ENVIRONMENT_CATEGORIES
export
{NONE} all
end

SHARED_PRIORITY_LEVELS
export
{NONE} all
end

feature -- Query

frozen is_valid_category (a_cat: NATURAL_8): BOOLEAN
-- Determines if `a_cat' is a valid logger category
--
-- `a_cat': A category identifier to validate.
-- `Result': True to indicate the category is valid; False otherwise.
do
Result := categories.is_valid_category (a_cat)
end

frozen is_valid_severity_level (a_level: INTEGER_8): BOOLEAN
-- Determines if `a_level' is a valid severity level
--
-- `a_level': A severity level.
-- `Result': True to indicate the level of severity is valid; False otherwise.
do
Result := priorities.is_valid_priority_level (a_level)
end

end
</eiffel>

== Adding Service Functionality ==

Still, the logger service has not functionality. The service interface is now at a stage where the actual service routines can be added. We're going to add three routines; two to log messages and another to clear the log.

{{Note|The logger created here is very simple. It would be highly likely for you to add routines to flush log entries and even provide a mutable status attribute to set an auto-flush mode. It's actually important to realize your design before releasing a service in a version of EiffelStudio, because once released then service interface may be used by others. In our case there is no flush routine, which means a later implementation of the logger service who's message flushing capabilities are expensive, will suffer bad performance penalties. The interface was already released without a flush routine so now a flush has to be performed every time a log message is added because existing consumer clients are not using the new service version's flush routine.

When designing a service it's necessary to think how the service might be used by EiffelStudio, the Eiffel compiler and what might happen in the future. In the case of the logger you may have one EiffelStudio SKU that presents logged information in an embedded EiffelStudio tool, in another it may be pushed to the OS event log, in another it may be written to a file. Or, you might have all three available and a preference to indicate how added log messages are handled.}}

Here is the basic interface with the previous interface members elided for clarity.

<eiffel>
deferred class
LOGGER_SERVICE_I

...

feature -- Extension

put_message (a_msg: STRING_32; a_cat: NATURAL_8)
-- Logs a message.
--
-- `a_msg': Message text to log.
-- `a_cat': A message category, see {ENVIRONMENT_CATEGORIES}.
require
a_msg_attached: a_msg /= Void
not_a_msg_is_empty: not a_msg.is_empty
a_cat_is_empty_is_valid_category: is_valid_category (a_cat)
do
put_message_with_severity (a_msg, a_cat, {PRIORITY_LEVELS}.normal)
end

put_message_with_severity (a_msg: STRING_32; a_cat: NATURAL_8; a_level: INTEGER_8)
-- Logs a message specifying a severity level.
--
-- `a_msg': Message text to log.
-- `a_cat': A message category, see {ENVIRONMENT_CATEGORIES}.
-- `a_level': A severity level for the message, See {PRIORITY_LEVELS}.
require
a_msg_attached: a_msg /= Void
not_a_msg_is_empty: not a_msg.is_empty
a_cat_is_empty_is_valid_category: is_valid_category (a_cat)
a_level_is_valid_severity_level: is_valid_severity_level (a_level)
deferred
end

feature -- Removal

clear_log
-- Clear any cached log data
deferred
end

...

end
</eiffel>

The simpler <eiffel>put_message</eiffel> service routine has already been implemented by calling the more specific <eiffel>put_message_with_severity</eiffel>, using a default severity level. Now our service has a simple and more specific versions of logging routines with zero-cost to a service implementer, and value-add to the logger service clients as they will not have to use the specific version each time a log message is to be added.

=== Adding Events ===
To be a good service citizen of EiffelStudio it is highly desirable to provide events service consumers can hook up to. Not all services have events but it so happens that the logger service is interacted with in a way that tools or other services may be interested in; a message is added and messaged are cleared.

Griffin provides its own even mechanism using <eiffel>EVENT_TYPE</eiffel>, which is an extremely powerful event abstraction that is simple to use.

To facilitate event hooks we'll add the events <eiffel>message_logged_events</eiffel> to notify subscribes when a message is added, and <eiffel>cleared_events</eiffel> to notify subscribes when a clear operation was performed.

<eiffel>
deferred class
LOGGER_SERVICE_I

...

feature -- Events

message_logged_events: EVENT_TYPE [TUPLE [service: LOGGER_SERVICE_I; message: STRING_32;
category: NATURAL_8; level: INTEGER_8]]
-- Events called when a message has been logged
deferred
result_attached: Result /= Void
result_consistent: Result = Result
end

cleared_events: EVENT_TYPE [TUPLE [service: LOGGER_SERVICE_I]]
-- Events called when the messages have been cleared from the log
deferred
result_attached: Result /= Void
result_consistent: Result = Result
end

...

end
</eiffel>

Note, the events are deferred also. This given a logger service implementation the option to implement the events as attributes or deferred-evaluation functions, for performance and memory footprint optimizations. The postcondition <eiffel>result_consistent</eiffel> ensures that any deferred-evaluation/once-per-object implementation actually performs the correct per-object caching.

Respecting the events could be implemented using lazy-evaluation no class invariants have been added to ensure the event attributes are always attached, because (A) in deferred-evaluation they may not be attached until called and (B) evaluating the class invariants would remove any optimization benefits of deferred-evaluation as they would be evaluated after the service has been created.

{{Note|For events implemented as attributes it's desirable for the implementation to add the appropriate invariants to ensure the events at in an attached state after the logger service has been created.}}

== Creating a Consumer ==
Consumer helper classes are a nice addition to adding a new service. It makes working with an added service so much easier and it take only a minute to create a consumer.

A consumer is a helper class that provides cached access to a service. Service consumers can then simply inherit one or more consumer helper classes to gain access to desired services.

{{Note|As a convention all service consumer helper classes are suffix by <eiffel>_SERVICE_CONSUMER</eiffel>.}}

Below is literally all the code you need to create a consumer helper class for your service. It simply renames the features from a generic class base class as to provide non-conflict feature names when using multiple service consumer helper classes from a single class.

<eiffel>
class
LOGGER_SERVICE_CONSUMER

inherit
SERVICE_CONSUMER [LOGGER_SERVICE_S]
rename
service as logger_service,
is_service_available as is_logger_service_available,
internal_service as internal_logger_service
end

end
</eiffel>

== Creating the Service Implementation ==
The ground work has been laid for basing the implementation of the logger service on. In this tutorial we are simply going to make use of the [[Event List Service]], which does an fantastic job of proving facilities for adding and removing added object (call event items). It means our implementation is basically a proxy to another service. Using another service saves time and effort to go from design to integration. The additional benefit of using the [[Event List Service]] is that an EiffelStudio tool can be created very quick to display the logged messages because EiffelStudio foundations provides base implementation for tools built using consuming the [[Event List Service]] (for those that are interested see <eiffel>ES_EVENT_LIST_TOOL_BASE</eiffel> and <eiffel>ES_CLICKABLE_EVENT_LIST_TOOL_BASE</eiffel>.)

Below is the stub implementation for the effective logger service.

{{Note|Just like service interfaces, interfaces and service consumers, effective service classes should always yield the name of the service with the <eiffel>_S</eiffel> suffix removed.}}

<eiffel>
class
LOGGER_SERVICE

inherit
LOGGER_SERVICE_S

SAFE_AUTO_DISPOSABLE

EVENT_LIST_SERVICE_CONSUMER
export
{NONE} all
end

create
make

feature {NONE} -- Initialization

make
-- Initialize logger service.
do
-- Initialize events
create message_logged_events
create cleared_events

-- Set up automatic cleaning of event object
auto_dispose (message_logged_events)
auto_dispose (cleared_events)
end

feature -- Extension

put_message_with_severity (a_msg: STRING_32; a_cat: NATURAL_8; a_level: INTEGER_8)
-- Logs a message specifying a severity level.
--
-- `a_msg': Message text to log.
-- `a_cat': A optional message category.
-- `a_level': A serverity level for the message.
do
end

feature -- Removal

clear_log
-- Clear any cached log data
do
end

feature -- Events

message_logged_events: EVENT_TYPE [TUPLE [service: LOGGER_SERVICE_S; message: STRING_32;
category: NATURAL_8; level: INTEGER_8]]
-- Events called when a message has been logged

cleared_events: EVENT_TYPE [TUPLE [service: LOGGER_SERVICE_S]]
-- Events called when the messages have been cleared from the log

invariant
message_logged_events_attached: not is_zombie implies message_logged_events /= Void
cleared_events_attached: not is_zombie implies cleard_events /= Void

end
</eiffel>

The service is an implementation of the previously defined logger service interface (<eiffel>LOGGER_SERVICE_S</eiffel>) so we have to inherit the interface so later the SOA core can validated the service when it's registered, but more on this later.

Also inherited is <eiffel>SAFE_AUTO_DISPOSABLE</eiffel>, a memory resource management base class for handling automatically disposing of class objects when then class object itself is disposed. As the service hosts two events, after creation of those event they are added to the auto-dispose pool to automatic disposal. This saves the logger service from having to implement <eiffel>safe_disposable</eiffel> and performing the resource management manually. For more information on resource management see [[EiffelStudio Memory Management]].

As stated this implementation is actually using the [[Event List Service]] so access to the service is provided using the service consumer <eiffel>EVENT_LIST_SERVICE_CONSUMER</eiffel>.

A creation routine <eiffel>make</eiffel> has been added for the class to create the implemented event attribute objects and register them with the auto disposable pool from <eiffel>SAFE_AUTO_DISPOSABLE</eiffel>. As recommended the events, implemented as attributes, have class invariants to ensure their validity for the life time of the object.

All the other routines are empty stubs waiting to be implemented.

=== Supporting the Event List Service: Event Items ===
The [[Event List Service]] make used of an entity call an [[Event List Service#Event Items|event item]]. So, for the logger service to effectively use the [[Event List Service]] it must implement an [[Event List Service#Event Items|event item]] for a log message.

Fortunately [[Griffin]] already provides much of the implementation to implement basic [[Event List Service#Event Items|event items]] through <eiffel>EVENT_LIST_ITEM</eiffel>. <eiffel>EVENT_LIST_ITEM</eiffel> however does not provide all the implementation required for an logger based-[[Event List Service#Event Items|event item]], there are still a few deferred routines that require implementation. One of these function <eiffel>type</eiffel> is important for agnostic [[Event List Service#Event Items|event item]] identification as the implementation for an [[Event List Service#Event Items|event item]] should not be relied upon by any other part of [[EiffelStudio]] other than the implementation aspect responsible for creating it, in this case the implementation of the logger service - <eiffel>LOGGER_SERVICE</eiffel>.

==== Event List Item Types ====
An [[Event List Service#Event Items|event item]] type corresponds to a type identifier found in <eiffel>EVENT_LIST_ITEM_TYPES</eiffel>. As the logger service is a new service introducing a new type of [[Event List Service#Event Items|event item]] the logger service needs to add a new type identifier. Open <eiffel>EVENT_LIST_ITEM_TYPES</eiffel> and add the following code:

<eiffel>
log: NATURAL_8 = 2
-- Logger event list item type
</eiffel>

Even though the type constant identifier is given the value of ''2'', the value should be unique. If you are using a version of [[EiffelStudio]] where ''2'' is already taken by another type constant identifier, pick the next available index.

==== A Log Event Item Abstraction ====

To support clean abstraction and separation from the underlying implementation, a derived [[Event List Service#Event Items|event item]] interface for a log message should be created, implementing <eiffel>type</eiffel> from <eiffel>EVENT_LIST_ITEM_I</eiffel>. <eiffel>type</eiffel> should return the new type constant identifier - <eiffel>log</eiffel> - added to <eiffel>EVENT_LIST_ITEM_TYPES</eiffel>.

Create a new deferred class <eiffel>LOGGER_EVENT_LIST_ITEM_I</eiffel> and copy the following code into it:

<eiffel>
deferred class
LOGGER_EVENT_LIST_ITEM_I

inherit
EVENT_LIST_ITEM_I

feature -- Access

frozen type: NATURAL_8
-- Event list item type identifier, see {EVENT_LIST_ITEM_TYPES}
once
Result := {EVENT_LIST_ITEM_TYPES}.log
end

end
</eiffel>

==== A Log Event Item Implementation ====

Using the newly create log interface <eiffel>LOGGER_EVENT_LIST_ITEM_I</eiffel>, create a new class <eiffel>LOGGER_EVENT_LIST_ITEM</eiffel>, which will be the [[Event List Service#Event Items|event list item]] <eiffel>LOGGER_SERVICE</eiffel> will use to push log messages to the [[Event List Service]].

Copy and paste the following code into the new class:

<eiffel>
class
LOGGER_EVENT_LIST_ITEM

inherit
LOGGER_EVENT_LIST_ITEM_I

EVENT_LIST_ITEM
rename
make as make_event_list_item
end

create
make

feature {NONE} -- Initialization

make (a_category: like category; a_description: like description; a_level: like priority)
-- Initialize a new event list error item.
--
-- `a_category': Log category, see {ENVIRONMENT_CATEGORIES}.
-- `a_description': Log message.
-- `a_level': Serverity level of the logged message.
require
a_category_is_valid_category: is_valid_category (a_category)
a_description_attached: a_description /= Void
not_a_description_is_empty: not a_description.is_empty
a_level_is_valid_priority: is_valid_priority (a_level)
do
make_event_list_item (a_category, a_level, Void)
description := a_description
ensure
category_set: category = a_category
description_set: description = a_description
priority_set: priority = a_level
end

feature -- Access

description: STRING_32
-- Log message description

feature -- Query

is_valid_data (a_data: like data): BOOLEAN
-- Determines is the user data `a_data' is valid for the current event item.
--
-- `a_data': The user data to validate.
-- `Result': True if the user data is valid; False otherwise.
do
Result := True
end

invariant
description_attached: description /= Void
not_description_is_empty: not description.is_empty

end
</eiffel>

The new logger service [[Event List Service#Event Items|event list item]] implements the deferred functions <eiffel>EVENT_LIST_ITEM</eiffel> leaves deferred. Namely <eiffel>description</eiffel> and <eiffel>is_valid_data</eiffel>.

<eiffel>description</eiffel> is implemented as an attribute and the class invariants taken from <eiffel>EVENT_LIST_ITEM_I.description</eiffel>'s postconditions. The description will serve a the holder of a log message.

<eiffel>is_valid_data</eiffel> determines if an custom data is valid for the [[Event List Service#Event Items|event list item]] but seeing as not custom data is used by the logger [[Event List Service#Event Items|event list item]] it's implemented returning always <eiffel>True</eiffel>.

Finally, <eiffel>LOGGER_EVENT_LIST_ITEM</eiffel>'s creation routine <eiffel>make</eiffel> is used to set the information passed through <eiffel>LOGGER_SERVICE_I.put_message_with_severity_level</eiffel> on the log [[Event List Service#Event Items|event list item]]. The <eiffel>category</eiffel> and <eiffel>priority</eiffel> are members of <eiffel>EVENT_LIST_ITEM</eiffel> and are effective implementations of the deferred parent declarations of <eiffel>EVENT_LIST_ITEM_I</eiffel>

With the log [[Event List Service#Event Items|event list item]] created it's time to finish up the implementation of the logger service.

=== Implementing the Service Routines ===
The final stage to complete the implementation of the logger service is to implement the deferred features of <eiffel>LOGGER_SERVICE_S</eiffel>. The logger service is quite compact and only two features require implementing; <eiffel>put_message_with_severity</eiffel> and <eiffel>clear_log</eiffel>.

<eiffel>put_message_with_severity</eiffel> will basically take the information passed in and create an instance of <eiffel>LOGGER_EVENT_LIST_ITEM</eiffel> and push it to the [[Event List Service]], if it's available.

{{Warning|When working with services it is important to remember that they might not be available for one reason or another, even if they are available when you are debugging. There are multiple reasons for their absence, one being a service did not make the final release because of bugs and time constraints.}}

In order to push and remove ###event list items### to and from the [[Event List Service]], the [[Event List Service]] interface requires a "[[Event List Service#Context Cookie|Context Cookie]]". A context cookie allows the [[Event List Service]] to track where ###event list items### were added from. Using a [[Event List Service#Context Cookie|context cookie]] enables tools and services to remove all ###event list items### added by that tool or service in a single step. The benefit of this, apart from simplicity, is the tool or service does not have to manually track what it pushed to the [[Event List Service]] for later removal.

Adding the following code to <eiffel>LOGGER_SERVICE</eiffel> will support <eiffel>put_message_with_severity</eiffel> and <eiffel>clear_log</eiffel> in their endeavors:

<eiffel>
feature {NONE} -- Access

context_cookie: UUID
-- Context cookie for event list service
once
create Result.make_from_string ("E1FFE100-0106-4145-A53F-ED44CE92714D")
end
</eiffel>

<eiffel>put_message_with_severity</eiffel> is not done with. The logger service exposes events, one of which should be published to any subscribers when a message is logged. <eiffel>put_message_with_severity</eiffel> will need to publish the event also.

The full implementation of <eiffel>put_message_with_severity</eiffel> looks like this:

<eiffel>
feature -- Extension

put_message_with_severity (a_msg: STRING_32; a_cat: NATURAL_8; a_level: INTEGER_8)
-- Logs a message specifiying a severity level.
--
-- `a_msg': Message text to log.
-- `a_cat': A optional message category.
-- `a_level': A severity level for the message.
local
l_item: like create_event_list_log_item
do
if is_event_list_service_available then
l_item := create_event_list_log_item (a_msg, a_cat, a_level)
event_list_service.put_event_item (context_cookie, l_item)
end

-- Publish events
message_logged_events.publish ([Current, a_msg, a_cat, a_level])
end
</eiffel>

To support better design, <eiffel>put_message_with_severity</eiffel> should not actually create an instance of <eiffel>LOGGER_EVENT_LIST_ITEM</eiffel> directly. It's possible for another party to take and extend the logger service creating even more specialized log ##event list items###, in which case the descendant logger service should not have to reimplement <eiffel>put_message_with_severity</eiffel> in order to change the type of log ##event list items### pushed to the [[Event List Service]]. To facilitate, instead a factory function will be used to create the log ##event list items###, allowing service extenders to create specialize log ##event list items###.

The factory function, used to create the log ###event list item###:

<eiffel>
feature {NONE} -- Factory

create_event_list_log_item (a_msg: STRING_32; a_cat: NATURAL_8;
a_level: INTEGER_8): EVENT_LIST_LOG_ITEM_I
-- Creates a new event list item for a log message.
--
-- `a_msg': Message text to log.
-- `a_cat': A message category, see {ENVIRONMENT_CATEGORIES}.
-- `a_level': A severity level for the message, See {PRIORITY_LEVELS}.
-- `Result': An event list service item.
require
a_msg_attached: a_msg /= Void
not_a_msg_is_empty: not a_msg.is_empty
a_cat_is_empty_is_valid_category: is_valid_category (a_cat)
a_level_is_valid_severity_level: is_valid_severity_level (a_level)
do
create {EVENT_LIST_LOG_ITEM} Result.make (a_cat, a_msg, a_level)
ensure
result_attached: Result /= Void
result_is_log_item: Result.type = {EVENT_LIST_ITEM_TYPES}.log
end
</eiffel>

The last part to complete the logger event service is implementing <eiffel>clear_log</eiffel>.

<eiffel>clear_log</eiffel> is contains fairly rudementry logic thanks to the [[Event List Service]]. In a single line of code the logger service can remove all pushed log ##event list items### from the [[Event List Service]] using:

<eiffel>
event_list_service.prune_event_items (context_cookie)
</eiffel>

In addition to removing all the pushed log ##event list items### <eiffel>clear_log</eiffel> must also publish the event <eiffel>cleared_events</eiffel> to notify all subscribes of the purge.

The full implementation of <eiffel>clear_log</eiffel> is as follows:

<eiffel>
feature -- Removal

clear_log
-- Clear any cached log data
do
if is_event_list_service_available then
event_list_service.prune_event_items (context_cookie)
end

-- Publish events
cleared_events.publish ([Current])
end
</eiffel>

That's it! Coming this far you have created all of the interfaces and implementation needed to use the service. All that's left to do now is to proffer the service so other tools and services can make use of the logger service.

== Proffering a Service ==
To permit access to the logger service you need to register the service with a [[Service#Service Containers|service container]. Typically most [[Service|services]] in [[EiffelStudio]] will be registered in <eiffel>ES_ABSTRACT_GRAPHIC.add_core_services</eiffel>.

In <eiffel>ES_ABSTRACT_GRAPHIC.add_core_services</eiffel> add the following line of code to register the logger service.

<eiffel>
a_container.add_service_with_activator ({LOGGER_SERVICE_S},
agent create_logger_service, False)
</eiffel>

The above line of code registers the logger service with a "[[Services#Delayed Activation|activator]]", which ensures the service is only created when it's requested. This saves on start up performance degradation as well as not using up memory required to instantiate and run the logger service. It may seem small in the scope of a single, tiny service but every little counts and that count grows with ever services registered.

Services are registered using their service interface type. One reason why the <eiffel>_S</eiffel> suffix is used is for identity. When querying for a service you know instinctively to look for a class name ending <eiffel>_S</eiffel>, which is assignable to an Eiffel variable of the same type.

To actually create the logger service, a factory function is used which has the added advantage of allowing descendants to redefine the default service returned (or return no service at all.)

<eiffel>
feature {NONE} -- Service factories

create_logger_service: LOGGER_SERVICE_S
-- Creates the logger service
do
create {LOGGER_SERVICE}Result.make
end
</eiffel>

== Next Steps ==
With the all the code in place, the logger service is read to be interacted with. In the [[Tutorial: Consuming a Service|next tutorial]] on [[Services|services]] we will put the logger service to use in existing functionality in EiffelStudio.

In the [[Tutorial: Creating a Service-Based Tool|final tutorial]] we'll explorer using [[EiffelStudio Foundations]] to create a dockable Eiffel tool to display the logged messages.

== SVN Patch ==
For the full tutorial code, please grab an SVN patch from [[here]].

Testing Tool (Specification)

2008-06-13T20:44:58Z

Arnofiva: Comment on tag syntax

[[Category:Testing]]

{{UnderConstruction}}

== Main functionalities ==

=== Add unit/system level tests ===

Semantically there is no difference between unit tests and system level tests. This way all tests can be written in Eiffel in a conforming way.

A test is a routine having the prefix '''test''' in a class inheriting from '''TEST_SET'''. In general features in classes specifically used for testing should be exported at most to {TESTING_CLASS}. This is to prevent testing code from remaining in a finalized system. If you write a helper class for your test routines, let it inherit from '''TESTING_CLASS''' (Note: '''TEST_SET''' already inherits from '''TESTING_CLASS'''). Additionally you should make leaf test sets frozen and make sure you never directly reference testing classes in your project code.

==== System level test specifics ====

Since system level testing often relies on external items like files, '''SYSTEM_LEVEL_TEST_SET''' provides a number of helper routines accessing them. These classes will probably have to be in an special testing library, since they also make use of other libraries such as the process library.

==== Config file ====

For each target in a configuration file you may define a testing folder in which test classes will be created. Other files needed for testing can be put there as well. It could be included as a special type of cluster, so classes in that folder will be compiled.

Proposal: '''/"location_of_ecf_file"/testing/"target_name"'''

{{Note| Some cases require special testing folder when automatically creating new test cases (e.g. in a writable library, since it might use classes which are not visible from the library). The test executor could use that folder to place log files. Also system level tests rely on a location for files (such as text files containing the expected output).}}

==== Additional information ====

The indexing clause can be used to specify which classes and routines are tested by the test routine. Any specifications in the class indexing clause will apply to all tests in that class. Note '''testing_covers''' in the following examples.

==== Examples ====

Example unit tests '''test_append''' and '''test_boolean'''
<eiffel>

frozen class TEST_STRING

inherit

TEST_SET
redefine
set_up
end

feature {NONE} -- Initialization

set_up
do
create s.make (10)
end

feature {TESTER} -- Access

s: STRING

feature {TESTER} -- Test routines

test_append
indexing
testing: "covers.STRING.append, platform.os.winxp"
require
set_up: s /= Void and then s.is_empty
do
s.append ("12345")
assert_string_equality ("append", s, "12345")
end

test_boolean
indexing
testing: "covers.STRING.is_boolean, covers.STRING.to_boolean"
require
set_up: s /= Void and then s.is_empty
do
s.append ("True")
assert_true ("boolean", s.is_boolean and then s.to_boolean)
end

end

</eiffel>

Example system level test '''test_version''' (Note: '''SYSTEM_LEVEL_TEST_SET''' inherits from '''TEST_SET''' and provides basic functionality for executing external commands, including the system currently under development):

<eiffel>

indexing
testing_covers: "all"

frozen class TEST_MY_APP

inherit

SYSTEM_LEVEL_TEST_SET

feature {TESTER} -- Test routines

test_version
indexing
testing: "platform.os.linux.i386"
do
run_system_with_args ("--version")
assert_string_equality ("version", last_output, "my_app version 0.1")
end

end

</eiffel>

=== Manage and run test suite ===

[[Image:testing_set-view.png|right|400px|thumb|Standard view listing existing test sets and the tests they contain]]
[[Image:testing_cut-view.png|right|400px|thumb|Predefined ''Class tested'' view listing classes/features of the system together with the associated tests (Note: since test_boolean is tagged to cover multiple features, it also appears multiple times in the view)]]
[[Image:testing_user-view.png|right|400px|thumb|User defined view (by simply typing part of the tag), where the tool creates a view based on how the tests are tagged (see Examples above)]]

The tool should have an own icon for displaying test cases (test routines). In this example it is a Lego block. Especially for views like ''list all tests for this routine'', it is important to see the difference between the actual routine and its tests. Also the tool has more of a vertical layout. Since the number of tests is comparable to the number of classes in the system, it makes sense the tools have the same layout. Also it allows to have tabs in the bottom for displaying further information, such as execution details (output, call stack, etc.).

The '''menu bar''' includes following buttons:
* Create new manual test case (opens wizard)
** if test class is dropped on button, the wizard will suggest to create new test in that class
** if normal class (or feature) is dropped on button, wizard will suggest to create test for the class (or feature)
* Menu for generating new test (defaults to last chosen one?)
** if normal class/feature is dropped on button, generate tests for that class/feature

* Menu for executing tests in background (defaults to last chosen one?)
** if any class/feature is dropped on button, run tests associated with class/feature
* Run test in debugger (must have a test selected or dropped on button to start)
* Stop any execution (background or debugger)

* Opens settings dialog for testing

* Status indicating how many tests we have ran so far and
* how many failing ones there are

'''View''' defines in which way the test cases are listed (see below).

'''Filter''' can be used to type keywords for showing only test cases having tags including the keywords (see below). It's a drop down so predefined filter patterns can be used (such as ''outcome.fail'').

The '''grid''' contains a tree view of all test cases (test cases are always in leaves). Multiples columns for more information. Currently there are two indications whether a test fails or not (column and icons). Obviously it only needs one - they are both shown just to see the difference. The advantage with using icons is that less space is needed. Coloring the background of a row containing a failing test case would be an option as well.

==== Tags ====

Each test can have a number of tags. Tags can be a single string or hierarchically structured with dots ('.'). For example, a test with tag ''covers.STRING.append'' means that this test is a regression test for {STRING}.append. There are a number of implicit tags for each test, such like the ''name'' tag ({TEST_STRING}.test_append has the implicit tag ''name.TEST_STRING.test_append'').

{{Note|Tags are defined as strings, but in the view we sometimes want a tag to represent a class, or a feature. The way this is done right now is that the view basically knows if the tag starts with "covers." it is followed by a class and feature name. Another approach would be to define such tags like this: "covers.{CLASS_NAME}.feature_name". This would allow user defined tags to have clickable nodes in the view. We could also introduces other special tags such like dates/times.}}

==== Different views ====

Based on the notion of tags, we are able to define different views. The default view ''Test sets'' simply shows a hierarchical tree for every ''name.X'' tag. This enables us to define more views, such as ''Class tested'', which displays every ''covers.X'' tag. Note that with other tags than ''name.'' some tests might get listed multiple times where other not containing such a tag must be listed explicitly. The main advantage is that the user can define his own views based on any type of tags.

{{Note|The tools should support multiple selection. This is important for executing a number of selected test routines, showing passed execution results, etc. Also when selecting a e.g. class node it should execute all leaves below that node.}}

==== Running tests ====
[[Image:testing_run-menu.jpg]]

The '''run''' menu provides different options for running tests in the background:

* Run all tests in system
* Run currently failing ones
* Run test for classes last modified (better description needed here)
* Only run tests shown below
* Only run tests which are selected below

{{Note|We should have two different views for displaying testing history. One structured by test sessions (list of test execution containing all test routines for each session) and one listing recent executions for a single test routine.}}

=== Generate tests automatically ===

[[Image:testing_generate-menu.jpg]]

The '''generate''' menu lets you generate new tests for all classes in system (randomly picked?) or for classes which where last modified.

=== Extract tests from a running application ===

This is the a simple example of an extracted test case (note that '''EXTRACTED_TEST_SET''' inherits from '''TEST_SET''' and implements all functionality for executing an extracted test).

<eiffel>

class
TEST_STRING_001

inherit

EXTRACTED_TEST_SET

feature {NONE} -- Initialization

set_up_routine is
-- <Precursor>
do
routine_under_test := agent {STRING}.append_integer
end

feature {TESTER} -- Test routines

test_append_integer is
-- Call `routine_under_test' with input provided by `context'.
indexing
tag: "covers.STRING.append_integer"
do
call_routine_under_test
end

feature {NONE} -- Access

context: ARRAY [TUPLE [id: STRING; type: STRING; inv: BOOLEAN; attributes: ARRAY [STRING]]] is
-- <Precursor>
once
Result := <<
["#operand", "TUPLE [STRING, INTEGER]", True, << "#2", "110" >>],
["#2", "STRING", True, << "this is an integer: " >>]
>>
end

end -- class TEST_STRING_001

</eiffel>

'''EXTRACTED_TEST_SET''' implements '''set_up''' (frozen), but has a deferred feature '''set_up_routine''' which assigns the proper agent to '''routine_under_test'''. This basically replaces the missing reflection functionality for calling features. '''context''' is also deferred in '''EXTACTED_TEST_SET''' and contains all data from the heap and call stack which was reachable by the routine at extraction time. Each TUPLE represents an object, where `inv' defines whether the object should fulfill its invariant or not (if the object was on the stack at extraction time, this does not have to be the case).

This design lets us have the entire test in one file. This is practical especially in the situation where a user should submit such a test as a bug report after experiencing a crash.
The drawback currently is that the design only allows us to have one test per class. The reason for that is mainly the set_up procedure. Creating all objects in '''context''' must be done during set_up. If there is a failure, the set_up will be blamed instead of the actual test routine, which makes the test not fail but invalid. This can happen e.g. if one of the objects in the context does not fulfill its invariant, which again could result from simply editing the class being tested. Any suggestions welcome!

=== Background test execution ===
== Open questions ==
(This section should disappear as the questions get answered.)

== See also ==

* [[Testing Tool (Architecture)]]
* [[Eweasel]]
* [[CddBranch]]
* [[Eiffel Testing Tool]]

Testing Tool (Architecture)

2008-06-12T23:57:59Z

Arnofiva: Moved missplaced title

Testing Tool (Specification)

2008-06-12T23:13:10Z

Arnofiva: Comment on covers view

[[Category:Testing]]

{{UnderConstruction}}

== Main functionalities ==

=== Add unit/system level tests ===

Semantically there is no difference between unit tests and system level tests. This way all tests can be written in Eiffel in a conforming way.

A test is a routine having the prefix '''test''' in a class inheriting from '''TEST_SET'''. In general features in classes specifically used for testing should be exported at most to {TESTING_CLASS}. This is to prevent testing code from remaining in a finalized system. If you write a helper class for your test routines, let it inherit from '''TESTING_CLASS''' (Note: '''TEST_SET''' already inherits from '''TESTING_CLASS'''). Additionally you should make leaf test sets frozen and make sure you never directly reference testing classes in your project code.

==== System level test specifics ====

Since system level testing often relies on external items like files, '''SYSTEM_LEVEL_TEST_SET''' provides a number of helper routines accessing them. These classes will probably have to be in an special testing library, since they also make use of other libraries such as the process library.

==== Config file ====

For each target in a configuration file you may define a testing folder in which test classes will be created. Other files needed for testing can be put there as well. It could be included as a special type of cluster, so classes in that folder will be compiled.

Proposal: '''/"location_of_ecf_file"/testing/"target_name"'''

{{Note| Some cases require special testing folder when automatically creating new test cases (e.g. in a writable library, since it might use classes which are not visible from the library). The test executor could use that folder to place log files. Also system level tests rely on a location for files (such as text files containing the expected output).}}

==== Additional information ====

The indexing clause can be used to specify which classes and routines are tested by the test routine. Any specifications in the class indexing clause will apply to all tests in that class. Note '''testing_covers''' in the following examples.

==== Examples ====

Example unit tests '''test_append''' and '''test_boolean'''
<eiffel>

frozen class TEST_STRING

inherit

TEST_SET
redefine
set_up
end

feature {NONE} -- Initialization

set_up
do
create s.make (10)
end

feature {TESTER} -- Access

s: STRING

feature {TESTER} -- Test routines

test_append
indexing
testing: "covers.STRING.append, platform.os.winxp"
require
set_up: s /= Void and then s.is_empty
do
s.append ("12345")
assert_string_equality ("append", s, "12345")
end

test_boolean
indexing
testing: "covers.STRING.is_boolean, covers.STRING.to_boolean"
require
set_up: s /= Void and then s.is_empty
do
s.append ("True")
assert_true ("boolean", s.is_boolean and then s.to_boolean)
end

end

</eiffel>

Example system level test '''test_version''' (Note: '''SYSTEM_LEVEL_TEST_SET''' inherits from '''TEST_SET''' and provides basic functionality for executing external commands, including the system currently under development):

<eiffel>

indexing
testing_covers: "all"

frozen class TEST_MY_APP

inherit

SYSTEM_LEVEL_TEST_SET

feature {TESTER} -- Test routines

test_version
indexing
testing: "platform.os.linux.i386"
do
run_system_with_args ("--version")
assert_string_equality ("version", last_output, "my_app version 0.1")
end

end

</eiffel>

=== Manage and run test suite ===

[[Image:testing_set-view.png|right|400px|thumb|Standard view listing existing test sets and the tests they contain]]
[[Image:testing_cut-view.png|right|400px|thumb|Predefined ''Class tested'' view listing classes/features of the system together with the associated tests (Note: since test_boolean is tagged to cover multiple features, it also appears multiple times in the view)]]
[[Image:testing_user-view.png|right|400px|thumb|User defined view (by simply typing part of the tag), where the tool creates a view based on how the tests are tagged (see Examples above)]]

The tool should have an own icon for displaying test cases (test routines). In this example it is a Lego block. Especially for views like ''list all tests for this routine'', it is important to see the difference between the actual routine and its tests. Also the tool has more of a vertical layout. Since the number of tests is comparable to the number of classes in the system, it makes sense the tools have the same layout. Also it allows to have tabs in the bottom for displaying further information, such as execution details (output, call stack, etc.).

The '''menu bar''' includes following buttons:
* Create new manual test case (opens wizard)
** if test class is dropped on button, the wizard will suggest to create new test in that class
** if normal class (or feature) is dropped on button, wizard will suggest to create test for the class (or feature)
* Menu for generating new test (defaults to last chosen one?)
** if normal class/feature is dropped on button, generate tests for that class/feature

* Menu for executing tests in background (defaults to last chosen one?)
** if any class/feature is dropped on button, run tests associated with class/feature
* Run test in debugger (must have a test selected or dropped on button to start)
* Stop any execution (background or debugger)

* Opens settings dialog for testing

* Status indicating how many tests we have ran so far and
* how many failing ones there are

'''View''' defines in which way the test cases are listed (see below).

'''Filter''' can be used to type keywords for showing only test cases having tags including the keywords (see below). It's a drop down so predefined filter patterns can be used (such as ''outcome.fail'').

The '''grid''' contains a tree view of all test cases (test cases are always in leaves). Multiples columns for more information. Currently there are two indications whether a test fails or not (column and icons). Obviously it only needs one - they are both shown just to see the difference. The advantage with using icons is that less space is needed. Coloring the background of a row containing a failing test case would be an option as well.

==== Tags ====

Each test can have a number of tags. Tags can be a single string or hierarchically structured with dots ('.'). For example, a test with tag ''covers.STRING.append'' means that this test is a regression test for {STRING}.append. There are a number of implicit tags for each test, such like the ''name'' tag ({TEST_STRING}.test_append has the implicit tag ''name.TEST_STRING.test_append'').

==== Different views ====

Based on the notion of tags, we are able to define different views. The default view ''Test sets'' simply shows a hierarchical tree for every ''name.X'' tag. This enables us to define more views, such as ''Class tested'', which displays every ''covers.X'' tag. Note that with other tags than ''name.'' some tests might get listed multiple times where other not containing such a tag must be listed explicitly. The main advantage is that the user can define his own views based on any type of tags.

{{Note|The tools should support multiple selection. This is important for executing a number of selected test routines, showing passed execution results, etc. Also when selecting a e.g. class node it should execute all leaves below that node.}}

==== Running tests ====
[[Image:testing_run-menu.jpg]]

The '''run''' menu provides different options for running tests in the background:

* Run all tests in system
* Run currently failing ones
* Run test for classes last modified (better description needed here)
* Only run tests shown below
* Only run tests which are selected below

{{Note|We should have two different views for displaying testing history. One structured by test sessions (list of test execution containing all test routines for each session) and one listing recent executions for a single test routine.}}

=== Generate tests automatically ===

[[Image:testing_generate-menu.jpg]]

The '''generate''' menu lets you generate new tests for all classes in system (randomly picked?) or for classes which where last modified.

=== Extract tests from a running application ===

This is the a simple example of an extracted test case (note that '''EXTRACTED_TEST_SET''' inherits from '''TEST_SET''' and implements all functionality for executing an extracted test).

<eiffel>

class
TEST_STRING_001

inherit

EXTRACTED_TEST_SET

feature {NONE} -- Initialization

set_up_routine is
-- <Precursor>
do
routine_under_test := agent {STRING}.append_integer
end

feature {TESTER} -- Test routines

test_append_integer is
-- Call `routine_under_test' with input provided by `context'.
indexing
tag: "covers.STRING.append_integer"
do
call_routine_under_test
end

feature {NONE} -- Access

context: ARRAY [TUPLE [id: STRING; type: STRING; inv: BOOLEAN; attributes: ARRAY [STRING]]] is
-- <Precursor>
once
Result := <<
["#operand", "TUPLE [STRING, INTEGER]", True, << "#2", "110" >>],
["#2", "STRING", True, << "this is an integer: " >>]
>>
end

end -- class TEST_STRING_001

</eiffel>

'''EXTRACTED_TEST_SET''' implements '''set_up''' (frozen), but has a deferred feature '''set_up_routine''' which assigns the proper agent to '''routine_under_test'''. This basically replaces the missing reflection functionality for calling features. '''context''' is also deferred in '''EXTACTED_TEST_SET''' and contains all data from the heap and call stack which was reachable by the routine at extraction time. Each TUPLE represents an object, where `inv' defines whether the object should fulfill its invariant or not (if the object was on the stack at extraction time, this does not have to be the case).

This design lets us have the entire test in one file. This is practical especially in the situation where a user should submit such a test as a bug report after experiencing a crash.
The drawback currently is that the design only allows us to have one test per class. The reason for that is mainly the set_up procedure. Creating all objects in '''context''' must be done during set_up. If there is a failure, the set_up will be blamed instead of the actual test routine, which makes the test not fail but invalid. This can happen e.g. if one of the objects in the context does not fulfill its invariant, which again could result from simply editing the class being tested. Any suggestions welcome!

=== Background test execution ===
== Open questions ==
(This section should disappear as the questions get answered.)

== See also ==

* [[Testing Tool (Architecture)]]
* [[Eweasel]]
* [[CddBranch]]
* [[Eiffel Testing Tool]]

Testing Tool (Architecture)

2008-06-12T23:08:01Z

Arnofiva: Scheme of different components needed for test creation/execution

Testing Tool (Architecture)

2008-06-12T22:20:11Z

Arnofiva: Added "See also" section

Testing Tool (Architecture)

2008-06-12T22:18:28Z

Arnofiva: Initial description of tester system and communication

Testing Tool (Specification)

2008-06-12T21:53:41Z

Arnofiva: typo...

[[Category:Testing]]

{{UnderConstruction}}

== Main functionalities ==

=== Add unit/system level tests ===

Semantically there is no difference between unit tests and system level tests. This way all tests can be written in Eiffel in a conforming way.

A test is a routine having the prefix '''test''' in a class inheriting from '''TEST_SET'''. In general features in classes specifically used for testing should be exported at most to {TESTING_CLASS}. This is to prevent testing code from remaining in a finalized system. If you write a helper class for your test routines, let it inherit from '''TESTING_CLASS''' (Note: '''TEST_SET''' already inherits from '''TESTING_CLASS'''). Additionally you should make leaf test sets frozen and make sure you never directly reference testing classes in your project code.

==== System level test specifics ====

Since system level testing often relies on external items like files, '''SYSTEM_LEVEL_TEST_SET''' provides a number of helper routines accessing them. These classes will probably have to be in an special testing library, since they also make use of other libraries such as the process library.

==== Config file ====

For each target in a configuration file you may define a testing folder in which test classes will be created. Other files needed for testing can be put there as well. It could be included as a special type of cluster, so classes in that folder will be compiled.

Proposal: '''/"location_of_ecf_file"/testing/"target_name"'''

{{Note| Some cases require special testing folder when automatically creating new test cases (e.g. in a writable library, since it might use classes which are not visible from the library). The test executor could use that folder to place log files. Also system level tests rely on a location for files (such as text files containing the expected output).}}

==== Additional information ====

The indexing clause can be used to specify which classes and routines are tested by the test routine. Any specifications in the class indexing clause will apply to all tests in that class. Note '''testing_covers''' in the following examples.

==== Examples ====

Example unit tests '''test_append''' and '''test_boolean'''
<eiffel>

frozen class TEST_STRING

inherit

TEST_SET
redefine
set_up
end

feature {NONE} -- Initialization

set_up
do
create s.make (10)
end

feature {TESTER} -- Access

s: STRING

feature {TESTER} -- Test routines

test_append
indexing
testing: "covers.STRING.append, platform.os.winxp"
require
set_up: s /= Void and then s.is_empty
do
s.append ("12345")
assert_string_equality ("append", s, "12345")
end

test_boolean
indexing
testing: "covers.STRING.is_boolean, covers.STRING.to_boolean"
require
set_up: s /= Void and then s.is_empty
do
s.append ("True")
assert_true ("boolean", s.is_boolean and then s.to_boolean)
end

end

</eiffel>

Example system level test '''test_version''' (Note: '''SYSTEM_LEVEL_TEST_SET''' inherits from '''TEST_SET''' and provides basic functionality for executing external commands, including the system currently under development):

<eiffel>

indexing
testing_covers: "all"

frozen class TEST_MY_APP

inherit

SYSTEM_LEVEL_TEST_SET

feature {TESTER} -- Test routines

test_version
indexing
testing: "platform.os.linux.i386"
do
run_system_with_args ("--version")
assert_string_equality ("version", last_output, "my_app version 0.1")
end

end

</eiffel>

=== Manage and run test suite ===

[[Image:testing_set-view.png|right|400px|thumb|Standard view listing existing test sets and the tests they contain]]
[[Image:testing_cut-view.png|right|400px|thumb|Predefined ''Class tested'' view listing classes/features of the system together with the associated tests]]
[[Image:testing_user-view.png|right|400px|thumb|User defined view (by simply typing part of the tag), where the tool creates a view based on how the tests are tagged (see Examples above)]]

The tool should have an own icon for displaying test cases (test routines). In this example it is a Lego block. Especially for views like ''list all tests for this routine'', it is important to see the difference between the actual routine and its tests. Also the tool has more of a vertical layout. Since the number of tests is comparable to the number of classes in the system, it makes sense the tools have the same layout. Also it allows to have tabs in the bottom for displaying further information, such as execution details (output, call stack, etc.).

The '''menu bar''' includes following buttons:
* Create new manual test case (opens wizard)
** if test class is dropped on button, the wizard will suggest to create new test in that class
** if normal class (or feature) is dropped on button, wizard will suggest to create test for the class (or feature)
* Menu for generating new test (defaults to last chosen one?)
** if normal class/feature is dropped on button, generate tests for that class/feature

* Menu for executing tests in background (defaults to last chosen one?)
** if any class/feature is dropped on button, run tests associated with class/feature
* Run test in debugger (must have a test selected or dropped on button to start)
* Stop any execution (background or debugger)

* Opens settings dialog for testing

* Status indicating how many tests we have ran so far and
* how many failing ones there are

'''View''' defines in which way the test cases are listed (see below).

'''Filter''' can be used to type keywords for showing only test cases having tags including the keywords (see below). It's a drop down so predefined filter patterns can be used (such as ''outcome.fail'').

The '''grid''' contains a tree view of all test cases (test cases are always in leaves). Multiples columns for more information. Currently there are two indications whether a test fails or not (column and icons). Obviously it only needs one - they are both shown just to see the difference. The advantage with using icons is that less space is needed. Coloring the background of a row containing a failing test case would be an option as well.

==== Tags ====

Each test can have a number of tags. Tags can be a single string or hierarchically structured with dots ('.'). For example, a test with tag ''covers.STRING.append'' means that this test is a regression test for {STRING}.append. There are a number of implicit tags for each test, such like the ''name'' tag ({TEST_STRING}.test_append has the implicit tag ''name.TEST_STRING.test_append'').

==== Different views ====

Based on the notion of tags, we are able to define different views. The default view ''Test sets'' simply shows a hierarchical tree for every ''name.X'' tag. This enables us to define more views, such as ''Class tested'', which displays every ''covers.X'' tag. Note that with other tags than ''name.'' some tests might get listed multiple times where other not containing such a tag must be listed explicitly. The main advantage is that the user can define his own views based on any type of tags.

{{Note|The tools should support multiple selection. This is important for executing a number of selected test routines, showing passed execution results, etc. Also when selecting a e.g. class node it should execute all leaves below that node.}}

==== Running tests ====
[[Image:testing_run-menu.jpg]]

The '''run''' menu provides different options for running tests in the background:

* Run all tests in system
* Run currently failing ones
* Run test for classes last modified (better description needed here)
* Only run tests shown below
* Only run tests which are selected below

{{Note|We should have two different views for displaying testing history. One structured by test sessions (list of test execution containing all test routines for each session) and one listing recent executions for a single test routine.}}

=== Generate tests automatically ===

[[Image:testing_generate-menu.jpg]]

The '''generate''' menu lets you generate new tests for all classes in system (randomly picked?) or for classes which where last modified.

=== Extract tests from a running application ===

This is the a simple example of an extracted test case (note that '''EXTRACTED_TEST_SET''' inherits from '''TEST_SET''' and implements all functionality for executing an extracted test).

<eiffel>

class
TEST_STRING_001

inherit

EXTRACTED_TEST_SET

feature {NONE} -- Initialization

set_up_routine is
-- <Precursor>
do
routine_under_test := agent {STRING}.append_integer
end

feature {TESTER} -- Test routines

test_append_integer is
-- Call `routine_under_test' with input provided by `context'.
indexing
tag: "covers.STRING.append_integer"
do
call_routine_under_test
end

feature {NONE} -- Access

context: ARRAY [TUPLE [id: STRING; type: STRING; inv: BOOLEAN; attributes: ARRAY [STRING]]] is
-- <Precursor>
once
Result := <<
["#operand", "TUPLE [STRING, INTEGER]", True, << "#2", "110" >>],
["#2", "STRING", True, << "this is an integer: " >>]
>>
end

end -- class TEST_STRING_001

</eiffel>

'''EXTRACTED_TEST_SET''' implements '''set_up''' (frozen), but has a deferred feature '''set_up_routine''' which assigns the proper agent to '''routine_under_test'''. This basically replaces the missing reflection functionality for calling features. '''context''' is also deferred in '''EXTACTED_TEST_SET''' and contains all data from the heap and call stack which was reachable by the routine at extraction time. Each TUPLE represents an object, where `inv' defines whether the object should fulfill its invariant or not (if the object was on the stack at extraction time, this does not have to be the case).

This design lets us have the entire test in one file. This is practical especially in the situation where a user should submit such a test as a bug report after experiencing a crash.
The drawback currently is that the design only allows us to have one test per class. The reason for that is mainly the set_up procedure. Creating all objects in '''context''' must be done during set_up. If there is a failure, the set_up will be blamed instead of the actual test routine, which makes the test not fail but invalid. This can happen e.g. if one of the objects in the context does not fulfill its invariant, which again could result from simply editing the class being tested. Any suggestions welcome!

=== Background test execution ===
== Open questions ==
(This section should disappear as the questions get answered.)

== See also ==

* [[Testing Tool (Architecture)]]
* [[Eweasel]]
* [[CddBranch]]
* [[Eiffel Testing Tool]]

Testing Tool (Specification)

2008-06-12T21:51:40Z

Arnofiva: Added screen shot of user defined view/tags

[[Category:Testing]]

{{UnderConstruction}}

== Main functionalities ==

=== Add unit/system level tests ===

Semantically there is no difference between unit tests and system level tests. This way all tests can be written in Eiffel in a conforming way.

A test is a routine having the prefix '''test''' in a class inheriting from '''TEST_SET'''. In general features in classes specifically used for testing should be exported at most to {TESTING_CLASS}. This is to prevent testing code from remaining in a finalized system. If you write a helper class for your test routines, let it inherit from '''TESTING_CLASS''' (Note: '''TEST_SET''' already inherits from '''TESTING_CLASS'''). Additionally you should make leaf test sets frozen and make sure you never directly reference testing classes in your project code.

==== System level test specifics ====

Since system level testing often relies on external items like files, '''SYSTEM_LEVEL_TEST_SET''' provides a number of helper routines accessing them. These classes will probably have to be in an special testing library, since they also make use of other libraries such as the process library.

==== Config file ====

For each target in a configuration file you may define a testing folder in which test classes will be created. Other files needed for testing can be put there as well. It could be included as a special type of cluster, so classes in that folder will be compiled.

Proposal: '''/"location_of_ecf_file"/testing/"target_name"'''

{{Note| Some cases require special testing folder when automatically creating new test cases (e.g. in a writable library, since it might use classes which are not visible from the library). The test executor could use that folder to place log files. Also system level tests rely on a location for files (such as text files containing the expected output).}}

==== Additional information ====

The indexing clause can be used to specify which classes and routines are tested by the test routine. Any specifications in the class indexing clause will apply to all tests in that class. Note '''testing_covers''' in the following examples.

==== Examples ====

Example unit tests '''test_append''' and '''test_boolean'''
<eiffel>

frozen class TEST_STRING

inherit

TEST_SET
redefine
set_up
end

feature {NONE} -- Initialization

set_up
do
create s.make (10)
end

feature {TESTER} -- Access

s: STRING

feature {TESTER} -- Test routines

test_append
indexing
testing: "covers.STRING.append, platform.os.winxp"
require
set_up: s /= Void and then s.is_empty
do
s.append ("12345")
assert_string_equality ("append", s, "12345")
end

test_boolean
indexing
testing: "covers.STRING.is_boolean, covers.STRING.to_boolean"
require
set_up: s /= Void and then s.is_empty
do
s.append ("True")
assert_true ("boolean", s.is_boolean and then s.to_boolean)
end

end

</eiffel>

Example system level test '''test_version''' (Note: '''SYSTEM_LEVEL_TEST_SET''' inherits from '''TEST_SET''' and provides basic functionality for executing external commands, including the system currently under development):

<eiffel>

indexing
testing_covers: "all"

frozen class TEST_MY_APP

inherit

SYSTEM_LEVEL_TEST_SET

feature {TESTER} -- Test routines

test_version
indexing
testing: "platform.os.linux.x86_64"
do
run_system_with_args ("--version")
assert_string_equality ("version", last_output, "my_app version 0.1")
end

end

</eiffel>

=== Manage and run test suite ===

[[Image:testing_set-view.png|right|400px|thumb|Standard view listing existing test sets and the tests they contain]]
[[Image:testing_cut-view.png|right|400px|thumb|Predefined ''Class tested'' view listing classes/features of the system together with the associated tests]]
[[Image:testing_user-view.png|right|400px|thumb|User defined view (by simply typing part of the tag), where the tool creates a view based on how the tests are tagged (see Examples above)]]

The tool should have an own icon for displaying test cases (test routines). In this example it is a Lego block. Especially for views like ''list all tests for this routine'', it is important to see the difference between the actual routine and its tests. Also the tool has more of a vertical layout. Since the number of tests is comparable to the number of classes in the system, it makes sense the tools have the same layout. Also it allows to have tabs in the bottom for displaying further information, such as execution details (output, call stack, etc.).

The '''menu bar''' includes following buttons:
* Create new manual test case (opens wizard)
** if test class is dropped on button, the wizard will suggest to create new test in that class
** if normal class (or feature) is dropped on button, wizard will suggest to create test for the class (or feature)
* Menu for generating new test (defaults to last chosen one?)
** if normal class/feature is dropped on button, generate tests for that class/feature

* Menu for executing tests in background (defaults to last chosen one?)
** if any class/feature is dropped on button, run tests associated with class/feature
* Run test in debugger (must have a test selected or dropped on button to start)
* Stop any execution (background or debugger)

* Opens settings dialog for testing

* Status indicating how many tests we have ran so far and
* how many failing ones there are

'''View''' defines in which way the test cases are listed (see below).

'''Filter''' can be used to type keywords for showing only test cases having tags including the keywords (see below). It's a drop down so predefined filter patterns can be used (such as ''outcome.fail'').

The '''grid''' contains a tree view of all test cases (test cases are always in leaves). Multiples columns for more information. Currently there are two indications whether a test fails or not (column and icons). Obviously it only needs one - they are both shown just to see the difference. The advantage with using icons is that less space is needed. Coloring the background of a row containing a failing test case would be an option as well.

==== Tags ====

Each test can have a number of tags. Tags can be a single string or hierarchically structured with dots ('.'). For example, a test with tag ''covers.STRING.append'' means that this test is a regression test for {STRING}.append. There are a number of implicit tags for each test, such like the ''name'' tag ({TEST_STRING}.test_append has the implicit tag ''name.TEST_STRING.test_append'').

==== Different views ====

Based on the notion of tags, we are able to define different views. The default view ''Test sets'' simply shows a hierarchical tree for every ''name.X'' tag. This enables us to define more views, such as ''Class tested'', which displays every ''covers.X'' tag. Note that with other tags than ''name.'' some tests might get listed multiple times where other not containing such a tag must be listed explicitly. The main advantage is that the user can define his own views based on any type of tags.

{{Note|The tools should support multiple selection. This is important for executing a number of selected test routines, showing passed execution results, etc. Also when selecting a e.g. class node it should execute all leaves below that node.}}

==== Running tests ====
[[Image:testing_run-menu.jpg]]

The '''run''' menu provides different options for running tests in the background:

* Run all tests in system
* Run currently failing ones
* Run test for classes last modified (better description needed here)
* Only run tests shown below
* Only run tests which are selected below

{{Note|We should have two different views for displaying testing history. One structured by test sessions (list of test execution containing all test routines for each session) and one listing recent executions for a single test routine.}}

=== Generate tests automatically ===

[[Image:testing_generate-menu.jpg]]

The '''generate''' menu lets you generate new tests for all classes in system (randomly picked?) or for classes which where last modified.

=== Extract tests from a running application ===

This is the a simple example of an extracted test case (note that '''EXTRACTED_TEST_SET''' inherits from '''TEST_SET''' and implements all functionality for executing an extracted test).

<eiffel>

class
TEST_STRING_001

inherit

EXTRACTED_TEST_SET

feature {NONE} -- Initialization

set_up_routine is
-- <Precursor>
do
routine_under_test := agent {STRING}.append_integer
end

feature {TESTER} -- Test routines

test_append_integer is
-- Call `routine_under_test' with input provided by `context'.
indexing
tag: "covers.STRING.append_integer"
do
call_routine_under_test
end

feature {NONE} -- Access

context: ARRAY [TUPLE [id: STRING; type: STRING; inv: BOOLEAN; attributes: ARRAY [STRING]]] is
-- <Precursor>
once
Result := <<
["#operand", "TUPLE [STRING, INTEGER]", True, << "#2", "110" >>],
["#2", "STRING", True, << "this is an integer: " >>]
>>
end

end -- class TEST_STRING_001

</eiffel>

'''EXTRACTED_TEST_SET''' implements '''set_up''' (frozen), but has a deferred feature '''set_up_routine''' which assigns the proper agent to '''routine_under_test'''. This basically replaces the missing reflection functionality for calling features. '''context''' is also deferred in '''EXTACTED_TEST_SET''' and contains all data from the heap and call stack which was reachable by the routine at extraction time. Each TUPLE represents an object, where `inv' defines whether the object should fulfill its invariant or not (if the object was on the stack at extraction time, this does not have to be the case).

This design lets us have the entire test in one file. This is practical especially in the situation where a user should submit such a test as a bug report after experiencing a crash.
The drawback currently is that the design only allows us to have one test per class. The reason for that is mainly the set_up procedure. Creating all objects in '''context''' must be done during set_up. If there is a failure, the set_up will be blamed instead of the actual test routine, which makes the test not fail but invalid. This can happen e.g. if one of the objects in the context does not fulfill its invariant, which again could result from simply editing the class being tested. Any suggestions welcome!

=== Background test execution ===
== Open questions ==
(This section should disappear as the questions get answered.)

== See also ==

* [[Testing Tool (Architecture)]]
* [[Eweasel]]
* [[CddBranch]]
* [[Eiffel Testing Tool]]

2008-06-10T20:57:07Z

Arnofiva: Testing tool menu for running test cases

Testing tool menu for running test cases

File:Testing generate-menu.jpg

2008-06-10T20:56:35Z

Arnofiva: Testing tool menu for generating new tests

Testing tool menu for generating new tests