Difference between revisions of "Eweasel How To"

(Getting eweasel)
(Setting up your environment)
 
(20 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Category:Testing]]
+
[[Category:Eweasel]]
 
+
 
==Getting eweasel==
 
==Getting eweasel==
First you need to get the source code of eweasel. It is part of the EiffelStudio SVN repository under trunk/eweasel.
+
First you need to get the source code of eweasel. It is part of the EiffelStudio SVN repository under trunk/eweasel. Let's suppose you have a '''Test''' directory available, here is what you need to do in this directory:
 +
 
 +
svn co https://svn.eiffel.com/eiffelstudio/trunk/eweasel eweasel
  
 
==Setting up an EiffelStudio delivery for eweasel==
 
==Setting up an EiffelStudio delivery for eweasel==
To make things safer, it is recommended that you duplicate your EiffelStudio delivery for the purpose of testing. It is also recommended that you use the source control system to perform this.
+
To perform testing, you need a modified version of the EiffelStudio delivery, refer to those [[Building_a_delivery|instructions]] for building one. Note that you need as many deliveries as platforms you are testing (thankfully the deliveries can be all done in the same location since we distinguish them through the spec/$ISE_PLATFORM/ paradigm).
  
 
==Setting up your environment==
 
==Setting up your environment==
You need to define '''ISE_EIFFEL''', '''ISE_LIBRARY''', '''ISE_PLATFORM''' and '''EWEASEL''' correctly.
+
You need to define the following environment variable:
 +
* '''ISE_EIFFEL''' should point to your EiffelStudio delivery (made above)
 +
* '''ISE_LIBRARY''' should point to the same value as '''$ISE_EIFFEL'''
 +
* '''ISE_PLATFORM''' should be your platform name (e.g. windows, win64, linux-x86, solaris-sparc, etc...). However for .NET, it has to be '''dotnet'''.
 +
* '''EWEASEL''' should be ''Test/eweasel'' (replace Test by the absolute path to the '''Test''' directory)
 +
* '''EWEASEL_OUTPUT''' should be set to a directory location used by the test engine to perform the compilations and keeping any outputs produced by the tests. If not set it will be in '''$EWEASEL/tmp'''.
  
For '''ISE_EIFFEL''', '''ISE_LIBRARY''' and '''ISE_PLATFORM''', the definition is obvious.
+
And you also need to update your '''PATH''' environment variable so it reads:
 +
* On Unix: ''$EWEASEL/spec/$ISE_PLATFORM/bin''
 +
* On Windows: ''$EWEASEL/spec/%ISE_PLATFORM%/bin''
  
For '''EWEASEL''' environment variable:
+
We recommend that you do the above in a script that you will launch each time you want to perform the testing. That way those settings won't interfere with the official delivery of EiffelStudio.
EWEASEL=Test/eweasel
+
  
Then on Windows only, you need to update your C compiler flags so that it always generates a DOS application (on Windows default is a GUI application) because eweasel needs to redirect the output of the applications to a file.
+
===Windows only===
 +
{{Outdated|The steps described in this section were required in the past and are not needed anymore. It is kept for historical reasons for those who would like to run eweasel on old version of the repository.}}
 +
You need to update your C compiler flags so that it always generates a DOS application (on Windows default is a GUI application) because eweasel needs to redirect the output of the applications to a file. It depends on the C compiler:
 +
* For Microsoft, edit the file ''$ISE_EIFFEL/studio/config/$ISE_PLATFORM/msc/config.eif'' and modify it so that it specifies '''-SUBSYSTEM:CONSOLE''' instead of '''-SUBSYSTEM:WINDOWS'''.
 +
* For MinGW, edit the file ''$ISE_EIFFEL/studio/config/$ISE_PLATFORM/mingw/config.eif'' and modify it so that is specifies '''-mconsole''' instead of '''-mwindows'''.
 +
* For Borland, edit the file ''$ISE_EIFFEL/studio/config/$ISE_PLATFORM/bcb/config.eif'' and modify it so that it specifies '''-ap''' instead of '''-aa'''.
  
For Microsoft, edit the file $ISE_EIFFEL/studio/config/windows/msc/config.eif and modify it so that it specifyies SUBSYSTEM:CONSOLE instead of -SUBSYSTEM:WINDOWS.
+
You may also want to read the [[Dr_Watson|Dr Watson]] page as otherwise on non-working tests, the eweasel regression will stop until you click OK on the debugging dialog.
  
For Borland, edit the file $ISE_EIFFEL/studio/config/windows/bcb/config.eif and modify it so that it specifyies -ap instead of -aa.
+
==Compiling eweasel==
 
+
You need to compile the eweasel engine to run the tests. To do so, go in ''$EWEASEL/compilation'' and execute the '''install_eweasel''' script (.bat for windows).
===.NET===
+
If you specify '''dotnet''' for '''ISE_PLATFORM''' the scripts mentioned in the next section will assume you are compiling for .NET.
+
  
 
==Launching the test==
 
==Launching the test==
Line 30: Line 40:
 
* run_eweasel_catalog: run the tests from a specified catalog
 
* run_eweasel_catalog: run the tests from a specified catalog
 
* run_eweasel_filter: run one particular test
 
* run_eweasel_filter: run one particular test
 +
* run_eweasel_keyword: run tests matching a specified keyword
 +
 +
On a recent machine (from 2006), running all the tests should take about 5h or less.
 +
 +
==Analyzing the results==
 +
You should use the [[Eweasel_Comparator|Eweasel Comparator]] tool and compare your results with one of the reference one from the results subdirectory of eweasel.

Latest revision as of 06:15, 19 December 2019

Getting eweasel

First you need to get the source code of eweasel. It is part of the EiffelStudio SVN repository under trunk/eweasel. Let's suppose you have a Test directory available, here is what you need to do in this directory:

svn co https://svn.eiffel.com/eiffelstudio/trunk/eweasel eweasel

Setting up an EiffelStudio delivery for eweasel

To perform testing, you need a modified version of the EiffelStudio delivery, refer to those instructions for building one. Note that you need as many deliveries as platforms you are testing (thankfully the deliveries can be all done in the same location since we distinguish them through the spec/$ISE_PLATFORM/ paradigm).

Setting up your environment

You need to define the following environment variable:

  • ISE_EIFFEL should point to your EiffelStudio delivery (made above)
  • ISE_LIBRARY should point to the same value as $ISE_EIFFEL
  • ISE_PLATFORM should be your platform name (e.g. windows, win64, linux-x86, solaris-sparc, etc...). However for .NET, it has to be dotnet.
  • EWEASEL should be Test/eweasel (replace Test by the absolute path to the Test directory)
  • EWEASEL_OUTPUT should be set to a directory location used by the test engine to perform the compilations and keeping any outputs produced by the tests. If not set it will be in $EWEASEL/tmp.

And you also need to update your PATH environment variable so it reads:

  • On Unix: $EWEASEL/spec/$ISE_PLATFORM/bin
  • On Windows: $EWEASEL/spec/%ISE_PLATFORM%/bin

We recommend that you do the above in a script that you will launch each time you want to perform the testing. That way those settings won't interfere with the official delivery of EiffelStudio.

Windows only

Outdated: This article is to be considered outdated when using the latest version of EiffelStudio or it's tools. The steps described in this section were required in the past and are not needed anymore. It is kept for historical reasons for those who would like to run eweasel on old version of the repository.

You need to update your C compiler flags so that it always generates a DOS application (on Windows default is a GUI application) because eweasel needs to redirect the output of the applications to a file. It depends on the C compiler:

  • For Microsoft, edit the file $ISE_EIFFEL/studio/config/$ISE_PLATFORM/msc/config.eif and modify it so that it specifies -SUBSYSTEM:CONSOLE instead of -SUBSYSTEM:WINDOWS.
  • For MinGW, edit the file $ISE_EIFFEL/studio/config/$ISE_PLATFORM/mingw/config.eif and modify it so that is specifies -mconsole instead of -mwindows.
  • For Borland, edit the file $ISE_EIFFEL/studio/config/$ISE_PLATFORM/bcb/config.eif and modify it so that it specifies -ap instead of -aa.

You may also want to read the Dr Watson page as otherwise on non-working tests, the eweasel regression will stop until you click OK on the debugging dialog.

Compiling eweasel

You need to compile the eweasel engine to run the tests. To do so, go in $EWEASEL/compilation and execute the install_eweasel script (.bat for windows).

Launching the test

In $EWEASEL/bin you will find scripts call run_eweasel, run_eweasel_catalog and run_eweasel_filter (add the .bat extension for Windows) which helps you launch tests.

  • run_eweasel: copy the right `general.cfg' file to your delivery, checkout the latest version of the libraries from the EiffelStudio delivery, precompiles the precompiled libraries needed by eweasel, launch eweasel and store the results in a file.
  • run_eweasel_catalog: run the tests from a specified catalog
  • run_eweasel_filter: run one particular test
  • run_eweasel_keyword: run tests matching a specified keyword

On a recent machine (from 2006), running all the tests should take about 5h or less.

Analyzing the results

You should use the Eweasel Comparator tool and compare your results with one of the reference one from the results subdirectory of eweasel.