<type>documentation</type>
<domain>testing</domain>
<id>running</id>
<title>Running and Debugging Tests</title>
<date>$Date: 2006-05-24 22:53:07 -0400 (Wed, 24 May 2006) $</date>
<version>$Revision: 13098 $</version>
<product>pro+</product>

Tests can be run and debugged from the ``Testing`` menu, in the following ways:

* ``Run All Tests`` runs all the unit tests listed in the ``Testing`` tool.

* ``Run Tests in Current File`` runs all the tests found in the current editor.

* ``Run Tests at Cursor`` runs the test or tests at the caret or selection 
  in the current editor

* ``Run Failed Tests`` reruns all the tests marked as failed in the 
  ``Testing`` tool.

* ``Run Stale Tests`` reruns all the tests marked as stale in the ``Testing`` tool,
  based on edits made since the tests were last run.

* ``Run Tests Again`` reruns all the tests that were run the last time 
  tests were run.

Test files or individual tests may be selected in the ``Testing`` tool and run with
the ``Run Tests`` button or using the items in the right-click context menu.

Tests are run in the order they are shown in the ``Testing`` tool.

To stop running tests, press ``Abort Tests`` in the ``Testing`` tool or select ``Abort
Running Tests`` from the ``Testing`` menu.

To clear the previous test results from the ``Testing`` tool, use ``Clear Results`` in the
right-click context menu.

Debugging
---------

For each of the run options, there is an equivalent debug option that will run the tests
in the debugger.  These are in the ``Debug`` group of the ``Testing`` menu.

When tests are debugged, output goes to the ``Debug I/O`` tool and the contents of the
``Testing`` tool are not updated with the results of the test.

**Unexpected Exceptions**

Some testing frameworks such as ``pytest`` may stop at internal exceptions that should be
ignored by clicking on ``Ignore this exception location`` in the ``Exceptions`` tool. This
occurs when the testing framework raises and then handles ``AssertionError`` in order to
probe the capabilities of the running Python. By default, Wing will always stop on
assertions, even if they are handled, because in most cases a failing assertion indicates
a bug in code. Once ignored, Wing won't stop on these internal exceptions again and
debugging can proceed as usual.

Specifying Environment and Command Line Arguments
-------------------------------------------------

The Python environment used to run unit tests, and also any command line arguments to pass
to the tests, can be set with ``Environment`` under the ``Testing`` tab of ``Project
Properties``. 

To select different environments for different test files, set ``Environment`` instead in
``File Properties`` for each file. ``File Propertiess` are accessed by right-clicking on
the test file in the editor or in the ``Testing`` or ``Project`` tools.

Execution Options
-----------------

There are several options available for how Wing runs unit tests.

**Process Model**

When multiple test files are run at once, they may be run in a separate process for each
file (the default), or all test files in one directory may be run in a single process.
This is selected with `Process Model
<script:view_project_properties#highlighted_attrib="testing.process-model">`__ under the
``Testing`` tab of `Project Properties <ref:proj/project-properties>`__.

In the ``Per-Module`` model, Wing is running the equivalent of the following
command line::

  cd /path/to/files 
  python -m unittest one.py 
  python -m unittest two.py 
  
In the ``Per-Package model``, Wing is instead running the equivalent of::

  cd /path/to/files 
  python -m unittest one.py two.py
  
In both cases all tests should be run, but two processes are used in the first case and 
only one in the second case.  Which model you choose depends on the requirements of your
test suite.

**Running Tests Concurrently**

Two or more test processes may be run in parallel by increasing the `Number of
Processes
<script:view_project_properties#highlighted_attrib="testing-max-running-processes">`__
under the ``Testing`` tab of `Project Properties <ref:proj/project-properties>`__.
This can increase performance on systems with multiple CPU cores, but may introduce
problems if the tests do not handle concurrency well.

**Running Test Packages**

When test files that are located in a package (a directory that contains
``__init__.py``), they may be loaded either as package modules, or as top-level modules.
Each testing framework defines a default behavior for this case, but this can be
overridden using `Run as Package Modules
<script:view_project_properties#highlighted_attrib="testing.run-as-package-module">`__
under the ``Testing`` tab of `Project Properties <ref:proj/project-properties>`__.

When files are loaded individually as package modules, Wing is running the equivalent of::

  python -m unittest package.module
  
When files are loaded as a top-level package, Wing is running the equivalent of::

  python -m unittest module

