Using zope.testrunner

Installation

Buildout-based projects

zope.testrunner is often used for projects that use buildout:

[buildout]
develop = .
parts = ... test ...

[test]
recipe = zc.recipe.testrunner
eggs = mypackage

The usual buildout process

python bootstrap.py
bin/buildout

creates a bin/test script that will run the tests for mypackage.

Tip

zc.recipe.testrunner takes care to specify the right --test-path option in the generated script. You can add other options (such as --tests-pattern) too; check zc.recipe.testrunner’s documentation for details.

Virtualenv-based projects

pip install zope.testrunner and you’ll get a zope-testrunner script. Run your tests with

zope-testrunner --test-path=path/to/your/source/tree

Your source code needs to be available for the testrunner to import, so you need to run python setup.py install or pip install -e . into the same virtualenv.

Some useful command-line options to get you started

-p	show a progress indicator
-v	increase verbosity
-c	colorize the output
-t test	specify test names (one or more regexes)
-m module	specify test modules (one or more regexes)
-s package	specify test packages (one or more regexes)
--list-tests	show names of tests instead of running them
-x	stop on first error or failure
-D, --pdb	enable post-mortem debugging of test failures
--help	show all command-line options (there are many more!)

For example

bin/test -pvc -m test_foo -t TestBar

runs all TestBar tests from a module called test_foo.py.

Writing tests

zope.testrunner expects to find your tests inside your package directory, in a subpackage or module named tests. Test modules in a test subpackage should be named test*.py.

Tip

You can change these assumptions with --tests-pattern and --test-file-pattern test runner options.

Tests themselves should be classes inheriting from unittest.TestCase, and if you wish to use doctests, please tell the test runner where to find them and what options to use for them in by supplying a function named test_suite.

Example:

import unittest
import doctest

class TestArithmetic(unittest.TestCase):

    def test_two_plus_two(self):
        self.assertEqual(2 + 2, 4)


def doctest_string_formatting():
    """Test Python string formatting

        >>> print('{} + {}'.format(2, 2))
        2 + 2

    """

def test_suite():
    return unittest.TestSuite([
        unittest.defaultTestLoader.loadTestsFromName(__name__),
        doctest.DocTestSuite(),
        doctest.DocFileSuite('../README.txt',
                             optionflags=doctest.ELLIPSIS),
    ])

Test grouping

In addition to per-package and per-module filtering, zope.testrunner has other mechanisms for grouping tests:

• layers allow you to have shared setup/teardown code to be used by a group of tests, that is executed only once, and not for each test. Layers are orthogonal to the usual package/module structure and are specified by setting the layer attribute on test suites.
• levels allow you to group slow-running tests and not run them by default. They’re specified by setting the level attribute on test suites to an int.

Other features

zope.testrunner can profile your tests, measure test coverage, check for memory leaks, integrate with subunit, shuffle the test execution order, and run multiple tests in parallel.