Using zope.testrunner


Buildout-based projects

zope.testrunner is often used for projects that use buildout:

develop = .
parts = ... test ...

recipe = zc.recipe.testrunner
eggs = mypackage

The usual buildout process


creates a bin/test script that will run the tests for mypackage.


zc.recipe.testrunner takes care to specify the right --test-path option in the generated script. You can add other options (such as --tests-pattern) too; check zc.recipe.testrunner’s documentation for details.

Virtualenv-based projects

pip install zope.testrunner and you’ll get a zope-testrunner script. Run your tests with

zope-testrunner --test-path=path/to/your/source/tree

Your source code needs to be available for the testrunner to import, so you need to run python install or pip install -e . into the same virtualenv.

Some useful command-line options to get you started


show a progress indicator


increase verbosity


colorize the output

-t test

specify test names (one or more regexes)

-m module

specify test modules (one or more regexes)

-s package

specify test packages (one or more regexes)


show names of tests instead of running them


stop on first error or failure

-D, --pdb

enable post-mortem debugging of test failures

--xml path

generate XML reports to be written at the given path


show all command-line options (there are many more!)

For example

bin/test -pvc -m test_foo -t TestBar

runs all TestBar tests from a module called

Writing tests

zope.testrunner expects to find your tests inside your package directory, in a subpackage or module named tests. Test modules in a test subpackage should be named test*.py.


You can change these assumptions with --tests-pattern and --test-file-pattern test runner options.

Tests themselves should be classes inheriting from unittest.TestCase, and if you wish to use doctests, please tell the test runner where to find them and what options to use for them in by supplying a function named test_suite.


import unittest
import doctest

class TestArithmetic(unittest.TestCase):

    def test_two_plus_two(self):
        self.assertEqual(2 + 2, 4)

def doctest_string_formatting():
    """Test Python string formatting

        >>> print('{} + {}'.format(2, 2))
        2 + 2


def test_suite():
    return unittest.TestSuite([

Test grouping

In addition to per-package and per-module filtering, zope.testrunner has other mechanisms for grouping tests:

  • layers allow you to have shared setup/teardown code to be used by a group of tests, that is executed only once, and not for each test. Layers are orthogonal to the usual package/module structure and are specified by setting the layer attribute on test suites.

  • levels allow you to group slow-running tests and not run them by default. They’re specified by setting the level attribute on test suites to an int.

Other features

zope.testrunner can profile your tests, measure test coverage, check for memory leaks, integrate with subunit, shuffle the test execution order, and run multiple tests in parallel.