Using zope.testrunner¶
Installation¶
Buildout-based projects¶
zope.testrunner is often used for projects that use buildout:
[buildout]
develop = .
parts = ... test ...
[test]
recipe = zc.recipe.testrunner
eggs = mypackage
The usual buildout process
python bootstrap.py
bin/buildout
creates a bin/test
script that will run the tests for mypackage.
Tip
zc.recipe.testrunner takes care to specify the right
--test-path
option in the generated script. You can add
other options (such as --tests-pattern
) too; check
zc.recipe.testrunner’s documentation for details.
Virtualenv-based projects¶
pip install zope.testrunner
and you’ll get a zope-testrunner
script. Run your tests with
zope-testrunner --test-path=path/to/your/source/tree
Your source code needs to be available for the testrunner to import,
so you need to run python setup.py install
or pip install -e
.
into the same virtualenv.
Some useful command-line options to get you started¶
- -p
show a progress indicator
- -v
increase verbosity
- -c
colorize the output
- -t test
specify test names (one or more regexes)
- -m module
specify test modules (one or more regexes)
- -s package
specify test packages (one or more regexes)
- --list-tests
show names of tests instead of running them
- -x
stop on first error or failure
- -D, --pdb
enable post-mortem debugging of test failures
- --xml path
generate XML reports to be written at the given path
- --help
show all command-line options (there are many more!)
For example
bin/test -pvc -m test_foo -t TestBar
runs all TestBar tests from a module called test_foo.py.
Writing tests¶
zope.testrunner
expects to find your tests inside your package
directory, in a subpackage or module named tests
. Test modules
in a test subpackage should be named test*.py
.
Tip
You can change these assumptions with --tests-pattern
and
--test-file-pattern
test runner options.
Tests themselves should be classes inheriting from
unittest.TestCase
, and if you wish to use doctests, please tell
the test runner where to find them and what options to use for them
in by supplying a function named test_suite
.
Example:
import unittest
import doctest
class TestArithmetic(unittest.TestCase):
def test_two_plus_two(self):
self.assertEqual(2 + 2, 4)
def doctest_string_formatting():
"""Test Python string formatting
>>> print('{} + {}'.format(2, 2))
2 + 2
"""
def test_suite():
return unittest.TestSuite([
unittest.defaultTestLoader.loadTestsFromName(__name__),
doctest.DocTestSuite(),
doctest.DocFileSuite('../README.txt',
optionflags=doctest.ELLIPSIS),
])
Test grouping¶
In addition to per-package and per-module filtering, zope.testrunner has other mechanisms for grouping tests:
layers allow you to have shared setup/teardown code to be used by a group of tests, that is executed only once, and not for each test. Layers are orthogonal to the usual package/module structure and are specified by setting the
layer
attribute on test suites.levels allow you to group slow-running tests and not run them by default. They’re specified by setting the
level
attribute on test suites to an int.
Other features¶
zope.testrunner can profile your tests, measure test coverage, check for memory leaks, integrate with subunit, shuffle the test execution order, and run multiple tests in parallel.