Basic API Usage

The test runner consists of an importable module. The test runner is used by providing scripts that import and invoke the run method from the module. The testrunner module is controlled via command-line options. Test scripts supply base and default options by supplying a list of default command-line options that are processed before the user-supplied command-line options are provided.

Typically, a test script does 2 things:

  • Adds the directory containing the zope package to the Python path.

  • Calls the test runner with default arguments and arguments supplied to the script.

    Normally, it just passes default/setup arguments. The test runner uses sys.argv to get the user’s input.

This testrunner_ex subdirectory contains a number of sample packages with tests. Let’s run the tests found here. First though, we’ll set up our default options:

>>> import os.path
>>> directory_with_tests = os.path.join(this_directory, 'testrunner-ex')
>>> defaults = [
...     '--path', directory_with_tests,
...     '--tests-pattern', '^sampletestsf?$',
...     ]

The default options are used by a script to customize the test runner for a particular application. In this case, we use two options:

path

Set the path where the test runner should look for tests. This path is also added to the Python path.

tests-pattern

Tell the test runner how to recognize modules or packages containing tests.

Now, if we run the tests, without any other options:

>>> from zope import testrunner
>>> import sys
>>> sys.argv = ['test']
>>> testrunner.run_internal(defaults)
Running zope.testrunner.layer.UnitTests tests:
  Set up zope.testrunner.layer.UnitTests in N.NNN seconds.
  Ran 156 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Running samplelayers.Layer1 tests:
  Tear down zope.testrunner.layer.UnitTests in N.NNN seconds.
  Set up samplelayers.Layer1 in N.NNN seconds.
  Ran 9 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Running samplelayers.Layer11 tests:
  Set up samplelayers.Layer11 in N.NNN seconds.
  Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Running samplelayers.Layer111 tests:
  Set up samplelayers.Layerx in N.NNN seconds.
  Set up samplelayers.Layer111 in N.NNN seconds.
  Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Running samplelayers.Layer112 tests:
  Tear down samplelayers.Layer111 in N.NNN seconds.
  Set up samplelayers.Layer112 in N.NNN seconds.
  Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Running samplelayers.Layer12 tests:
  Tear down samplelayers.Layer112 in N.NNN seconds.
  Tear down samplelayers.Layerx in N.NNN seconds.
  Tear down samplelayers.Layer11 in N.NNN seconds.
  Set up samplelayers.Layer12 in N.NNN seconds.
  Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Running samplelayers.Layer121 tests:
  Set up samplelayers.Layer121 in N.NNN seconds.
  Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Running samplelayers.Layer122 tests:
  Tear down samplelayers.Layer121 in N.NNN seconds.
  Set up samplelayers.Layer122 in N.NNN seconds.
  Ran 26 tests with 0 failures, 0 errors and 0 skipped in N.NNN seconds.
Tearing down left over layers:
  Tear down samplelayers.Layer122 in N.NNN seconds.
  Tear down samplelayers.Layer12 in N.NNN seconds.
  Tear down samplelayers.Layer1 in N.NNN seconds.
Total: 321 tests, 0 failures, 0 errors and 0 skipped in N.NNN seconds.
False

we see the normal testrunner output, which summarizes the tests run for each layer. For each layer, we see what layers had to be torn down or set up to run the layer and we see the number of tests run, with results.

The test runner returns a boolean indicating whether there were errors. In this example, there were no errors, so it returned False.

(Of course, the times shown in these examples are just examples. Times will vary depending on system speed.)