Reporting Results¶
Reporters are in charge of handling the state and outcome of test-runs. They might output machine- or human-readable reports on the console, or display the results in a graphical user interface.
-
get_reporter_by_name
(name, default='auto')[source]¶ Get an
AbstractReporter
by name, falling back on a default.Reporters are registered via setuptools entry points, in the
'attest.reporters'
group. A third-party reporter can thus register itself using this in itssetup.py
:setup( entry_points = { 'attest.reporters': [ 'name = import.path.to:callable' ] } )
Names for the built in reporters:
'fancy'
—FancyReporter
'plain'
—PlainReporter
'xunit'
–XUnitReporter
'quickfix'
—QuickFixReporter
'xml'
—XmlReporter
'auto'
—auto_reporter()
Parameters: - name – One of the above strings.
- default – The fallback reporter if no reporter has the supplied name,
defaulting to
'auto'
.
Raises KeyError: If neither the name or the default is a valid name of a reporter.
Return type: Callable returning an instance of an
AbstractReporter
.Changed in version 0.4: Reporters are registered via setuptools entry points.
-
get_all_reporters
()[source]¶ Iterable yielding the names of all registered reporters.
>>> list(get_all_reporters()) ['xml', 'plain', 'xunit', 'fancy', 'auto', 'quickfix']
New in version 0.4.
Testing from the Command-Line¶
-
auto_reporter
(**opts)[source]¶ Select a reporter based on the target output and installed dependencies.
This is the default reporter.
Parameters: opts – Passed to FancyReporter
if it is used.Return type: FancyReporter
if output is a terminal and the progressbar and pygments packages are installed, otherwise aPlainReporter
.Changed in version 0.5: A test_loader function attribute similar to
AbstractReporter.test_loader()
.
-
class
FancyReporter
(style=None, verbose=False, colorscheme=None)[source]¶ Heavily uses ANSI escape codes for fancy output to 256-color terminals. Progress of running the tests is indicated by a progressbar and failures are shown with syntax highlighted tracebacks.
Parameters: - style – Pygments style for tracebacks.
- verbose – Report on tests regardless of failure.
- colorscheme – If style is light or dark, maps token names to color names.
Styles
Available styles can be listed with
pygmentize -L styles
. The special values'light'
and'dark'
(referring to the terminal’s background) use the 16 system colors rather than assuming a 256-color terminal.Defaults to light or the environment variable
ATTEST_PYGMENTS_STYLE
.Changed in version 0.6: Added the 16-color styles light and dark and the complementary colorscheme option
Testing from the Editor¶
This is known to work with Vim, but can probably be used with other editors as well. The output mimics that used by many compilers for errors and warnings.
-
class
QuickFixReporter
[source]¶ Report failures in a format that’s understood by Vim’s quickfix feature.
Write a Makefile that runs your tests with this reporter and then from Vim you can do
:mak
. If there’s failures, Vim will jump to the first one by opening the offending file and positioning the cursor at the relevant line; you can jump between failures with:cn
and:cp
. For more information try :help quickfix.Example Makefile (remember to indent with tabs not spaces):
test: @python runtests.py -rquickfix
New in version 0.5.
Integrating Testing with Non-Python Tools¶
Writing New Reporters¶
-
class
AbstractReporter
[source]¶ Optional base for reporters, serves as documentation and improves errors for incomplete reporters.
-
begin
(tests)[source]¶ Called when a test run has begun.
Parameters: tests – The list of test functions we will be running.
-
failure
(result)[source]¶ Called when a test fails.
Parameters: result ( TestResult
) – Result data for the failing test.Changed in version 0.4: Parameters changed to result.
-
success
(result)[source]¶ Called when a test succeeds.
Parameters: result ( TestResult
) – Result data for the succeeding test.Changed in version 0.4: Parameters changed to result.
-
classmethod
test_loader
()[source]¶ Creates a basic unittest test loader using this reporter. This can be used to run tests via distribute, for example:
setup( test_loader='attest:FancyReporter.test_loader', test_suite='tests.collection', )
Now,
python setup.py -q test
is equivalent to:from attest import FancyReporter from tests import collection collection.run(FancyReporter)
If you want to run the tests as a normal unittest suite, try
test_suite()
instead:setup( test_suite='tests.collection.test_suite' )
New in version 0.5.
-
-
class
TestResult
(**kwargs)[source]¶ Container for result data from running a test.
New in version 0.4.
-
error
= None¶ The exception instance, if the test failed.
-
exc_info
= None¶ The
exc_info()
of the exception, if the test failed.
-
raw_traceback
¶ Like
traceback.extract_tb()
with uninteresting entries removed.New in version 0.5.
-
stderr
= None¶ A list of lines the test printed on the standard error.
-
stdout
= None¶ A list of lines the test printed on the standard output.
-
test
= None¶ The test callable.
-
test_name
¶ A representative name for the test, similar to its import path.
-
traceback
¶ The traceback for the exception, if the test failed, cleaned up.
-