Jedi Testing¶
The test suite depends on tox
and pytest
:
pip install tox pytest
To run the tests for all supported Python versions:
tox
If you want to test only a specific Python version (e.g. Python 2.7), it’s as easy as:
tox -e py27
Tests are also run automatically on Travis CI.
You want to add a test for Jedi? Great! We love that. Normally you should write your tests as Blackbox Tests. Most tests would fit right in there.
For specific API testing we’re using simple unit tests, with a focus on a simple and readable testing structure.
Blackbox Tests (run.py)¶
Jedi is mostly being tested by what I would call “Blackbox Tests”. These
tests are just testing the interface and do input/output testing. This makes a
lot of sense for Jedi. Jedi supports so many different code structures, that
it is just stupid to write 200‘000 unittests in the manner of
regression.py
. Also, it is impossible to do doctests/unittests on most of
the internal data structures. That’s why Jedi uses mostly these kind of
tests.
There are different kind of tests:
- completions / goto_definitions
#?
- goto_assignments:
#!
- usages:
#<
How to run tests?¶
Jedi uses pytest to run unit and integration tests. To run tests,
simply run py.test
. You can also use tox to run tests for
multiple Python versions.
Integration test cases are located in test/completion
directory
and each test cases are indicated by the comment #?
(completions /
definitions), #!
(assignments) and #<
(usages). There is also
support for third party libraries. In a normal test run they are not
being executed, you have to provide a --thirdparty
option.
In addition to standard -k and -m options in py.test, you can use
-T (–test-files) option to specify integration test cases to run.
It takes the format of FILE_NAME[:LINE[,LINE[,...]]]
where
FILE_NAME
is a file in test/completion
and LINE
is a line
number of the test comment. Here is some recipes:
Run tests only in basic.py
and imports.py
:
py.test test/test_integration.py -T basic.py -T imports.py
Run test at line 4, 6, and 8 in basic.py
:
py.test test/test_integration.py -T basic.py:4,6,8
See py.test --help
for more information.
If you want to debug a test, just use the --pdb
option.
Alternate Test Runner¶
If you don’t like the output of py.test
, there’s an alternate test runner
that you can start by running ./run.py
. The above example could be run by:
./run.py basic 4 6 8
The advantage of this runner is simplicity and more customized error reports. Using both runners will help you to have a quicker overview of what’s happening.
Auto-Completion¶
Uses comments to specify a test in the next line. The comment says, which results are expected. The comment always begins with #?. The last row symbolizes the cursor.
For example:
#? ['real']
a = 3; a.rea
Because it follows a.rea
and a is an int
, which has a real
property.
Goto Definitions¶
Definition tests use the same symbols like completion tests. This is possible because the completion tests are defined with a list:
#? int()
ab = 3; ab
Goto Assignments¶
Tests look like this:
abc = 1
#! ['abc=1']
abc
Additionally it is possible to add a number which describes to position of the test (otherwise it’s just end of line):
#! 2 ['abc=1']
abc
Refactoring Tests (refactor.py)¶
Refactoring tests work a little bit similar to Black Box tests. But the idea is here to compare two versions of code. Note: Refactoring is currently not in active development (and was never stable), the tests are therefore not really valuable - just ignore them.