Jedi Testing\uf0c1
The test suite depends on pytest
:
pip install pytest
If you want to test only a specific Python version (e.g. Python 3.8), it is as easy as:
python3.8 -m pytest
Tests are also run automatically on Travis CI.
You want to add a test for Jedi? Great! We love that. Normally you should write your tests as Blackbox Tests. Most tests would fit right in there.
For specific API testing we’re using simple unit tests, with a focus on a simple and readable testing structure.
Integration Tests (run.py)\uf0c1
Jedi is mostly being tested by what I would call “integration tests”. These tests are testing type inference with the public API. This makes a lot of sense for Jedi. Also, it is hard to write doctests/unittests for the internal data structures.
There are different kinds of tests:
- completions / inference
#?
- goto:
#!
- references:
#<
How to run tests?\uf0c1
Jedi uses pytest to run unit and integration tests. To run tests,
simply run pytest
.
Most integration test cases are located in the test/completion
directory
and each test case starts with one of these comments:
#?
(completions / inference)#!
(goto)#<
(references)
There is also support for third party libraries. In a normal test run they are
not being executed, you have to provide a --thirdparty
option.
In addition to pytest’s -k
and -m
options, you can use the
-T
(--test-files`) option to specify which test cases should run.
It takes the format of ``FILE_NAME[:LINE[,LINE[,...]]]
where
FILE_NAME
is a file in test/completion
and LINE
is a line
number of the test comment. Here are some examples:
Run tests only in completion/basic.py
and completion/imports.py
:
pytest test/test_integration.py -T basic.py -T imports.py
Run test at line 4, 6, and 8 in completion/basic.py
:
pytest test/test_integration.py -T basic.py:4,6,8
See pytest --help
for more information.
If you want to debug a test, just use the --pdb
option.
Alternate Test Runner\uf0c1
If you don’t like the output of pytest
, there’s an alternate test runner
that you can start by running ./run.py
. The above example could be run by:
./run.py basic 4 6 8 50-80
The advantage of this runner is simplicity and more customized error reports.
Auto-Completion Tests\uf0c1
Uses a comment to specify a test on the next line. The comment defines the expected completions. The comment always begins with #?. The last row symbolizes the cursor. For example:
#? ['upper']
a = 'foo'; a.upp
Inference Tests\uf0c1
Inference tests look very simliar. The difference is that inference tests don’t use brackets:
#? int()
ab = 3; ab
Goto Tests\uf0c1
Goto Tests look like this:
abc = 1
#! ['abc=1']
abc
Additionally it is possible to specify the column by adding a number, which describes the position of the test (otherwise it’s just the end of line):
#! 2 ['abc=1']
abc
Refactoring Tests (refactor.py)\uf0c1
Refactoring tests work a little bit similar to integration tests. But the idea
is here to compare two versions of code. If you want to add a new test case,
just look at the existing ones in the test/refactor
folder and copy them.