Doctest is a really clever builtin Python module which allows you to embed automated tests in documentation strings. If you're somewhat new to Python and that sounds complicated - it isn't. It is very easy to set up; I will show you how.
Let's say you have a file with a function which adds two numbers
together. Such a function obviously isn't useful, but I'll use it as a
simple example of how we can use doctest to both document and test
the function. So here's our file:
def add_two_numbers(a, b): return a + b
In case you're not familiar with how documentation strings in Python works, I will show you - let's add some documentation to the function.
def add_two_numbers(a, b): """ This function adds its two arguments together, and returns the result. """ return a + b
If we save this file as my_file.py, we can then start an interactive
Python interpreter and import it automatically with the '-i' option
(providing it's in the current directory). Then, we can use Python's
help() function to see the documentation:
$ python -i my_file.py
>>> help(add_two_numbers)
Help on function add_two_numbers in module __main__:
add_two_numbers(a, b)
This function adds its two arguments together, and returns the
result.
Okay, now that we know the basics of documentation strings, let's add
some tests to them. First, we need to launch a Python interpreter and
import my_file.py as before. Then, we can manually test our function
a few times and see that it works:
$ python3 -i my_file.py
>>> add_two_numbers(1, 2)
3
>>> add_two_numbers(2, 3)
5
>>> add_two_numbers(3, 5)
8
>>>
It seems to work as it should. Now, copy and paste your manual tests into the function's documentation string (taking care to get the indentation right), so the function ends up looking like this:
def add_two_numbers(a, b): """ This function adds its two arguments together, and returns the result. >>> add_two_numbers(1, 2) 3 >>> add_two_numbers(2, 3) 5 >>> add_two_numbers(3, 5) 8 >>> """ return a + b
The text you just copy and pasted into the documentation string, is
the automated tests! Those examples you added shows doctest how the
code should work. Now we can run the tests like this:
$ python3 -m doctest my_file.py
$
... except, that's a bit underwhelming, isn't it? It didn't print anything. Well, that's because all the tests passed. Let's add a bug to the function's last line, the return statement:
return a + b + 1
That "+ 1" shouldn't be there - it's a bug. Now let's see what happens
if we run the tests again:
$ python3 -m doctest my_file.py
**********************************************************************
File "/home/enfors/tmp/my_file.py", line 5, in my_file.add_two_numbers
Failed example:
add_two_numbers(1, 2)
Expected:
3
Got:
4
**********************************************************************
File "/home/enfors/tmp/my_file.py", line 7, in my_file.add_two_numbers
Failed example:
add_two_numbers(2, 3)
Expected:
5
Got:
6
**********************************************************************
File "/home/enfors/tmp/my_file.py", line 9, in my_file.add_two_numbers
Failed example:
add_two_numbers(3, 5)
Expected:
8
Got:
9
**********************************************************************
1 items had failures:
3 of 3 in my_file.add_two_numbers
***Test Failed*** 3 failures.
As you can see, all three tests failed. They show you what kind of output they expected for the test to pass, and what they got instead. That helps us locate the bug, and fix it (do that now, so you don't get these errors again).
Apart from testing the return values of functions, you can also test
what they print - anything you can see the output of in the
interactive interpreter, you can test with doctest. Let's add a
second function which adds two string together instead of numbers, and
add some tests to it in the same way:
def add_two_strings(a, b): """ This function adds its two string arguments together, and prints the result rather than returning it. >>> add_two_strings("foo", "bar") foobar >>> """ print(a + b)
If we run doctest on the file now, we see that we currently get no
errors (providing you fix the "+ 1" bug we added to the previous
function above):
$ python3 -m doctest my_file.py
$
But as soon as we add a little bug to our print statement at the end
of add_two_strings(), like so:
print(a + b + "bug")
... then we get an error when we run the tests:
$ python3 -m doctest my_file.py
**********************************************************************
File "/home/enfors/tmp/my_file.py", line 20, in my_file.add_two_strings
Failed example:
add_two_strings("foo", "bar")
Expected:
foobar
Got:
foobarbug
**********************************************************************
1 items had failures:
1 of 1 in my_file.add_two_strings
***Test Failed*** 1 failures.
So, to recap how to use doctest:
Write your function
Test it manually in the interpreter with python3 -i my_file.py
Copy the results of your manual test to the function's docstring
Test it with python3 -m doctest my_file.py
That's all you need to know to start using doctest. As experienced
developers can tell, you can't use doctest for all kinds of
automated tests, but you can do a lot with it. Any function that you
can test manually in the interpreter, you can test automatically with
doctest. And these tests are so easy to add, and they serve as
documentation too, so why not add them?
In another blog post, I show you how
you can use doctest to as a component of a simple "mini continuous
integration" system for projects where you use git for version
control, by adding a git hook that automatically tests the code you
commit and refuses to accept it if the tests do not pass.