Pytest Features, That You Need in Your (Testing) Life

Note: This was originally posted at martinheinz.dev

Testing your code is integral part of development and quality tests can save you lots of headache down the road. There are plenty of guides on testing in Python and specifically testing with Pytest, but there are quite a few features that I rarely see mentioned anywhere, but I often need. So, here goes list of tricks and features of Pytest, that I can’t live without (and you won’t be able to as well — soon).

Testing for Exceptions

Let’s start simple. You have a function that throws an exception and you want make sure that it happens under right conditions and includes correct message:

Here we can see simple context manager that Pytest provides for us. It allows us to specify type of exception that should be raised as well as message of said exception. If the exception is not raised in the block, then test fails. You can also inspect more attributes of the exception as the context manager returns ExceptionInfo class that has attributes such as type, value or traceback.

Filtering Warnings

With exceptions out of the way, let’s look at warnings. Sometimes you get bunch of warning messages in your logs from inside of libraries that you use. You can’t fix them and they really just create unnecessary noise, so let’s get rid of them:

Here we show 2 approaches — in first one, we straight up ignore all warnings of specified category by inserting a filter at the front of a filter list. This will cause your program to ignore all warnings of this category until it terminates, that might not always be desirable. With second approach, we use context manager that restores all warnings after exiting its scope. We also specify record=True, so that we can inspect list of issued (ignored) warnings if needs be.

Testing Standard Output and Standard Error Messages

Next, let’s look at following scenario: You have a command line tool that has bunch of functions that print messages to standard output but don’t return anything. So, how do we test that?

To solve this, Pytest provides fixture called capsys, which - well - captures system output. All you need to use it, is to add it as parameter to your test function. Next, after calling function that is being tested, you capture the outputs in form of tuple - (out, err), which you can then use in assert statements.

Patching Objects

Sometimes when testing, you might need to replace objects used in functions under test to provide more predictable dataset or to avoid said function from accessing resources that might be unavailable. mock.patch can help with that:

In this first example we can see that it’s possible to patch functions and then check how many times and with what arguments they were called. These patches can also be stacked both in form of decorator and context manager. Now, for some more powerful uses:

First example in above snippet is pretty straightforward — we replace method of SomeClass and make it return None. In the second, more practical example, we avoid being dependent on remote API/resource by replacing requests.get by mock and making it return object that we supply with suitable data.

There are many more things that mock module can do for you and some of them are pretty wild - including side effects, mocking properties, mocking non-existing attributes and much more. If you run into problem when writing tests, then you should definitely checkout docs for this module, because you might very well find solution there.

Sharing Fixtures with conftest.py

If you write a lot of tests, then at some point you will realize that it would be nice to have all Pytest fixtures in one place, from which you would be able to import them and therefore share across test files. This can be solved with conftest.py.

conftest.py is a file which resides at base of your test directory tree. In this file you can store all test fixtures and these are then automatically discovered by Pytest, so you don't even need to import them.

This is also helpful if you need to share data between multiple tests — just create fixture that returns the test data.

Another useful features is ability to specify scope of a fixture — this is important when you have fixtures that are very expensive to create, for example connections to database ( session scope) and on other end of spectrum are the one that need to be reset after every test case ( function scope). Possible values for fixture scope are: function, class, module, package and session.

Parametrizing Fixtures

We already talked about fixtures in above examples, so let’s go little deeper. What if you want to create a bit more generic fixtures by parametrizing them?

Above is an example of fixture that prepares SQLite testing database for each test. This fixture receives path to the database as parameter. This path is passed to the fixture using the request object, which attribute param is an iterable of all arguments passed to the fixture, in this case just one - the path. Here, this fixture first creates the database file (and could also populate it - omitted for clarity), then yields execution to the test and after test is finished, the fixture deletes the database file.

As for the test itself, we use @pytest.mark.parametrize with 3 arguments - first of them is name of the fixture, second is a list of argument values for the fixture which will become the request.param and finally keyword argument indirect=True, which causes the argument values to appear in request.param.

Last thing we need to do is add fixture as a argument to test itself and we are done.

There are situations, when it’s reasonable to skip some tests, whether it’s because of environment ( Linux vs Windows), internet connection, availability of resources or other. So, how do we do that?

This shows very simple example of how you can skip a valid test based on some condition — in this case based on whether the PostgreSQL server is running on the machine or not. There are many more cool features in Pytest related to skipping or anticipating failures and they are very well documented here, so I won’t go into more detail here as it seems redundant to copy and paste existing content here.

Hopefully these few tips will make your testing experience little more enjoyable and more efficient and therefore will motivate you to write few more tests then before. If you have any questions, feel free to reach out to me, also if you have some suggestions or tricks of your own, please share them here. 🙂