How to make sure your asyncio tests always run

 1 import asyncio
 2 import pytest
 3 
 4 
 5 async def my_coroutine() -> float:
 6     await asyncio.sleep(0.1)
 7     return 2.71828182845
 8 
 9 
10 @pytest.mark.asyncio
11 async def test_my_coroutine() -> None:
12     assert 2 < await my_coroutine() < 3

$ pytest -v example1.py
============== test session starts ===============
platform darwin -- Python 3.8.2, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 -- ...venv/bin/python
cachedir: .pytest_cache
rootdir: ...
plugins: asyncio-0.14.0
collected 1 item

example1.py::test_my_coroutine PASSED      [100%]

=============== 1 passed in 0.12s ================

How not to forget

With test files often spanning dozens of test functions, some of them synchronous, it is easy to forget the decorator. What happens then?

 1 import asyncio
 2 
 3 
 4 async def my_coroutine() -> float:
 5     await asyncio.sleep(0.1)
 6     return 2.71828182845
 7 
 8 
 9 async def test_my_coroutine() -> None:
10     assert 2 < await my_coroutine() < 3

Luckily, pytest developers have thought of that, so you get a helpful warning:

$ pytest -v example2.py
============== test session starts ===============
platform darwin -- Python 3.8.2, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 -- ...venv/bin/python
cachedir: .pytest_cache
rootdir: ...
plugins: asyncio-0.14.0
collected 1 item

example2.py::test_my_coroutine SKIPPED     [100%]

================ warnings summary ================
example2.py::test_my_coroutine
...venv/lib/python3.8/site-packages/_pytest/python.py:171: PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
You need to install a suitable plugin for your async framework, for example:
   - pytest-asyncio
   - pytest-trio
   - pytest-tornasync
   - pytest-twisted
   warnings.warn(PytestUnhandledCoroutineWarning(msg.format(nodeid)))

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========= 1 skipped, 1 warning in 0.02s ==========

This is pretty great. It tells you to install a plugin even though you already have it (plugins: asyncio-0.14.0) but it definitely points you in the right direction as it tells you that test_my_coroutine is being skipped.

Earlier versions of pytest did not even have this warning. The test function would not have been run at all, but was still marked as passing! Only an asyncio message RuntimeWarning: coroutine 'test_my_coroutine' was never awaited mixed in the output somewhere would be hinting at this omission. Over time, we discovered quite a few tests in our codebase that we thought were being run – but pytest was in fact quietly ignoring them. In that regard, this warning is a huge improvement.

How to check for @pytest.mark.asyncio automatically

Since linting and static analysis is an integral part of our development workflow we decided early on to add an automatic check that all async def test_... functions are decorated with this mark.

This proved to be trivial with the amazing bellybutton linting engine. Bellybutton allows you to create custom linting rules operating on the Abstract Syntax Tree. We created the following rule (lines 2–4) that finds asynchronous functions starting with test_ and checks whether they have the requisite decorator:

 1 rules:
 2   MissingPytestMarkAsyncio:
 3     description: "Asynchronous test function requires the `@pytest.mark.asyncio` decorator"
 4     expr: //AsyncFunctionDef[starts-with(@name, 'test_')]/decorator_list[not(Attribute[@attr = 'asyncio']/value/Attribute[@attr = 'mark']/value/Name[@id = 'pytest'])]
 5 
 6 default_settings: !settings
 7   included: ["*"]
 8   excluded: []
 9   allow_ignore: yes

Running Bellybutton with this configuration we get

$ bellybutton lint
example2.py:10       MissingPytestMarkAsyncio: Asynchronous test function requires the `@pytest.mark.asyncio` decorator
Linting failed (1 rule, 2 files, 1 violation).

We made this a part of our CI run (through our cq linter runner) and voilà, any commit that forgets the decorator will fail the build even before the tests are run.

Alternative: applying the mark on the module level

Finally, there is a way to avoid adding @pytest.mark.asyncio to every asynchronous test function in your module. Pytest allows you to mark the entire module by setting the global variable pytestmark:

 1 import asyncio
 2 import pytest
 3 
 4 
 5 pytestmark = pytest.mark.asyncio
 6 
 7 
 8 async def my_coroutine() -> float:
 9     await asyncio.sleep(0.1)
10     return 2.71828182845
11 
12 
13 async def test_my_coroutine() -> None:
14     assert 2 < await my_coroutine() < 3

If there were multiple asynchronous test functions this would apply the mark to all of them. In our code we prefer to be more explicit and add marks directly to tests, but this is a viable alternative if you like pytest's magic 😉

P.S. Even more magic: aiohttp

If you use aiohttp and its pytest plugin you will find that asynchronous test functions just work without you having to do anything at all. No decorators, no global marks. That's because the plugin, apart from providing useful fixtures, also hooks into test execution and ensures that coroutine test functions run in an event loop. Personally I prefer the explicit behaviour of pytest-asyncio but it's good to know about this auto-magical alternative too!