One of the challenges when writing unit and integration tests, is to avoid interactions between them.

Sometimes a test relies on being run in order after another test, because the first one sets up some environment the second one reuses. Other times some test randomly failing in mysterious ways, because its execution environment was polluted by another test. Environment variables are also a frequent source of side effects.

The worst I saw was a test passing while the function it tested was obviously wrong… because another test was mocking that function and the mock lingered on. Running the test on its own would get it to fail as expected, but when running the full test suite it would pass. Hours of debugging joy followed.

Those interactions lead to bugs which are often incredibly hard to understand and fix.

One solution is to run tests in random orders, and be on the look out for every seemingly transient failure. While this is certainly a very good thing to do, we can also think about more systematic solutions.

To that end, a year ago I wrote a Pytest plugin to run each test in a Bubblewrap sandbox. (the same technology as used by Flatpak) Each test gets its own new environment, isolated from the others.

I never actually used it in production, and therefore completely forgot about it. Yesterday, a conversation with a friend reminded me of it, and I decided to dust it off and publish it: https://pypi.org/project/pytest-bwrap/

In addition to isolating tests from each other, it also cuts them from the network by default, so you can be sure you're properly mocking all your network accesses.

The last feature is it can make some directories read-only, which is always useful, for example to test the error handling code.

I hope it can be useful to others. Do let me know if it is, or if you find problems with it. Especially if you're willing to help improving it. 🙂