Mocking out objects and methods

Mocking is the process of replacing objects used in your code with ones that make testing easier, but only while the tests are running.

This may mean replacing resources or dependencies, such as database connections or file paths, with ones that are isolated for testing. It may also mean replacing chunks of complex functionality that aren’t the subject of the test with mock objects that allow you to check that the mocked out functionality is being used as expected.

What to mock with

Python has a standard mock implementation in the form of unittest.mock which is also available as a rolling backport so that the latest features and bugfixes can be used in any version of Python.

For convenience, testfixtures provides a facade over both of these in the form of testfixtures.mock. The contents are identical and preference is given to the rolling backport if it is installed. The facade also contains any bugfixes that are critical to the operation of functionality provided by testfixtures.

Note

If using testfixtures.mock, you can install with the testfixtures[mock-backport] extra to ensure a compatible version of the mock backport is used.

Testfixtures also provides specialised mocks for dealing with dates and times and subprocesses.

How to mock

Testfixtures provides Replace, Replacer and the replace() decorator to mock out objects. These work in a similar way to unittest.mock.patch(), and have been around longer. They still provide a little more flexibility than patch(), so use whichever feels best in your codebase.

Methods of replacement

When using the tools provided by Testfixtures, there are three different methods of mocking out functionality that can be used to replace functions, classes or even individual methods on a class. Consider the following module:

tests.sample1

class X:

    def y(self):
        return "original y"

    @classmethod
    def aMethod(cls):
        return cls

    @staticmethod
    def bMethod():
        return 2

We want to mock out the y method of the X class, with, for example, the following function:

def mock_y(self):
     return 'mock y'

The context managers

The most flexible tool for replacement of a single thing, is the Replace context manager:

from testfixtures import Replace
from tests.sample1 import X

def test_function():
    with Replace(X.y, mock_y, container=X):
        print(X().y())

For the duration of the with block, the replacement is used:

>>> test_function()
mock y

For multiple replacements, the Replacer context manager is the best tool:

from testfixtures import Replacer
from tests.sample1 import X
from tests import sample1, sample3
import os

def mock_y(self) -> str:
    sample_env = os.environ.get('SAMPLE_ENV')
    sample_constant = sample3.SOME_CONSTANT
    sample_z = sample1.z()
    return f'mock y, {sample_env=}, {sample_constant=}, {sample_z=}'

def test_function():
    with Replacer() as replace:
        replace.on_class(X.y, mock_y)
        replace.in_module(sample1.z, lambda: 'mock z')
        replace.in_environ('SAMPLE_ENV', 'test')
        replace(sample3.SOME_CONSTANT, 24, name='SOME_CONSTANT', container=sample3)
        print(X().y())

For the duration of the with block, the replacement is used:

>>> test_function()
mock y, sample_env='test', sample_constant=24, sample_z='mock z'

Three other convenience context managers are provided for common replacement patterns:

• To replace or remove environment variables, use replace_in_environ():

  import os
  from testfixtures import replace_in_environ
  
  def test_function():
      with replace_in_environ('SOME_ENV_VAR', 1234):
          print(repr(os.environ['SOME_ENV_VAR']))
  

  For the duration of the with block, the replacement is used:

  >>> test_function()
  '1234'
  

  For more details, see Replacing environment variables.

• To replace methods on classes, including normal methods, class methods and static methods, use replace_on_class():

  from testfixtures import replace_on_class
  from tests.sample1 import X
  
  def test_function():
      instance = X()
      with replace_on_class(X.y, lambda self: 'mock y'):
          print(instance.y())
  

  For the duration of the with block, the replacement is used:

  >>> test_function()
  mock y
  

  For more details, see Replacing methods on classes.

• To replace functions in modules use replace_in_module():

  from testfixtures import replace_in_module
  from tests.sample1 import z as original_z
  
  def test_function():
      with replace_in_module(original_z, lambda: 'replacement z'):
          from tests.sample1 import z
          print(z())
  

  For the duration of the with block, the replacement is used:

  >>> test_function()
  replacement z
  

  For more details, see Replacing items in modules.

The decorator

If you want to replace different things in different test functions, you may find the decorator suits your needs better:

from testfixtures import replace

def mock_y(self):
     return 'mock y'

@replace(X.y, mock_y, container=X)
def test_function():
    print(X().y())

When using the decorator, the replacement is used for the duration of the decorated callable’s execution:

>>> test_function()
mock y

If you need to manipulate or inspect the object that’s used as a replacement, define the mock before decorating:

from testfixtures.mock import Mock, call
from testfixtures import compare, replace

@replace(X.y, Mock(), container=X)
def test_function(mocked_y: Mock) -> None:
    mocked_y.return_value = 'mock y'
    print(X().y())
    compare(mocked_y.mock_calls, expected=[call()])

The above still results in the same output:

>>> test_function()
mock y

Note

This method is not compatible with pytest’s fixture discovery stuff. Instead, put a fixture such as the following in your conftest.py:

from testfixtures import replace_on_class
from testfixtures.mock import Mock
import pytest

@pytest.fixture()
def mocked_y():
    m = Mock()
    with replace(X.y, Mock(), container=X):
        yield m

Manual usage

If you want to replace something for the duration of a doctest or you want to replace something for every test in a TestCase, then you can use the Replacer manually.

The instantiation and replacement are done in the set-up step of the TestCase or equivalent:

>>> from testfixtures import Replacer
>>> replacer = Replacer()
>>> replacer.on_class(X.y, mock_y)

The replacement then stays in place until removed:

>>> X().y()
'mock y'

Then, in the tear-down step of the TestCase or equivalent, the replacement is removed:

>>> replacer.restore()
>>> X().y()
'original y'

The restore() method can also be added as an addCleanup() if that is easier or more compact in your test suite.

Replacing more than one thing

Both the Replacer and the replace() decorator can be used to replace more than one thing at a time. For the former, this is fairly obvious:

from tests.sample1 import X

def test_function():
    with Replacer() as replace:
        replace.on_class(X.y, lambda self: 'mock y')
        replace.on_class(X.aMethod, lambda cls: 'mock method')
        x = X()
        print(x.y(), x.aMethod())

For the decorator, it’s less obvious but still pretty easy:

from testfixtures import replace_on_class
from testfixtures.mock import Mock

@replace(X.y, Mock(name='y', return_value='mock y'), container=X)
@replace(X.aMethod, Mock(name='a', return_value='mock method'), container=X, strict=False)
def test_function(aMethod, y):
    print(aMethod, y)
    x = X()
    print(x.y(), x.aMethod())

You’ll notice that you can still get access to the replacements, even though there are several of them:

>>> test_function()
<Mock name='a' id='...'> <Mock name='y' id='...'>
mock y mock method

Replacing things that may not be there

The following code shows a situation where hpy may or may not be present depending on whether the guppy package is installed or not.

tests.sample2

try:
    from guppy import hpy
    guppy = True
except ImportError:
    guppy = False


def dump(path):
    if guppy:
        hpy().heap().stat.dump(path)

To test the behaviour of the code that uses hpy in both of these cases, regardless of whether or not the guppy package is actually installed, we need to be able to mock out both hpy and the guppy global. This is done by doing non-strict replacement, as shown in the following TestCase:

from tests import sample2 as sample2_module
from tests.sample2 import dump, guppy
from testfixtures import replace
from testfixtures.mock import Mock, call

class Tests(unittest.TestCase):

    @replace(container=sample2_module, target=guppy, name='guppy', replacement=True)
    @replace(container=sample2_module, target='.hpy', replacement=Mock(), strict=False)
    def test_method(self, hpy):

        dump('somepath')

        compare(hpy.mock_calls, expected=[
                 call(),
                 call().heap(),
                 call().heap().stat.dump('somepath')
               ])

    @replace(container=sample2_module, target=guppy, name='guppy', replacement=False)
    @replace(container=sample2_module, target='.hpy', replacement=Mock(), strict=False)
    def test_method_no_heapy(self, hpy):

        dump('somepath')

        compare(hpy.mock_calls, expected=[])

Non-strict replacement using the strict keyword parameter is supported when calling a Replacer or any of its methods.

Replacing items in dictionaries and lists

Replace, Replacer and the replace() decorator can be used to replace items in dictionaries and lists.

If the dictionary is os.environ, then see Replacing environment variables.

For a lists such as this:

>>> sample_list = [1, 2, 3]

An element can be placed as follows:

>>> with Replace(container=sample_list, target='.1', replacement=42):
...     print(sample_list)
[1, 42, 3]

For dictionaries such as this:

>>> sample_dict = {1: 'a', 'z': 'b'}

String keys can be replaced as follows:

>>> with Replace(container=sample_dict, target='.z', replacement='c'):
...     print(sample_dict)
{1: 'a', 'z': 'c'}

For non-string keys, it takes a bit more work:

>>> from operator import getitem
>>> with Replace(
...     container=sample_dict, accessor=getitem, name=1, target='a', replacement='c'
... ):
...     print(sample_dict)
{1: 'c', 'z': 'b'}

For nested data structures such as this:

>>> nested = {'key': [1, 2, 3]}

Nested traversal can be used:

>>> with Replace(container=nested, target='.key.2', replacement=42):
...     print(nested)
{'key': [1, 2, 42]}

If your dictionary or other item-based traversal key contains periods, a different separator can be used:

>>> sample_dict = {'.foo': 'bar'}
>>> with Replace(container=sample_dict, target=':.foo', sep=':', replacement='baz'):
...     print(sample_dict)
{'.foo': 'baz'}

Removing attributes and dictionary items

Replace, Replacer and the replace() decorator can be used to remove attributes from objects and remove items from dictionaries.

For example, suppose you have a data structure like the following:

tests.sample1

some_dict = dict(
    key='value',
    complex_key=[1, 2, 3],
)

If you want to remove the key for the duration of a test, you can do so as follows:

from testfixtures import Replace, not_there
from tests.sample1 import some_dict

def test_function():
    with Replace(container=some_dict, target='.key', replacement=not_there):
        pprint(some_dict)

While the replacement is in effect, key is gone:

>>> test_function()
{'complex_key': [1, 2, 3]}

When it is no longer in effect, key is returned:

>>> pprint(some_dict)
{'complex_key': [1, 2, 3], 'key': 'value'}

If you want the whole some_dict dictionary to be removed for the duration of a test, you would do so as follows:

from testfixtures import Replace, not_there
from tests import sample1 as sample1_module
from tests.sample1 import some_dict

def test_function():
    with Replace(
        container=sample1, target=some_dict, name='some_dict', replacement=not_there
    ):
        print(hasattr(sample1, 'some_dict'))

While the replacement is in effect, some_dict is gone:

>>> test_function()
False

When it is no longer in effect, some_dict is returned:

>>> pprint(sample1.some_dict)
{'complex_key': [1, 2, 3], 'key': 'value'}

Replacing environment variables

To ensure an environment variable is present and set to a particular value, use in_environ():

>>> import os
>>> replace = Replacer()
>>> replace.in_environ('SOME_ENV_VAR', 1234)
>>> print(repr(os.environ['SOME_ENV_VAR']))
'1234'

If you want to make sure an environment variable is unset and not present, use not_there:

>>> replace.in_environ('SOME_ENV_VAR', not_there)
>>> 'SOME_ENV_VAR' in os.environ
False

Replacing methods on classes

To replace methods on classes, including normal methods, class methods and static methods, in a way that is friendly to static analysis, use on_class():

class MyClass:

    def normal_method(self, value: str) -> str:
        return 'original' + value

    @classmethod
    def class_method(cls, value: str) -> str:
        return 'original' + value

    @staticmethod
    def static_method(value: str) -> str:
        return 'original' + value

For normal methods, the replacement will be called with the correct self:

>>> instance = MyClass()
>>> replace = Replacer()
>>> replace.on_class(MyClass.normal_method, lambda self, value: type(self).__name__+value)
>>> print(instance.normal_method(':it'))
MyClass:it

For class methods, the replacement you provide will be wrapped in a classmethod if you have not already done so:

>>> replace.on_class(MyClass.class_method, lambda cls, value: cls.__name__+value)
>>> print(instance.class_method(':it'))
MyClass:it

Likewise, for static methods, the replacement you provide will be wrapped in a staticmethod if you have not already done so:

>>> replace.on_class(MyClass.static_method, lambda value: 'mocked'+value)
>>> print(instance.static_method(':it'))
mocked:it

If you need to replace a class attribute such as FOO in this example:

class MyClass:
    FOO = 1

It can be done like this:

>>> instance = MyClass()
>>> replace = Replacer()
>>> replace(MyClass.FOO, 42, container=MyClass, name='FOO')
42
>>> print(instance.FOO)
42

If you encounter methods that have an incorrect __name__, such as those returned by poorly implemented decorators:

def bad(f):
    def inner(self, x):
        return f(self, x)
    return inner

class SampleClass:

    @bad
    def method(self, x):
        return x*2

They can be replaced by specifying the correct name:

>>> instance = SampleClass()
>>> replace = Replacer()
>>> replace.on_class(SampleClass.method, lambda self, value: value*3, name='method')
>>> print(instance.method(2))
6

Replacing items in modules

To replace functions in modules use in_module():

>>> from tests.sample1 import z as original_z
>>> replace = Replacer()
>>> replace.in_module(original_z, lambda: 'replacement z')
>>> from tests.sample1 import z
>>> z()
'replacement z'

If you need to replace usage in a module other than the one where the function is defined, it can be done as follows

>>> from tests.sample1 import z
>>> from tests import sample3
>>> replace = Replacer()
>>> replace.in_module(z, lambda: 'replacement z', module=sample3)
>>> sample3.z()
'replacement z'

If you need to replace a module global, then you can use Replace as follows:

>>> from tests import sample3
>>> replacer = Replacer()
>>> replacer.replace(
...     sample3.SOME_CONSTANT, 43, container=sample3, name='SOME_CONSTANT'
... )
>>> from tests.sample3 import SOME_CONSTANT
>>> SOME_CONSTANT
43

Gotchas

• Make sure you replace the object where it’s used and not where it’s defined. For example, with the following code from the tests.sample1 package:

  from time import time
  
  
  def str_time():
      return str(time())
  

  You might be tempted to mock things as follows:

  >>> replace = Replacer()
  >>> replace.in_module(time.time, Mock())
  

  But this won’t work:

  >>> from tests.sample1 import str_time
  >>> type(float(str_time()))
  <... 'float'>
  

  You need to replace time() where it’s used, not where it’s defined:

  >>> from tests import sample1 as sample1_module
  >>> replace.in_module(time.time, Mock(), module=sample1_module)
  >>> str_time()
  "<...Mock...>"
  

  A corollary of this is that you need to replace all occurrences of an original to safely be able to test. This can be tricky when an original is imported into many modules that may be used by a particular test.

• You can’t replace whole top level modules, and nor should you want to! The reason being that everything up to the last dot in the replacement target specifies where the replacement will take place, and the part after the last dot is used as the name of the thing to be replaced:

  >>> Replacer().replace('sys', Mock())
  Traceback (most recent call last):
   ...
  ValueError: target must contain at least one dot!