Patch __call__ of a function

Question

I need to patch current datetime in tests. I am using this solution:

def _utcnow():
    return datetime.datetime.utcnow()


def utcnow():
    """A proxy which can be patched in tests.
    """
    # another level of indirection, because some modules import utcnow
    return _utcnow()

Then in my tests I do something like:

    with mock.patch('***.utils._utcnow', return_value=***):
        ...

But today an idea came to me, that I could make the implementation simpler by patching __call__ of function utcnow instead of having an additional _utcnow.

This does not work for me:

    from ***.utils import utcnow
    with mock.patch.object(utcnow, '__call__', return_value=***):
        ...

How to do this elegantly?

Answer 1

[EDIT]

Maybe the most interesting part of this question is Why I cannot patch somefunction.__call__?

Because the function don't use __call__'s code but __call__ (a method-wrapper object) use function's code.

I don't find any well sourced documentation about that, but I can prove it (Python2.7):

>>> def f():
...     return "f"
... 
>>> def g():
...     return "g"
... 
>>> f
<function f at 0x7f1576381848>
>>> f.__call__
<method-wrapper '__call__' of function object at 0x7f1576381848>
>>> g
<function g at 0x7f15763817d0>
>>> g.__call__
<method-wrapper '__call__' of function object at 0x7f15763817d0>

Replace f's code by g's code:

>>> f.func_code = g.func_code
>>> f()
'g'
>>> f.__call__()
'g'

Of course f and f.__call__ references are not changed:

>>> f
<function f at 0x7f1576381848>
>>> f.__call__
<method-wrapper '__call__' of function object at 0x7f1576381848>

Recover original implementation and copy __call__ references instead:

>>> def f():
...     return "f"
... 
>>> f()
'f'
>>> f.__call__ = g.__call__
>>> f()
'f'
>>> f.__call__()
'g'

This don't have any effect on f function. Note: In Python 3 you should use __code__ instead of func_code.

I Hope that somebody can point me to the documentation that explain this behavior.

You have a way to work around that: in utils you can define

class Utcnow(object):
    def __call__(self):
        return datetime.datetime.utcnow()


utcnow = Utcnow()

And now your patch can work like a charm.

---

Follow the original answer that I consider even the best way to implement your tests.

I've my own gold rule: never patch protected methods. In this case the things are little bit smoother because protected method was introduced just for testing but I cannot see why.

The real problem here is that you cannot to patch datetime.datetime.utcnow directly (is C extension as you wrote in the comment above). What you can do is to patch datetime by wrap the standard behavior and override utcnow function:

>>> with mock.patch("datetime.datetime", mock.Mock(wraps=datetime.datetime, utcnow=mock.Mock(return_value=3))):
...  print(datetime.datetime.utcnow())
... 
3

Ok that is not really clear and neat but you can introduce your own function like

def mock_utcnow(return_value):
    return mock.Mock(wraps=datetime.datetime, 
                     utcnow=mock.Mock(return_value=return_value)):

and now

mock.patch("datetime.datetime", mock_utcnow(***))

do exactly what you need without any other layer and for every kind of import.

Another solution can be import datetime in utils and to patch ***.utils.datetime; that can give you some freedom to change datetime reference implementation without change your tests (in this case take care to change mock_utcnow() wraps argument too).

Answer 2

When you patch __call__ of a function, you are setting the __call__ attribute of that instance. Python actually calls the __call__ method defined on the class.

For example:

>>> class A(object):
...     def __call__(self):
...         print 'a'
...
>>> a = A()
>>> a()
a
>>> def b(): print 'b'
...
>>> b()
b
>>> a.__call__ = b
>>> a()
a
>>> a.__call__ = b.__call__
>>> a()
a

Assigning anything to a.__call__ is pointless.

However:

>>> A.__call__ = b.__call__
>>> a()
b

TLDR;

a() does not call a.__call__. It calls type(a).__call__(a).

Links

There is a good explanation of why that happens in answer to "Why type(x).__enter__(x) instead of x.__enter__() in Python standard contextlib?".

This behaviour is documented in Python documentation on Special method lookup.

Answer 3

As commented on the question, since datetime.datetime is written in C, Mock can't replace attributes on the class (see Mocking datetime.today by Ned Batchelder). Instead you can use freezegun.

$ pip install freezegun

Here's an example:

import datetime

from freezegun import freeze_time

def my_now():
    return datetime.datetime.utcnow()


@freeze_time('2000-01-01 12:00:01')
def test_freezegun():
    assert my_now() == datetime.datetime(2000, 1, 1, 12, 00, 1)

As you mention, an alternative is to track each module importing datetime and patch them. This is in essence what freezegun does. It takes an object mocking datetime, iterates through sys.modules to find where datetime has been imported and replaces every instance. I guess it's arguable whether you can do this elegantly in one function.