Mocking file objects or iterables in python

Question

Which way is proper for mocking and testing code that iters object returned by open(), using mock library?

whitelist_data.py:

WHITELIST_FILE = "testdata.txt"

format_str = lambda s: s.rstrip().lstrip('www.')
whitelist = None

with open(WHITELIST_FILE) as whitelist_data:
    whitelist = set(format_str(line) for line in whitelist_data)

if not whitelist:
    raise RuntimeError("Can't read data from %s file" % WHITELIST_FILE)

def is_whitelisted(substr):
    return 1 if format_str(substr) in whitelist else 0

Here's how I try to test it.

import unittest
import mock 

TEST_DATA = """
domain1.com
domain2.com
domain3.com
"""

class TestCheckerFunctions(unittest.TestCase):

    def test_is_whitelisted_method(self):
        open_mock = mock.MagicMock()
        with mock.patch('__builtin__.open',open_mock):
            manager = open_mock.return_value.__enter__.return_value
            manager.__iter__ = lambda s: iter(TEST_DATA.splitlines())
            from whitelist_data import is_whitelisted
            self.assertTrue(is_whitelisted('domain1.com'))

if __name__ == '__main__':
    unittest.main()

Result of python tests.py is:

$ python tests.py

E
======================================================================
ERROR: test_is_whitelisted_method (__main__.TestCheckerFunctions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests.py", line 39, in test_is_whitelisted_method
    from whitelist_data import is_whitelisted
  File "/Users/supa/Devel/python/whitelist/whitelist_data.py", line 20, in <module>
    whitelist = set(format_str(line) for line in whitelist_data)
TypeError: 'Mock' object is not iterable

----------------------------------------------------------------------
Ran 1 test in 0.001s

UPD: Thanks to Adam, I've reinstalled mock library(pip install -e hg+https://code.google.com/p/mock#egg=mock) and updated tests.py. Works like a charm.

Answer 1

You're looking for a MagicMock. This supports iteration.

In mock 0.80beta4, patch returns a MagicMock. So this simple example works:

import mock

def foo():
    for line in open('myfile'):
        print line

@mock.patch('__builtin__.open')
def test_foo(open_mock):
    foo()
    assert open_mock.called

If you're running mock 0.7.x (It looks like you are), I don't think you can accomplish this with patch alone. You'll need to create the mock separately, then pass it into patch:

import mock

def foo():
    for line in open('myfile'):
        print line

def test_foo():
    open_mock = mock.MagicMock()
    with mock.patch('__builtin__.open', open_mock):
        foo()
        assert open_mock.called

---

Note - I've run these with py.test, however, these same approaches will work with unittest as well.