How can I mock sqlite3.Cursor

Question

I've been pulling my hair out trying to figure out how to mock the sqlite3.Cursor class specifically the fetchall method.

Consider the following code sample

import sqlite3

from mock import Mock, patch
from nose.tools import assert_false


class Foo:
    def check_name(name):
        conn = sqlite3.connect('temp.db')
        c = conn.cursor()
        c.execute('SELECT * FROM foo where name = ?', name)
        if len(c.fetchall()) > 0:
            return True
        return False


@patch('sqlite3.Cursor.fetchall', Mock(return_value=['John', 'Bob']))
def test_foo():
    foo = Foo()
    assert_false(foo.check_name('Cane'))

Running nosetests results in no fun error

E
======================================================================
ERROR: temp.test_foo
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/koddsson/.virtualenvs/temp/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/koddsson/.virtualenvs/temp/lib/python2.7/site-packages/mock.py", line 1214, in patched
    patching.__exit__(*exc_info)
  File "/home/koddsson/.virtualenvs/temp/lib/python2.7/site-packages/mock.py", line 1379, in __exit__
    setattr(self.target, self.attribute, self.temp_original)
TypeError: can't set attributes of built-in/extension type 'sqlite3.Cursor'

----------------------------------------------------------------------
Ran 1 test in 0.002s

FAILED (errors=1)

Should I not be able to mock the fetchall method or am I doing something horribly wrong?

Answer 1

You can't mock everything and databases are particularly tricky. I often find that the right thing to do (esp. with Sqlite since it's so easy) is to load up a test database with mock data and use that in the tests (i.e. fixtures). After all, what you really need to test is whether your code is accessing and querying the database correctly.

The question you are usually trying to answer in a test like this is "If there is X data in the DB and I execute query Y, does that query return Z like I expect", or at a higher level "If I pass parameter X to my method, does it return value Z (based on getting Y from the db)."

In your example, the real question is "Is SELECT * FROM foo where name = ? the right query in this method?" but you don't answer it if you mock the response.

Answer 2

I would take the approach of patching out sqlite3 imported in your module and then work from there.

Let's assume your module is named what.py.

I would patch out what.sqlite3 and then mock the return value of .connect().cursor().fetchall.

Here is a more complete example:

from mock import patch
from nose.tools import assert_true, assert_false

from what import Foo

def test_existing_name():
    with patch('what.sqlite3') as mocksql:
        mocksql.connect().cursor().fetchall.return_value = ['John', 'Bob']
        foo = Foo()
        assert_true(foo.check_name('John'))

Answer 3

I have found a way to mock sqlite3.Cursor in my tests:

cursor = MagicMock(Cursor)
cursor.fetchall.return_value = [{'column1': 'hello', 'column2': 'world'}]

I am pretty new in python but this is how I do it in Java.