turn off SQL logging while keeping settings.DEBUG?

Question

Django logs SQL operations to an internal buffer (whether logging to file or not) when settings.DEBUG=True. Because I have long-running process that does a lot of DB operations, this causes my development-mode instances of the program to grow in memory consumption very quickly.

I would like to disable the internal SQL logging mechanism while leaving settings.DEBUG turned on for my development: is this possible?

Django version 1.3.0.

Answer 1

When settings.DEBUG is True, Django uses CursorDebugWrapper instead of CursorWrapper. This is what appends the queries to connection.queries and consumes memory. I would monkey-patch the connection wrapper to always use CursorWrapper:

from django.conf import settings
from django.db.backends import BaseDatabaseWrapper
from django.db.backends.util import CursorWrapper

if settings.DEBUG:
    BaseDatabaseWrapper.make_debug_cursor = lambda self, cursor: CursorWrapper(cursor, self)

Disabling logging like others suggest won't fix the problem, because CursorDebugWrapper still stores the queries in connection.queries even if logging is off.

Answer 2

Yes, you can quiet the sql logging down by assigning a 'null handler' to the logger named 'django.db.backends'. I assume you use django's new dict-based logging setup? If so, this snippet ought to make it easy:

    ...
    'handlers': {
        'null': {
            'level': 'DEBUG',
            'class':'logging.NullHandler',
            },
    ...
    'loggers': {
        ... your regular logger 'root' or '' ....
        'django.db.backends': {
            'handlers': ['null'],  # Quiet by default!
            'propagate': False,
            'level':'DEBUG',
            },
    ...

Update: look at Brian's answer, too. I understood "logging" to mean the irritating logging of every sql statement. Brian talks about the internal memory logging of every query (and I guess he's right :-)

Answer 3

This worked for me (at least for Django 1.3.1):

from django.db import connection
connection.use_debug_cursor = False

I've found that variable inspecting Django source code (it is not documented), the relevant lines are found in django/db/backends/__init__.py (BaseDatabaseWrapper class):

def cursor(self):
    if (self.use_debug_cursor or
        (self.use_debug_cursor is None and settings.DEBUG)):
        cursor = self.make_debug_cursor(self._cursor())
    else:
        cursor = util.CursorWrapper(self._cursor(), self)
    return cursor

Answer 4

If still interested in tracing SQL operations for debugging purposes, you can also clean connection.queries list periodically to reclaim memory:

from django.db import connection

for i in range(start, count, size):
    objects = MyModel.objects.order_by('pk').all()[i:i + size]
    ...
    print connection.queries
    connection.queries = []