Logging requests to django-rest-framework

Question

For debugging purposes, I would like to use Django's logging mechanism to log each and every incoming request when it "arrives" at django-rest-framework's doorstep.

Djagno offers logging of its requests (only "warning" log level and above) in the following manner (from LOGGING section in settings.py):

'django.request': {
        'handlers': ['mail_admins'],
        'level': 'ERROR',
        'propagate': False,
 },

I'm looking to achieve something like this (notice: log level is DEBUG):

'rest_framework.request': {
        'handlers': ['logfile'],
        'level': 'DEBUG',
        'propagate': False,
 },

Is there a way I can do that without embedding a logger in to DRF's source code?
Is there maybe some sort of "Logging Backend" option in DRF I'm not aware of?

Answer 1

Here my current solution to get every REQUEST/RESPONSE in the log. I created a middleware compatible with the old middleware (Django < 1.10) and the new middleware that log every request/response. This solution is the best I found so far.

import logging
from django.utils.deprecation import MiddlewareMixin

_logger = logging.getLogger(__name__)

class LogRestMiddleware(MiddlewareMixin):
    """Middleware to log every request/response.
    Is not triggered when the request/response is managed using the cache
    """

    def _log_request(self, request):
        """Log the request"""
        user = str(getattr(request, 'user', ''))
        method = str(getattr(request, 'method', '')).upper()
        request_path = str(getattr(request, 'path', ''))
        query_params = str(["%s: %s" %(k,v) for k, v in request.GET.items()])
        query_params = query_params if query_params else ''

        _logger.debug("req: (%s) [%s] %s %s", user, method, request_path, query_params)

    def _log_response(self, request, response):
        """Log the response using values from the request"""
        user = str(getattr(request, 'user', ''))
        method = str(getattr(request, 'method', '')).upper()
        status_code = str(getattr(response, 'status_code', ''))
        status_text = str(getattr(response, 'status_text', ''))
        request_path = str(getattr(request, 'path', ''))
        size = str(len(response.content))

        _logger.debug("res: (%s) [%s] %s - %s (%s / %s)", user, method, request_path, status_code, status_text, size)

    def process_response(self, request, response):
        """Method call when the middleware is used in the `MIDDLEWARE_CLASSES` option in the settings. Django < 1.10"""
        self._log_request(request)
        self._log_response(request, response)
        return response

    def __call__(self, request):
        """Method call when the middleware is used in the `MIDDLEWARE` option in the settings (Django >= 1.10)"""
        self._log_request(request)
        response = self.get_response(request)
        self._log_response(request, response)
        return response

Answer 2

I found for me that the best and most flexible way was to add logging via a decorator. I simply add the decorator to each of the functions (post, get) that I want to log the request from, as opposed to it being a part of the overal view class. More control over what gets logged. These decorator take the request object passed in (arg[1]) and then logs parts of the request object to a file.

See https://github.com/slogan621/tscharts/commit/39ed479b04b7077f128774d3a203a86d6f68f03e for what amounts to a template for doing this (commit shows changes to settings.py needed to thread the logging into the file logging scheme I have in place, as well as the decorator and example usage).

Answer 3

Override the APIView.initial() method to add logging yourself.

Dispatch methods

The following methods are called directly by the view's .dispatch() method. These perform any actions that need to occur before or after calling the handler methods such as .get(), .post(), put(), patch() and .delete().

.initial(self, request, *args, **kwargs)
Performs any actions that need to occur before the handler method gets called. This method is used to enforce permissions and throttling, and perform content negotiation.

Answer 4

Here is the code from @Glyn Jackson's Answer:

in middleware/mixin.py

class RequestLogMiddleware(object):

    def initial(self, request, *args, **kwargs):
         super(RequestLogMiddleware, self).initial(request, *args, **kwargs)
         # store/log the request

in the View:

class ViewClass(RequestLogMiddleware, generics.RetrieveAPIView):
     ...

Answer 5

I made a generic RequestLogMiddleware that can be hooked into any Django View using decorator_from_middleware.

request_log/middleware.py

import socket
import time


class RequestLogMiddleware(object):
    def process_request(self, request):
        request.start_time = time.time()

    def process_response(self, request, response):

        if response['content-type'] == 'application/json':
            if getattr(response, 'streaming', False):
                response_body = '<<<Streaming>>>'
            else:
                response_body = response.content
        else:
            response_body = '<<<Not JSON>>>'

        log_data = {
            'user': request.user.pk,

            'remote_address': request.META['REMOTE_ADDR'],
            'server_hostname': socket.gethostname(),

            'request_method': request.method,
            'request_path': request.get_full_path(),
            'request_body': request.body,

            'response_status': response.status_code,
            'response_body': response_body,

            'run_time': time.time() - request.start_time,
        }

        # save log_data in some way

        return response

request_log/mixins.py

from django.utils.decorators import decorator_from_middleware

from .middleware import RequestLogMiddleware


class RequestLogViewMixin(object):
    """
    Adds RequestLogMiddleware to any Django View by overriding as_view.
    """

    @classmethod
    def as_view(cls, *args, **kwargs):
        view = super(RequestLogViewMixin, cls).as_view(*args, **kwargs)
        view = decorator_from_middleware(RequestLogMiddleware)(view)
        return view

my_django_rest_api/views.py

from rest_framework import generics

from ...request_log.mixins import RequestLogViewMixin

class SomeListView(
    RequestLogViewMixin,
    generics.ListAPIView
):
    ...