Is there a way to augment django QuerySets with ex

Question

I'm trying to add some extra attributes to the elements of a QuerySet so I can use the extra information in templates, instead of hitting the database multiple times. Let me illustrate by example, assuming we have Books with ForeignKeys to Authors.

>>> books = Book.objects.filter(author__id=1)
>>> for book in books:
...     book.price = 2  # "price" is not defined in the Book model
>>> # Check I can get back the extra information (this works in templates too):
>>> books[0].price
2
>>> # but it's fragile: if I do the following:
>>> reversed = books.reverse()
>>> reversed[0].price
Traceback (most recent call last):
  ...
AttributeError: 'Book' object has no attribute 'price'
>>> # i.e., the extra field isn't preserved when the QuerySet is reversed.

So adding attributes to the elements of a QuerySet works, so long as you don't use things like reverse().

My current workaround is to just use select_related() to fetch the extra information I need from the database again, even though I already have in memory.

Is there a better way to do it?

Answer 1

I would assume calling .reverse on a queryset s what is causing your issues. try this:

books = Book.objects.filter(author__id=1)
books_set = []
for book in books:
    book.price = 2
    books_set.append(book)

reverse = books_set.reverse()

Answer 2

Here is my version of alter_items proposed by Will Hardy.

Instead of a single value it allows different values of a custom attribute for each object: you can pass a mapping of values by object id.

It also automatically wraps the results of all methods that return QuerySet.

import types
from itertools import islice

from django.db.models import Model, QuerySet


class QuerySetWithCustomAttributes(object):
    def __init__(self, queryset, **custom_attrs):
        self.queryset = queryset
        self.custom_attrs = custom_attrs

    def __set_custom_attrs(self, obj):
        if isinstance(obj, Model):
            for key, val in self.custom_attrs.items():
                setattr(obj, key, val[obj.id])
        return obj

    def __iter__(self):
        for obj in self.queryset:
            yield self.__set_custom_attrs(obj)

    def __getitem__(self, ndx):
        if type(ndx) is slice:
            return self.__class__(self.queryset.__getitem__(ndx), **self.custom_attrs)
        else:
            return self.__set_custom_attrs(next(islice(self.queryset, ndx, ndx + 1)))

    def __len__(self):
        return len(self.queryset)

    def __apply(self, method):
        def apply(*args, **kwargs):
            result = method(*args, **kwargs)
            if isinstance(result, QuerySet):
                result = self.__class__(result, **self.custom_attrs)
            elif isinstance(result, Model):
                self.__set_custom_attrs(result)
            return result
        return apply

    def __getattr__(self, name):
        attr = getattr(self.queryset, name)
        if isinstance(attr, types.MethodType):
            return self.__apply(attr)
        else:
            return attr

Answer 3

If you iterate queryset before .reverse, then call the reverse and iterate the resulting queryset again then there will be 2 different SQL queries executed, .reverse() method won't reverse already fetched results, it will re-fetch the (possibly changed) results with an another SQL query. So what you're doing is not only fragile but also inefficient.

In order to avoid the second SQL query you can either reverse the queryset before iterating it or reverse a list with model instances in python using e.g. builtin 'reversed' function (see MattoTodd answer).

Answer 4

The error arises because qs.reverse() give rise to a new QuerySet instance, so you are not reversing the old one.

If you want to have a base QS on which to act you can do the following:

>>> augmented_books = Book.objects.extra(select={'price': 2))
>>> augmented_books[0].price
2
>>> augmented_books_rev = augmented_books.reverse()
>>> augmented_books_rev[0].price
2

Of course the select keyword can be much more complex, in fact it can be almost any any meaningful SQL snippet that could fit the [XXX] in

SELECT ..., [XXX] as price, ... FROM ... WHERE ... (etc)

EDIT

As pointed out in other responses, this solution may be inefficient.

If you are sure to get all the Book objects from the query, then you 'd better make one query, store it into a list and eventually reverse the resulting list.

If, on the other hand, you are getting the "head" and the "queue" of the table, making two queries is better, because you won't query all the "middle" useless objects.

Answer 5

This is an old question, but I'll add my solution because I needed it recently.

Ideally we could use some sort of proxy object for the QuerySet. Our proxy version would then make the changes during iteration.

It will be hard to cover all possible scenarios, QuerySet objects are a little complicated and are used in many different ways. But for the simple case of adding an attribute at the last minute because sending to a template (or generic view), the following might work:

class alter_items(object):

  def __init__(self, queryset, **kwargs):
    self.queryset = queryset
    self.kwargs = kwargs

  # This function is required by generic views, create another proxy
  def _clone(self):
    return alter_items(queryset._clone(), **self.kwargs)

  def __iter__(self):
    for obj in self.queryset:
      for key, val in self.kwargs.items():
        setattr(obj, key, val)
      yield obj

And then use this like so:

query = alter_items(Book.objects.all(), price=2)

Because it is not a true proxy, you may need to make further modifications depending on how it is used, but this is the rough approach. It would be nice if there were an easy way to make a proxy class in Python with new style classes. The external library wrapt might be useful if you want to go with a more complete implementation