Django-taggit prefetch_related

Question

I'm building a basic time logging app right now and I have a todo model that uses django-taggit. My Todo model looks like this:

class Todo(models.Model):
    project = models.ForeignKey(Project)
    description = models.CharField(max_length=300)
    is_done = models.BooleanField(default=False)
    billable = models.BooleanField(default=True)
    date_completed = models.DateTimeField(blank=True, null=True)
    completed_by = models.ForeignKey(User, blank=True, null=True)
    tags = TaggableManager()

    def __unicode__(self):
        return self.description

I'm trying to get a list of unique tags for all the Todos in a project and I have managed to get this to work using a set comprehension, however for every Todo in the project I have to query the database to get the tags. My set comprehension is:

unique_tags = { tag.name.lower() for todo in project.todo_set.all() for tag in todo.tags.all() }

This works just fine, however for every todo in the project it runs a separate query to grab all the tags. I was wondering if there is any way I can do something similar to prefetch_related in order to avoid these duplicate queries:

unique_tags = { tag.name.lower() for todo in project.todo_set.all().prefetch_related('tags') for tag in todo.tags.all() }

Running the previous code gives me the error:

'tags' does not resolve to a item that supports prefetching - this is an invalid parameter to prefetch_related().

I did see that someone asked a very similar question here: Optimize django query to pull foreign key and django-taggit relationship however it doesn't look like it ever got a definite answer. I was hoping someone could help me out. Thanks!

Answer 1

Slightly hackish soution:

ct = ContentType.objects.get_for_model(Todo)
todo_pks = [each.pk for each in project.todo_set.all()]
tagged_items = TaggedItem.objects.filter(content_type=ct, object_id__in=todo_pks)   #only one db query
unique_tags = set([each.tag for each in tagged_items])

Explanation

I say it is hackish because we had to use TaggedItem and ContentType which taggit uses internally.

Taggit doesn't provide any method for your particular use case. The reason is because it is generic. The intention for taggit is that any instance of any model can be tagged. So, it makes use of ContentType and GenericForeignKey for that.

The models used internally in taggit are Tag and TaggedItem. Model Tag only contains the string representation of the tag. TaggedItem is the model which is used to associate these tags with any object. Since the tags should be associatable with any object, TaggedItem uses model ContentType.

The apis provided by taggit like tags.all(), tags.add() etc internally make use of TaggedItem and filters on this model to give you the tags for a particular instance.

Since, your requirement is to get all the tags for a particular list of objects we had to make use of the internal classes used by taggit.

Answer 2

Taggit now supports prefetch_related directly on tag fields (in version 0.11.0 and later, released 2013-11-25).

This feature was introduced in this pull request. In the test case for it, notice that after prefetching tags using .prefetch_related('tags'), there are 0 additional queries for listing the tags.

Answer 3

Use django-tagging and method usage_for_model

 def usage_for_model(self, model, counts=False, min_count=None, filters=None):
    """
    Obtain a list of tags associated with instances of the given
    Model class.

    If ``counts`` is True, a ``count`` attribute will be added to
    each tag, indicating how many times it has been used against
    the Model class in question.

    If ``min_count`` is given, only tags which have a ``count``
    greater than or equal to ``min_count`` will be returned.
    Passing a value for ``min_count`` implies ``counts=True``.

    To limit the tags (and counts, if specified) returned to those
    used by a subset of the Model's instances, pass a dictionary
    of field lookups to be applied to the given Model as the
    ``filters`` argument.
    """

Answer 4

A slightly less hackish answer than akshar's, but only slightly...

You can use prefetch_related as long as you traverse the tagged_item relations yourself, using the clause prefetch_related('tagged_items__tag'). Unfortunately, todo.tags.all() won't take advantage of that prefetch - the 'tags' manager will still end up doing its own query - so you have to step over the tagged_items relation there too. This should do the job:

unique_tags = { tagged_item.tag.name.lower()
    for todo in project.todo_set.all().prefetch_related('tagged_items__tag')
    for tagged_item in todo.tagged_items.all() }