All the values of the many to many field : Django

Question

I have two models:

class Author(models.Model);
    name = models.CharField(max_length=255)

class Book(models.Model):
    title = models.CharField(max_length=255)
    authors = models.ManyToManyField(Author, null=True, blank=True)

Now, I want the info of all the books. So, I did:

book_info = Book.objects.all().values('title', 'authors__name')

And, it gives an output like (for 1 book having 2 authors):

[{'title': u'book1', 'authors__name': u'author1'},{'title': u'book1', 'authors__name': u'author2'}]

What I wanted was something like:

[{'title': u'book1', 'authors': [{'name':u'author1'},{'name':u'author2'}]}]

I may have more fields in the author model, so would like to get those fields as well.

Can I do this in a single query?

What can I do to get something like the desired result?

Answer 1

Django 1.4

Great question, use prefetch_related:

In [3]: [{'name': b.name, 'authors': [a.name for a in b.authors.all()]} for b in Book.objects.prefetch_related('authors')]
(0.000) SELECT "test_app_book"."id", "test_app_book"."name" FROM "test_app_book"; args=()
(0.000) SELECT ("test_app_book_authors"."book_id") AS "_prefetch_related_val", "test_app_author"."id", "test_app_author"."name" FROM "test_app_author" INNER JOIN "test_app_book_authors" ON ("test_app_author"."id" = "test_app_book_authors"."author_id") WHERE "test_app_book_authors"."book_id" IN (1, 2); args=(1, 2)
Out[3]: 
[{'authors': [u'a', u'b'], 'name': u'book'},
 {'authors': [u'b'], 'name': u'test'}]

Django 1.3

prefetch_related was introduced in Django 1.4. For Django 1.3, you need django-selectreverse:

In [19]: [{'name': b.name, 'authors': [a.name for a in b.authors_prefetch]} for b in Book.objects.select_reverse({'authors_prefetch': 'authors'})]
(0.000) SELECT "test_app_book"."id", "test_app_book"."name" FROM "test_app_book"; args=()
(0.001) SELECT (test_app_book_authors.book_id) AS "main_id", "test_app_author"."id", "test_app_author"."name" FROM "test_app_author" INNER JOIN "test_app_book_authors" ON ("test_app_author"."id" = "test_app_book_authors"."author_id") WHERE "test_app_book_authors"."book_id" IN (1, 2); args=(1, 2)
Out[19]: 
[{'authors': [u'a', u'b'], 'name': u'book'},
 {'authors': [u'b'], 'name': u'test'}]

Using django-selectreverse:

class Book(models.Model):
    name = models.CharField(max_length=100)
    authors = models.ManyToManyField(Author, null=True, blank=True)

    objects = ReverseManager()

Answer 2

To avoid doing two queries as in the descriptive answer from @jpic, you could just manually merge your results afterwards. It feels a bit hacky to me, but it works.

def merge_values(values):
    grouped_results = itertools.groupby(values, key=lambda value: value['id'])
    merged_values = []
    for k, g in grouped_results:
        groups = list(g)
        merged_value = {}
        for group in groups:
            for key, val in group.iteritems():
                if not merged_value.get(key):
                    merged_value[key] = val
                elif val != merged_value[key]:
                    if isinstance(merged_value[key], list):
                        if val not in merged_value[key]:
                            merged_value[key].append(val)
                    else:
                        old_val = merged_value[key]
                        merged_value[key] = [old_val, val]
        merged_values.append(merged_value)
    return merged_values

book_info = marge_values(Book.objects.all().values('title', 'authors__name'))

The merge_values function is take from this gist