Django REST Framework Return Only Objects Needed F

Question

Background

I'm currently using a ModelViewSet controller with get_queryset defined. The query for get_queryset returns all the objects associated with a given resource. For example a list of all the users in the system. This is great for trivial usage as the full list is always returned, paginated by the default pagination method, and then returned to the requestor as expected.

Reason

As our dataset gets larger and larger it seems unnecessary to query all the objects for a given endpoint, load them into a list, and then paginate that list each page load.

Question

Is there any functionality to only return the objects necessary for the given page the request is for?

For example lets say a request comes in for the first page with a page size of 25. Currently we query all the users in the database, paginate that list, and then return the first 25 objects. Then another request comes in for the second page. We have to query all the users again, paginate the list, and then return the second 25 objects.

Instead on the first request I'd like to only query for the first 25 users (using something like LIMIT and OFFSET), then provide that result along with the total count so the pagination method can provide the proper next, previous, and count properties. Then on the second request I'd like to use the request information to update the LIMIT and OFFSET and return the next 25.

Investigation

I'm aware of the ability to write a custom Pagination class but I was wondering if there was some existing process for achieving this.

Answer 1

Unless I misunderstood the question, it sounds like you are getting confused by queryset = User.objects.all(). This line doesn't query all users from a database because query sets are lazy.

It should already work as you are describing - get only the first page of a list from the database using LIMIT and OFFSET, and count the total amount of records using COUNT().

It case your rest pagination configuration isn't setup properly it's described in details here, should be pretty straight forward. If you pass ?page=1 param to your view it should work as expected.

(To not guess what queries are being executed you can install django debug toolbar and see raw sql.)