How to implement paging in Mongodb?

Question

I need to paginate a collection of articles (order by date - and nothing else). What is the standard way doing something like this in Mongodb?

I am not going to use the skip() method because of performance issues. Neither do I plan to use the $push method. The closest method I have seen is the range query method. But it seems to fail if any of the sorted items are removed.

Answer 1

If you can sort on an index, efficient pagination can be implemented using the "$min" and "$max" query modifiers or a range query. Make sure your index includes a unique property at the end (eg. "_id").

If you can't sort on an index, you could pre-process the full set of results and keep a list of the "_id" values in order. You could then take a range of that list and find a page of results using the "$in" query operator.

Answer 2

Range sorting should work well for you. First request will take first 10 items sorted by date:

db.articles.find({}).sort( { date : -1 } ).limit(10);

After this you will need store somewhere date of last item and use id in next paging request:

db.articles.find({"date": {$lt: storedDateOfLastItem}}).sort( { date : -1 } ).limit(10);

So, i guess it should work well for you. To estimate total count of pages you will need to use count.

But it seems to fail if any of the sorted items are removed.

If you will remove for example article from page #1 it for sure break page #2 because of stored last date will be changed. To avoid this you can estimate count of items that was before current saved date

db.articles.find({"date": {$gt: storedDateOfLastItem}}).sort( { date : -1 } ).count()

If this count was changed (let say 2 articled was removed). You need to updated storedDateOfLastItem

db.articles.find({"date": {$gt: storedDateOfLastItem}}).sort( { date : -1 } ).take(2)

Again taking storedDateOfLastItem from last item of above request and continue make paging.

But my opinion just keep this paging as it is without extra logic, because i suppose that article deletion is rare operation.

From mongodb documentation:

Paging Costs Unfortunately skip can be (very) costly and requires the server to walk from the beginning of the collection, or index, to get to the offset/skip position before it can start returning the page of data (limit). As the page number increases skip will become slower and more cpu intensive, and possibly IO bound, with larger collections.

Range based paging provides better use of indexes but does not allow you to easily jump to a specific page.