How to implement pagination when using amazon Dyna

Question

I want to use amazon Dynamo DB with rails.But I have not found a way to implement pagination.

I will use AWS::Record::HashModel as ORM.

This ORM supports limits like this:

People.limit(10).each {|person| ... } 

But I could not figured out how to implement following MySql query in Dynamo DB.

SELECT * 
  FROM  `People` 
 LIMIT 1 , 30

Answer 1

You issue queries using LIMIT. If the subset returned does not contain the full table, a "LastEvaluatedKey" value is returned. You use this value as the ExclusiveStartKeyin the next query. And so on...

From the DynamoDB Developer Guide.

Answer 2

I faced a similar problem.

The generic pagination approach is, use "start index" or "start page" and the "page length". 

The "ExclusiveStartKey" and "LastEvaluatedKey" based approach is very DynamoDB specific.

I feel this DynamoDB specific implementation of pagination should be hidden from the API client/UI.

Also in case, the application is serverless, using service like Lambda, it will be not be possible to maintain the state on the server. The other side is the client implementation will become very complex.

I came with a different approach, which I think is generic ( and not specific to DynamoDB)

When the API client specifies the start index, fetch all the keys from the table and store it into an array.

Find out the key for the start index from the array, which is specified by the client.

Make use of the ExclusiveStartKey and fetch the number of records, as specified in the page length.

If the start index parameter is not present, the above steps are not needed, we don't need to specify the ExclusiveStartKey in the scan operation.

This solution has some drawbacks -

We will need to fetch all the keys when the user needs pagination with start index.

We will need additional memory to store the Ids and the indexes. Additional database scan operations ( one or multiple to fetch the keys )

But I feel this will be very easy approach for the clients, which are using our APIs. The backward scan will work seamlessly. If the user wants to see "nth" page, this will be possible.

Answer 3

You can provide 'page-size' in you query to set the result set size. The response of DynamoDB contains 'LastEvaluatedKey' which will indicate the last key as per the page size. If response does't contain 'LastEvaluatedKey' it means there are no results left to fetch. Use the 'LastEvaluatedKey' as 'ExclusiveStartKey' while fetching next time.

I hope this helps.

DynamoDB Pagination