AWS DynamoDB - Pick a record/item randomly?

Question

Any ideas how to pick an item/record randomly from a DynamoDB table? I don't believe there are any provisions for this in the API.

I thought about maintaining a table of NumericId|MyOtherKey ("NumericIdTable") and then generating a random number between 0 and the total number of records I have, then getting that item from NumericIdTable but it's not going to work in the long-run.

Thoughts/ideas welcome.

Answer 1

The naive way would be 1) use describe table call to get N (the total number of rows) in this table 2) select a random number i between 1 and N 3) scan. stop until you have seen i rows

I am thinking about a better way to do this. I will update when I have a good answer.

Answer 2

One approach I came up with to pick a random item from a DynamoDB Table:

1. Generate a random RangeKey over all possible RangeKeys in your Table
2. Query the Table with this RangeKey and the RangeKeyCondition GreaterThan and a Limit of 1

For example if you use a UUID as Identifier for your RangeKey you could get your random Item like the following

RandomRangeKey = new UUID
RandomItem = Query( "HashKeyValue": "KeyOfRandomItems",
                    "RangeKeyCondition": { "AttributeValueList":
                                "RandomRangeKey",
                                "ComparisonOperator":"GT"}, 
                    "Limit": 1 )

This way you get a random Item and only consume 1 read capacity.

There is a chance to miss the first query for a random variable by generating a smaller UUID than the smallest one used in the table. This chance scales down with the table scaling up and you can easily send another request using the SmallerThan Comparison on the same random key, which then ensures a hit for a random item.

---

If your Tabledesign doesn't allow randomizable RangeKeys you could follow your approach and create a separate RandomItem table and store the ID under a randomizable RangeKey. A possible table structure for this would be

*RandomItemTable
   TableName - HashKey
   UUID - Rangekey
   ItemId

Keep in mind, for this approach you need to manage the redundancy between the original table and the randomization table.

Answer 3

If you're using GUID as your Hash Key for the table, you can do something like this:

var client = new AmazonDynamoDBClient();

var lastKeyEvaluated = new Dictionary<string, AttributeValue>() 
{ 
    { "YOUR_HASH_KEY", new AttributeValue(Guid.NewGuid().ToString()) } 
};

var request = new ScanRequest()
{
    TableName = YOUR_TABLE_NAME,
    ExclusiveStartKey = lastKeyEvaluated,
    Limit = 1
};
var response = client.Scan(request);

This will give you a random record every time since it generates a random GUID as the lastKeyEvaluated.