Limit list length in redis

Question

I'm using redis lists and pushing to new items to a list. The problem is I really only need the most recent 10 items in a list.

I'm using lpush to add items to a list and lrange to get the most recent 10.

Is there anyway to drop items after a certain number? I'll end up with lists that may have 1,000's of items and can cause performance issues with latency.

Thank you!

Answer 1

You can use LTRIM intermittently after any LPUSH, no need to call LTRIM after every LPUSH as that would add to overall latency in your app ( though redis is really fast, but you can save lots of LPUSH operations )

Here is a pseudo code to achieve an LTRIM on approximately every 5th LPUSH:

LPUSH mylist 1
random_int = some random number between 1-5
if random_int == 1:  # trim my list with 1/5 chance
   LTRIM mylist 0 10

Though your list may grow to be a few elements more than 10 elements at times, but it will surely get truncated at regular intervals. This approach is good for most practical purposes and saves a lot of LTRIM operations, keeping your pushes fast.

Answer 2

The following code,

• pushes the item to the list,
• keep the size fixed to 10,
• and returns the most recent 10 elements

in a transaction.

MULTI
LPUSH list "item1"
LTRIM list 0 9
LRANGE list 0 9
EXEC

Answer 3

After every lpush, call ltrim to trim the list to 10 elements

See http://redis.io/commands/ltrim

Answer 4

Just an alternative. According to official doc of LPUSH, it returns the length of the list after the push operations. You can set a threshold length like k (in your case k > 10) and call LTRIM when returned length is bigger than k. Sample pseudo code as follows:

len = LPUSH mylist xxx 
if len > k:   
  LTRIM mylist 0 9
LRANGE mylist 0 9

It's more controllable than random method. Greater k triggers less LTRIM but with more memory cost. You can adjust k according to how often you want to call LTRIM since calling extra command is more expensive.