What’s an efficient way to take multiple items out

Question

I have list (it can be a set if that’s the only way). I have a number of clients connected, the list will be being filled up externally one at a time.

I need an efficient way to take N items out of that list, as a batch, into one client. I don’t care which client it ends up on, just that the first N items pushed into the list end up with one client, and the next N may end up on another (or the same).

I’m really not sure how I could achieved this with lists, I can block for pop but only one at a time. I’d really prefer the ordering from lists.

I was thinking I could use sets, have all clients hitting Redis with SCARD. When they detect the count is =>, SPOP for N. That should get me at least one client filled with N, and every client that receives fewer than N returns them back to the set.

The fact I’m repeating the SCARD feels really bad, and passing things back in unreliable.

Is there anything more baked in to support this sort of flow, or a cleaner algorithm (that ideally has a list?)

Answer 1

You can wrap the logic into a Lua script, to make sure the client gets N sequential items.

local key = KEYS[1]
local N = tonumber(ARGV[1])

local size = redis.call("llen", key)

if size < N then return {} end    -- ensure that there're at least N items

local res = {}
for i = 1, N do                   -- get N items in a transaction
    res[i] = redis.call("lpop", key);
end

return res