I want to create analytics using redis - basic counters per object, per hour/day/week/month/year and total
what redis data structure would be effective for this and how can I avoid doing many calls to redis?
would it better to have each model have this sets of keys:
hash - model:<id>:years
=> every year has a counter
hash - model:<id>:<year>
=> every month has a counter
hash - model:<id>:<year>:<month>
=> every day has a counter
hash - model:<id>:<year>:<month>:<day>
=> every hour has a counter
if this scheme is correct, how would I chart this data without doing many calls to redis? I would have to loop on all year in model:<id>:years
and fetch the month, then loop on the month, etc? Or I just grab all fields and their values from all keys as a batch request and then process that in the server?
It's better to use a zset for this instead of an hash. Using timestamp
as score you will be able to retrieve data for specific time range
For a date range you will use model:<id>:<year>:<month>
, for an hour range (using model:<id>:<year>:<month>:<day>
) and so on...
Indeed, if the date range is larger than a month (e.g. from January 1st 2014 to March 20th 2014), you will have to retrieve multiple zset (model:<id>:2014:01
, model:<id>:2014:02
and model:<id>:2014:03
) and merge the results.
If you really want to do a date range inside a single request, you can always store day precision data inside model:<id>:<year>
. And if you want to handle date range over multiple years, you will just need to have a single zset e.g. model:<id>:byDay
.
However, please note that storing historical data will increase memory consumption over time so you should already think about data retention. With Redis you can either use EXPIRE
on zset or do it yourself with crons.