Aggregating price data to different time horizon i

Question

Hi I'm looking to roll up minutely data in a data.table to 5 minutely (or 10 minutely) horizon. I know this is easily done via using xts and the to.minutes5 function, but I prefer not to use xts in this instance as the data set is rather large. Is there an easy way to do this in data.table ?

Data example : In this example the period between 21.30 to 21.34 (both inclusive) would have just one row with t = 21.30, open = 0.88703 , high = 0.88799 , low = 0.88702 , close = 0.88798, volume = 43 (note the data from 21.35 itself is ignored).

                      t    open    high     low   close volume
 1: 2010-01-03 21:27:00 0.88685 0.88688 0.88685 0.88688      2
 2: 2010-01-03 21:28:00 0.88688 0.88688 0.88686 0.88688      5
 3: 2010-01-03 21:29:00 0.88688 0.88704 0.88687 0.88703      7
 4: 2010-01-03 21:30:00 0.88703 0.88795 0.88702 0.88795     10
 5: 2010-01-03 21:31:00 0.88795 0.88795 0.88774 0.88778      7
 6: 2010-01-03 21:32:00 0.88778 0.88778 0.88753 0.88760      8
 7: 2010-01-03 21:33:00 0.88760 0.88781 0.88760 0.88775     11
 8: 2010-01-03 21:34:00 0.88775 0.88799 0.88775 0.88798      7
 9: 2010-01-03 21:35:00 0.88798 0.88803 0.88743 0.88782      8
10: 2010-01-03 21:36:00 0.88782 0.88782 0.88770 0.88778      6

Output from dput(head(myData)) as requested by GSee. I want to use the data.table for storing some more derived fields based on this original data. So, even if I did use xts to roll up these price bars, I'll have to put them in a data table somehow, so I'd appreciate any tips around the correct way to hold data.table with xts items.

structure(list(t = structure(c(1241136000, 1241136060, 1241136120, 
1241136180, 1241136240, 1241136300), class = c("POSIXct", "POSIXt"
), tzone = "Europe/London"), open = c(0.89467, 0.89467, 0.89472, 
0.89473, 0.89504, 0.895), high = c(0.89481, 0.89475, 0.89473, 
0.89506, 0.8951, 0.895), low = c(0.89457, 0.89465, 0.89462, 0.89473, 
0.89486, 0.89486), close = c(0.89467, 0.89472, 0.89473, 0.89504, 
0.895, 0.89488), volume = c(96L, 14L, 123L, 49L, 121L, 36L)), .Names = c("t", 
"open", "high", "low", "close", "volume"), class = c("data.table", 
"data.frame"), row.names = c(NA, -6L), .internal.selfref = <pointer: 0x0000000000100788>)

Answer 1

You can use the endpoints function (which is written in C) from xts on POSIXt vectors. endpoints finds the position of the last element of a certain time period. By convention, 1:05 would not be included in the same bar as 1:00. So, the data that you provided dput for (which is different than the printed data above it) will have 2 bars.

Assuming dt is your data.table:

library(data.table)
library(xts)

setkey(dt, t)  # make sure the data.table is sorted by time.
ep <- endpoints(dt$t, "minutes", 5)[-1] # remove the first value, which is 0
dt[ep, grp:=seq_along(ep)]              # create a column to group by
dt[, grp:=na.locf(grp, fromLast=TRUE)]  # fill in NAs

dt[, list(t=last(t), open=open[1], high=max(high), low=min(low), 
          close=last(close), volume=sum(volume)), by=grp]

   grp                   t    open   high     low   close volume
1:   1 2009-05-01 01:04:00 0.89467 0.8951 0.89457 0.89500    403
2:   2 2009-05-01 01:05:00 0.89500 0.8950 0.89486 0.89488     36