tidyr provides the unnest function that help expanding list columns.
This is similar to the much (20x) faster ungroup function in kdb.
I am looking for a similar (but much faster) function that, assuming a data.table that contains several list columns, each with the same number of element on each row, would expand the data.table.
This an extension of this post.
library(data.table)
library(tidyr)
t = Sys.time()
DT = data.table(a=c(1,2,3),
b=c('q','w','e'),
c=list(rep(t,2),rep(t+1,3),rep(t,0)),
d=list(rep(1,2),rep(20,3),rep(1,0)))
print(DT)
a b c d
1: 1 q 2016-01-09 09:55:14,2016-01-09 09:55:14 1,1
2: 2 w 2016-01-09 09:55:15,2016-01-09 09:55:15,2016-01-09 09:55:15 20,20,20
3: 3 e
print(unnest(DT))
Source: local data frame [5 x 4]
a b c d
(dbl) (chr) (time) (dbl)
1 1 q 2016-01-09 09:55:14 1
2 1 q 2016-01-09 09:55:14 1
3 2 w 2016-01-09 09:55:15 20
4 2 w 2016-01-09 09:55:15 20
5 2 w 2016-01-09 09:55:15 20
Here is my own attempt... that seems to be 2x quicker but should be largely improved...
dtUngroup <- function(DT){
colClasses <- lapply(DT,FUN=class)
listCols <- colnames(DT)[colClasses=='list']
if(length(listCols)>0){
nonListCols <- setdiff(colnames(DT),listCols)
nbListElem <- unlist(DT[,lapply(.SD,FUN=lengths),.SDcols=(listCols[1L])])
DT1 <- DT[,lapply(.SD,FUN=rep,times=(nbListElem)),.SDcols=(nonListCols)]
DT1[,(listCols):=DT[,lapply(.SD,FUN=function(x) do.call('c',x)),.SDcols=(listCols)]]
return(DT1)
}
return(DT)
}
dtUngroup(DT)[]
a b c d
1: 1 q 2016-01-09 09:55:14 1
2: 1 q 2016-01-09 09:55:14 1
3: 2 w 2016-01-09 09:55:15 20
4: 2 w 2016-01-09 09:55:15 20
5: 2 w 2016-01-09 09:55:15 20