Optimizing for loop in big data frame

问题:

I have a large data frame (6 million rows) with one row for entry times and next one for exit times of the same unit (id). I need to put them together.

Original data looks something like the following (please bear in mind that some "id" may entry and exit twice like in case of id=1):

df <- read.table(header=T, text='id   time
1  "15/12/2014 06:30"
1   "15/12/2014 06:31"
1 "15/12/2014 06:34"
1 "15/12/2014 06:35"
2  "15/12/2014 06:36"
2  "15/12/2014 06:37"
3 "15/12/2014 06:38"
3 "15/12/2014 06:39"')

Output that I need:

id  entry   exit
1   15/12/2014 06:30    15/12/2014 06:31
2   15/12/2014 06:34    15/12/2014 06:35
3   15/12/2014 06:36    15/12/2014 06:37
4   15/12/2014 06:38    15/12/2014 06:39

Right now I tried a for loop which picks the id and entry time from row 1 and the exit time from time from row2, and puts them together:

for (i in 1:nrow(df)){
outputdf[i,1] <- df[i+i-1,1]
outputdf[i,2] <- df[i+i-1,2]
outputdf[i,3] <- df[i+i-1+1,2]
}

The problem is that it is very inefficient (works for 10k subsets but not for my 6million data frame). I need something that takes less than a minute at least. I have 6 million rows in the df. Do you know any alternative faster than this loop to match rows?

回答1:

You could try

  library(data.table)
  dcast.data.table(setDT(df)[ ,c('.id', 'Seq'):= 
        list(c('entry', 'exit'), gl(.N,2, .N))], id+Seq~.id, value.var='time')

  #   id Seq            entry             exit
  #1:  1   1 15/12/2014 06:30 15/12/2014 06:31
  #2:  1   2 15/12/2014 06:34 15/12/2014 06:35
  #3:  2   3 15/12/2014 06:36 15/12/2014 06:37
  #4:  3   4 15/12/2014 06:38 15/12/2014 06:39

data

 df <- structure(list(id = c(1L, 1L, 1L, 1L, 2L, 2L, 3L, 3L), time = 
   structure(1:8, .Label = c("15/12/2014 06:30", 
 "15/12/2014 06:31", "15/12/2014 06:34", "15/12/2014 06:35", "15/12/2014 06:36", 
 "15/12/2014 06:37", "15/12/2014 06:38", "15/12/2014 06:39"), class
   = "factor")),.Names = c("id", "time"), class = "data.frame", row.names
  = c(NA, -8L))

回答2:

Maybe I'm missing something, but how about this??

indx   <- seq(1,nrow(df)-1,2)
result <- with(df,data.frame(seq=seq(indx),id=id[indx],entry=time[indx],exit=time[indx+1]))
result
#   seq id            entry             exit
# 1   1  1 15/12/2014 06:30 15/12/2014 06:31
# 2   2  1 15/12/2014 06:34 15/12/2014 06:35
# 3   3  2 15/12/2014 06:36 15/12/2014 06:37
# 4   4  3 15/12/2014 06:38 15/12/2014 06:39