Can I read 1 big CSV file in parallel in R? [dupli

问题:

This question already has answers here:

R: Is it possible to parallelize / speed-up the reading in of a 20 million plus row CSV into R? (3 answers)

Closed 5 years ago.

I have a big csv file and it takes ages to read. Can I read this in parallel in R using a package like "parallel" or related? I've tried using mclapply, but it is not working.

回答1:

Based upon the comment by the OP, fread from the data.table package worked. Here's the code:

library(data.table)
dt <- fread("myFile.csv")

In the OP's case, read in time for a 1.2GB file with read.csv it took about 4-5 minutes and just 14 seconds with fread.