How to read.table() multiple files into a single t

Question

I have filenames named <InputData>.<TestName>.csv and I'd like to make graphs for each test. The best way I can see to do this is to make one R table for each TestName. Each test produces the same columns of data, so I'd like to pull in all the data for each test into an R datatable with an extra column for the inputdata.

I'd like to do:

read.tables(c("B217.SE.csv", "C10.SE.csv"), sep=",")

produces (for example):

       Filename  col1   col2
1   B217.SE.csv     1      2
2   B217.SE.csv     2      4
3   C10.SE.csv      3      1
4   C10.SE.csv      4      5

What's the right way to do this? Some existing function I don't know about? Writing it out in the R language using a for loop?

Answer 1

I can't test it on your data, but you will want to use an apply type function like this:

data <- do.call("rbind", lapply(c("file1", "file2"), function(fn) 
           data.frame(Filename=fn, read.csv(fn)
))

Or, you can simplify it by using plyr. Here's a rough simulation of how that would work (using data frames instead of files):

> df1 <- data.frame(c1=1:5, c2=rnorm(5))
> df2 <- data.frame(c1=3:7, c2=rnorm(5))

In this case I will use get instead of read.csv:

> data <- ldply(c("df1", "df2"), function(dn) data.frame(Filename=dn, get(dn)))
> data
  Filename c1          c2
1  df1  1 -0.15679732
2  df1  2 -0.19392102
3  df1  3  0.01369413
4  df1  4 -0.73942829
5  df1  5 -1.27522427
6  df2  3 -0.33944114
7  df2  4 -0.12509065
8  df2  5  0.11225053
9  df2  6  0.88460684
10 df2  7 -0.70710520

Edit

Taking Marek's suggestion, you can either overwrite or create your own function:

read.tables <- function(file.names, ...) {
    require(plyr)
    ldply(file.names, function(fn) data.frame(Filename=fn, read.csv(fn, ...)))
}

data <- read.tables(c("filename1.csv", "filename2.csv"))

Answer 2

Try this:

## take files.
files <- list.files(pattern=".csv")
## read data using loop
DF <- NULL
for (f in files) {
   dat <- read.csv(f, header=T, sep="\t", na.strings="", colClasses="character")
   DF <- rbind(DF, dat)
}