How can I calculate an empirical CDF in R?

Question

I'm reading a sparse table from a file which looks like:

1 0 7 0 0 1 0 0 0 5 0 0 0 0 2 0 0 0 0 1 0 0 0 1
1 0 0 1 0 0 0 3 0 0 0 0 1 0 0 0 1
0 0 0 1 0 0 0 2 0 0 0 0 1 0 0 0 1 0 1 0 0 1
1 0 0 1  0 3 0 0 0 0 1 0 0 0 1
0 0 0 1 0 0 0 2 0 0 0 0 1 0 0 0 1 0 1 0 0 1 1 2 1 0 1 0 1

Note row lengths are different.

Each row represents a single simulation. The value in the i-th column in each row says how many times value i-1 was observed in this simulation. For example, in the first simulation (first row), we got a single result with value '0' (first column), 7 results with value '2' (third column) etc.

I wish to create an average cumulative distribution function (CDF) for all the simulation results, so I could later use it to calculate an empirical p-value for true results.

To do this I can first sum up each column, but I need to take zeros for the undef columns.

How do I read such a table with different row lengths? How do I sum up columns replacing 'undef' values with 0'? And finally, how do I create the CDF? (I can do this manually but I guess there is some package which can do that).

Answer 1

This will read the data in:

dat <- textConnection("1 0 7 0 0 1 0 0 0 5 0 0 0 0 2 0 0 0 0 1 0 0 0 1
1 0 0 1 0 0 0 3 0 0 0 0 1 0 0 0 1
0 0 0 1 0 0 0 2 0 0 0 0 1 0 0 0 1 0 1 0 0 1
1 0 0 1  0 3 0 0 0 0 1 0 0 0 1
0 0 0 1 0 0 0 2 0 0 0 0 1 0 0 0 1 0 1 0 0 1 1 2 1 0 1 0 1")
df <- data.frame(scan(dat, fill = TRUE, what = as.list(rep(1, 29))))
names(df) <- paste("Val", 1:29)
close(dat)

Resulting in:

> head(df)
  Val 1 Val 2 Val 3 Val 4 Val 5 Val 6 Val 7 Val 8 Val 9 Val 10 Val 11 Val 12
1     1     0     7     0     0     1     0     0     0      5      0      0
2     1     0     0     1     0     0     0     3     0      0      0      0
3     0     0     0     1     0     0     0     2     0      0      0      0
4     1     0     0     1     0     3     0     0     0      0      1      0
5     0     0     0     1     0     0     0     2     0      0      0      0
....

If the data are in a file, provide the file name instead of dat. This code presumes that there are a maximum of 29 columns, as per the data you supplied. Alter the 29 to suit the real data.

We get the column sums using

df.csum <- colSums(df, na.rm = TRUE)

the ecdf() function generates the ECDF you wanted,

df.ecdf <- ecdf(df.csum)

and we can plot it using the plot() method:

plot(df.ecdf, verticals = TRUE)

Answer 2

You can use the ecdf() (in base R) or Ecdf() (from the Hmisc package) functions.