Creating folds manually for K-fold cross-validatio

Question

I am trying to make a K-fold CV regression model using K=5. I tried using the "boot" package cv.glm function, but my pc ran out of memory because the boot package always computes a LOOCV MSE next to it. So I decided to do it manually, but I ran in to the following problem. I try to divide my dataframe into 5 vectors of equal length containing a sample of 1/5 of the rownumbers of my df, but i get unexplainable lengths from the 3rd fold.

a <- sample((d<-1:1000), size = 100, replace = FALSE)
b <- sample((d<-1:1000), size = 100, replace = FALSE)
c <- sample((d<-1:1000), size = 100, replace = FALSE)
df <- data.frame(a,b,c)
head(df)

# create first fold (correct: n=20)
set.seed(5)
K1row <- sample(x = nrow(df), size = (nrow(df)/5), replace = FALSE, prob = NULL)
str(K1row) # int [1:20] 21 68 90 28 11 67 50 76 88 96 ...

# create second fold (still going strong: n=20)
set.seed(5)
K2row <- sample(x = nrow(df[-K1row,]), size = ((nrow(df[-K1row,]))/4), replace = FALSE, prob = NULL)
str(K2row) # int [1:20] 17 55 72 22 8 53 40 59 69 76 ...

# create third fold (this is where it goes wrong: n=21)
set.seed(5)
K3row <- sample(x = nrow(df[-c(K1row,K2row),]), size = ((nrow(df[-c(K1row,K2row),]))/3), replace = FALSE, prob = NULL)
str(K3row) # int [1:21] 13 44 57 18 7 42 31 47 54 60 ...

# create fourth fold (and it gets worse: n=26)
set.seed(5)
K4row <- sample(x = nrow(df[-c(K1row,K2row,K3row),]), size = ((nrow(df[-c(K1row,K2row,K3row),]))/2), replace = FALSE, prob = NULL)
str(K4row) # int [1:26] 11 35 46 14 6 33 25 37 43 5 ...

The vector length seems to increase from K=3. Can anyone explain to me what I'm doing wrong?! My code (and reasoning) seems logical, but the outcome says otherwise.. My Many thanks in advance!

Answer 1

It's because K1row and K2row have some elements in common. You are effectively sampling with replacement. The method below uses modulo to split up rows evenly.

set.seed(5)
rand <- sample(nrow(df))

K1row <- rand[rand %% 5 + 1 == 1]
K2row <- rand[rand %% 5 + 1 == 2]
K3row <- rand[rand %% 5 + 1 == 3]
K4row <- rand[rand %% 5 + 1 == 4]
K5row <- rand[rand %% 5 + 1 == 5]