How to use a cross validation test with MATLAB?

Question

I would like to use 10-fold Cross-validation to evaluate a discretization in MATLAB. I should first consider the attributes and the class column.

Answer 1

Let's say you want to perform 10-fold cross-validation for regularized least squares.

% Given X and y, where y = X*beta + noise. 
lambda_range = 0:0.5:10;
cv_MSE = zeros(size(lambda_range));
for i = 1:length(lambda_range)
   regf=@(X,y,Xtest)(Xtest*(inv(X'*X+lambda_range(i)*eye(size(X,2)))*X'*y));
   cv_MSE(i) = crossval('mse',X,y,'Predfun',regf,'kfold',10);
end
[~,idx]= min(cv_MSE);
lambda = lambda_range(idx); 

Answer 2

In Statistics Toolbox there is CROSSVAL function, which performs 10-fold cross validation by default. Check it out.

Another function CROSSVALIND exists in Bioinformatics Toolbox.

Also there is an open source Generic-CV tool: http://www.cs.technion.ac.il/~ronbeg/gcv/

Answer 3

If you would rather write your own xval wrapper rather than using built-in functions, I often use randperm() to generate random orderings of my data, which you can then partition using a 90% (or your favorite value) cutoff point.