MATLAB - generate confusion matrix from classifier

Question

I have some test data and labels:

testZ = [0.25, 0.29, 0.62, 0.27, 0.82, 1.18, 0.93, 0.54, 0.78, 0.31, 1.11, 1.08, 1.02];

testY = [1 1 1 1 1 2 2 2 2 2 2 2 2];

I then sort them:

[sZ, ind] = sort(testZ); %%Sorts Z, and gets indexes of Z
sY = testY(ind); %%Sorts Y by index
[N, n] = size(testZ');

This will then give the sorted Y data. At each element of the sorted Y data, I want to classify each point to the left as being of type 1 and everything to the right being class 2; This will then be repeated for every point of the data. How can I do this and find out for each element the variables:

• TP(true positive) - the elements correctly marked as 1
• FP(false positive) - the elements incorrectly marked as 1
• TN(true negative) - the elements correctly marked as 2
• FN(false negatives) - the elements incorrectly marked as 2

The purpose of this is so that I can create an ROC curve for the classifier as part of some school work.

Answer 1

Here is the code for plotting ROC and finding AUC value:

tot_op = testZ;
targets = testY;
th_vals= sort(tot_op);

for i = 1:length(th_vals)
  b_pred = (tot_op>=th_vals(i,1));
  TP = sum(b_pred == 1 & targets == 2);
  FP = sum(b_pred == 1 & targets == 1);
  TN = sum(b_pred == 0 & targets == 1);
  FN = sum(b_pred == 0 & targets == 2);
  sens(i) = TP/(TP+FN);
  spec(i) = TN/(TN+FP);
end


figure(2);
cspec = 1-spec;
cspec = cspec(end:-1:1);
sens = sens(end:-1:1);
plot(cspec,sens,'k');

AUC = sum(0.5*(sens(2:end)+sens(1:end-1)).*(cspec(2:end) - cspec(1:end-1)));
fprintf('\nAUC: %g \n',AUC);

Above code is the modified version given on http://www.dcs.gla.ac.uk/~srogers/firstcourseml/matlab/chapter5/svmroc.html

Answer 2

You can loop over the points and then use logical indexing and element-wise Boolean operators to get most of what you want.

 for i = 1:length(sY)
      classification = [ones(1,i-1) 2*ones(1,length(sy)-i+1)];
      isTruePositive = ( (sY == classification) & (sY == 1) );
      numberOfTruePositive = sum(isTruePositive);
      % Similar for other cases.

      % Use the result in the loop or store it somewhere - as written here
      % variables are over-written each iteration in the loop.
 end

I didn't run this, so you might need some tweaks, but this should get you most of the way.