ROC curve and libsvm

Question

Given a ROC curve drawn with plotroc.m (see here):

1. Theoretical question: How to select the best threshold to be used?
2. Programming qeuestion: How to induce the libsvm classifier to work with the selected (best) threshold?

Answer 1

ROC curve is plot generated by plotting fraction of true positive on y-axis versus fraction of false positive on x-axis. So, co-ordinates of any point (x,y) on ROC curve indicates FPR and TPR value at particular threshold. As shown in figure, we find the point (x,y) on ROC curve which corresponds to the minimum distance of that point from top-left corner (i.e given by(0,1)) of plot. The threshold value corresponding to that point is the required threshold. Sorry, I am not permitted to put any image, so couldn't explain with figure. But, for more details about this click ROC related help

Secondly, In libsvm, svmpredict function returns you probability of data sample belonging to a particular class. So, if that probability(for positive class) is greater than threshold (obtained from ROC plot) then we can classify the sample to positive class. These few lines might be usefull to you:

    [pred_labels,~,p] = svmpredict(target_labels,feature_test,svmStruct,'-b 1');

% where, svmStruct is structure returned by svmtrain function.

    op = p(:,svmStruct.Label==1);  % This gives probability for positive
% class (i.e whose label is 1 )

Now if this variable 'op' is greater than threshold then we can classify the corresponding test sample to positive class. This can be done as

op_labels = op>th; % where 'th' is threshold obtained from ROC