I have fitted a SVM model and created the ROC curve with ROCR package. How can I compute the Area Under the Curve (AUC)?
set.seed(1)
tune.out=tune(svm ,Negative~.-Positive, data=trainSparse, kernel ="radial",ranges=list(cost=c(0.1,1,10,100,1000),gamma=c(0.5,1,2,3,4) ))
summary(tune.out)
best=tune.out$best.model
##prediction on the test set
ypred = predict(best,testSparse, type = "class")
table(testSparse$Negative,ypred)
###Roc curve
yhat.opt = predict(best,testSparse,decision.values = TRUE)
fitted.opt = attributes(yhat.opt)$decision.values
rocplot(fitted.opt,testSparse ["Negative"], main = "Test Data")##
Your example doesn't seem to be complete, so I can't seem to be able to run it and alter it accordingly, but try plugging in something along the lines of:
...
prediction.obj <- prediction(...)
perf <- performance(prediction.obj, measure = "auc")
print("AUC: ", perf@y.values)
You can append it after sandipan's code, which gives you the plot alone.
Refer to the ROCR manual for performance
, page 5: ftp://ftp.auckland.ac.nz/pub/software/CRAN/doc/packages/ROCR.pdf
"auc"
is one of the possible measures performance
can yield.
Start with the prediction
Method from the ROCR
Package.
pred_ROCR <- prediction(df$probabilities, df$target)
to get the ROC in a plot:
roc_ROCR <- performance(pred_ROCR, measure = "tpr", x.measure = "fpr")
plot(roc_ROCR, main = "ROC curve", colorize = T)
abline(a = 0, b = 1)
and get the AUC Value:
auc_ROCR <- performance(pred_ROCR, measure = "auc")
auc_ROCR <- auc_ROCR@y.values[[1]]
Try this:
tune.out=tune(svm ,Negative~.-Positive, data=trainSparse, kernel ="radial",
ranges=list(cost=c(0.1,1,10,100,1000),gamma=c(0.5,1,2,3,4),
probability = TRUE)) # train svm with probability option true
summary(tune.out)
best=tune.out$best.model
yhat.opt = predict(best,testSparse,probability = TRUE)
# Roc curve
library(ROCR)
# choose the probability column carefully, it may be
# probabilities[,1] or probabilities[,2], depending on your factor levels
pred <- prediction(attributes(yhat.opt)$probabilities[,2], testSparse$Negative)
perf <- performance(pred,"tpr","fpr")
plot(perf,colorize=TRUE)
Calculate AUC
# Outcome Flag & Predicted probability
roc_val <-roc(testing.label,gbmPred)
plot(roc_val,col='blue')
auc(roc_val)