How to compute & plot Equal Error Rate (EER) from

问题:

I have the following values against FAR/FRR. i want to compute EER rates and then plot in matlab.

FAR              FRR
19.64            20
21.29            18.61
24.92            17.08
19.14            20.28
17.99            21.39
16.83            23.47
15.35            26.39
13.20            29.17
7.92             42.92
3.96             60.56
1.82             84.31
1.65             98.33
26.07            16.39
29.04            13.13
34.49            9.31
40.76            6.81
50.33            5.42
66.83            1.67
82.51            0.28

Is there any matlab function available to do this. can somebody explain this to me. Thanks.

回答1:

Let me try to answer your question

1) For your data EER can be the mean/max/min of [19.64,20]

1.1) The idea of EER is try to measure the system performance against another system (the lower the better) by finding the equal(if not equal then at least nearly equal or have the min distance) between False Alarm Rate (FAR) and False Reject Rate (FRR, or missing rate) .

Refer to your data, [19.64,20] gives min distance, thus it could used as EER, you can take mean/max/min value of these two value, however since it means to compare between systems, thus make sure other system use the same method(mean/max/min) to pick EER value.

The difference among mean/max/min can be ignored if the there are large amount of data. In some speaker verification task, there will be 100k data sample.

2) To understand EER ,better compute it by yourself, here is how:

two things you need to know:

A) The system score for each test case (trial)

B) The true/false for each trial

After you have A and B, then you can create [trial, score,true/false] pairs then sort it by the score value, after that loop through the score, eg from min-> max. At each loop assume threshold is that score and compute the FAR,FRR. After loop through the score find the FAR,FRR with "equal" value.

For the code you can refer to my pyeer.py , in function processDataTable2

https://github.com/StevenLOL/Research_speech_speaker_verification_nist_sre2010/blob/master/SRE2010/sid/pyeer.py

This function is written for the NIST SRE 2010 evaluation.

4) There are other measures similar to EER, such as minDCF which only play with the weights of FAR and FRR. You can refer to "Performance Measure" of http://www.nist.gov/itl/iad/mig/sre10results.cfm

5) You can also refer to this package https://sites.google.com/site/bosaristoolkit/ and DETware_v2.1.tar.gz at http://www.itl.nist.gov/iad/mig/tools/ for computing and plotting EER in Matlab

Plotting in DETWare_v2.1

Pmiss=1:50;Pfa=50:-1:1;
Plot_DET(Pmiss/100.0,Pfa/100.0,'r')

回答2:

FAR(t) and FRR(t) are parameterized by threshold, t. They are cumulative distributions, so they should be monotonic in t. Your data is not shown to be monotonic, so if it is indeed FAR and FRR, then the measurements were not made in order. But for the sake of clarity, we can order:

    FAR     FRR
1   1.65    98.33
2   1.82    84.31
3   3.96    60.56
4   7.92    42.92
5   13.2    29.17
6   15.35   26.39
7   16.83   23.47
8   17.99   21.39
9   19.14   20.28
10  19.64   20
11  21.29   18.61
12  24.92   17.08
13  26.07   16.39
14  29.04   13.13
15  34.49   9.31
16  40.76   6.81
17  50.33   5.42
18  66.83   1.67
19  82.51   0.28

This is for increasing FAR, which assumes a distance score; if you have a similarity score, then FAR would be sorted in decreasing order.

Loop over FAR until it is larger than FRR, which occurs at row 11. Then interpolate the cross over value between rows 10 and 11. This is your equal error rate.