ROUGE evaluation method gives zero value

问题:

I have set all parameters as discribed in http://kavita-ganesan.com/rouge-howto. But I get zero values of precision recall and f-1. Please, Help me what can i do?

回答1:

If you have set all parameters right and are not getting any error while running rouge then probably you are doing the following mistake while making your summary files in html format.

rouge does not handle whitespaces properly 
thus
<a name="1">[1]</a> <a href="#1" id= 1>
<a name="1">[1]</a> <a href="#1" id=1>
are not the same

In the first case you will not be shown any error but the output will be zero. In second case you get the right output.

Hope this helps..

回答2:

The settings.xml file should look something like this:

<ROUGE_EVAL version="1.5.5">
    <EVAL ID="1">
        <PEER-ROOT>systems</PEER-ROOT>
        <MODEL-ROOT>models</MODEL-ROOT>
        <INPUT-FORMAT TYPE="SPL" />
        <PEERS>
            <P ID="1">peer.txt</P>
        </PEERS>
        <MODELS>
           <M ID="A">modelA.txt</M>
           <M ID="B">modelB.txt</M>
           <M ID="C">modelC.txt</M>
           <M ID="D">modelD.txt</M>
        </MODELS>
    </EVAL>
</ROUGE_EVAL>

Although your input format type might be different, I find that SPL works for .txt and SEE is for HTML.

One thing that tripped me up was: <M ID="A">modelA.txt</M>, I had it as <P ID="A">modelA.txt</P>, ROUGE didn't complain, it just came out with 0 for every value. So keep an eye out for small things like that.