lucene encoding problem in zend framework

Question

i use of lucene search indexer .

it work nice for english language, but i use of persian in my site and it can`t index for this language

for example "سلام"

i use of this code for create document:

public function __construct($class, $key, $title,$contents, $summary, $createdBy, $dateCreated)
    {
        $this->addField(Zend_Search_Lucene_Field::Keyword('docRef', "$class:$key"));
        $this->addField(Zend_Search_Lucene_Field::UnIndexed('class', $class));
        $this->addField(Zend_Search_Lucene_Field::UnIndexed('key', $key));
        $this->addField(Zend_Search_Lucene_Field::Keyword('title', $title ,'utf-8'));
        $this->addField(Zend_Search_Lucene_Field::unStored('contents', $contents , 'UTF-8'));
        $this->addField(Zend_Search_Lucene_Field::text('summary', $summary , 'UTF-8'));
        $this->addField(Zend_Search_Lucene_Field::Keyword('dateCreated', $dateCreated));
    }

Answer 1

I was having the same problem reported by @afsane and then I tried the solution provided by @ArneRie. It did solve my problem though after some testing I realized the first line was not needed (at least in my current setup).

So the solution that worked for me was to explicitly set the default analyzer before creating my index:

Zend_Search_Lucene_Analysis_Analyzer::setDefault(
    new Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8_CaseInsensitive());
$index = Zend_Search_Lucene::create('/path/to/my/index');

I did not need to explicitly set the default analyzer before opening the index for querying though.

Answer 2

Add this (best place bootstrap)

    Zend_Search_Lucene_Search_QueryParser::setDefaultEncoding('utf-8');
    Zend_Search_Lucene_Analysis_Analyzer::setDefault(
        new Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8_CaseInsensitive ()
    );