How to get more out of Lucene.net

Question

I'm trying to incorporate Lucene.net in my web search.

Currently I have a lucene.net index that contains +1 million documents with 7 fields each. The last field is the "all" field that has the content of the previous fields concatenated. Searching the all field is just EXTREMELY fast :)

But I feel there is more to be found here. How can I make a search that searches one or more space separated strings over all the fields without using the "all" field?
I want to be able to give weights to certain fields. Furthermore it would be really nice if the search contained information on WHERE the hit took place so I can show it in the result.

I think this is all possible, but I don't immideatelly see how.
Any help?

Answer 1

I don't think you need to maintain an "all" field.

1. Have a look into using a "MultiFieldQueryParser". Rather than taking a single default field to be used by the query parser, it accepts an array of field names (in addition to the index analyser).
2. Term boost should work as per "QueryParser" (i.e. no special action required). I should add that I've found the standard scoring seems OK for me (length of field, number of matches etc) without using boosted terms.
3. Lucene.Net (well, certainly the SVN 2.3 builds at the moment) includes a port of the Highlight package from the Java source. It does have a couple of quirks (not least of which is that it can be tricky to get going in the first place), but it basically works.

Good luck

Answer 2

We do something similar, the trick is to specify fields in your query string:

(+Tier1:ribbon^1)^4 OR (+Tier2:ribbon^1)^4 OR (+Tier3:ribbon^1) OR (+Tier4:q*ribbon*^1)^12

In the above example, the user searched for "ribbon" in our application. We have different segments of data in different fields, and the final field "Tier4" contains all the previous terms concatenated together. We prepend the field with a "q", so we can do leading wild-cards, also:

(+Tier4:q*ribbon*^1)^12

Lastly, we use boosts with the caret (^). This ends up weighting things differently. It took a while to get boosts right, and I'm still not 100% happy with them, but they do make a big impact.

Answer 3

You have to get Lucene in Action. Although about original (that is Java) Lucene implementation, it contains all the information you need: about boosts, highlighters, qwery parsers, etc.