Find all available values for a field in lucene .n

Question

If I have a field x, that can contain a value of y, or z etc, is there a way I can query so that I can return only the values that have been indexed?

Example x available settable values = test1, test2, test3, test4

Item 1 : Field x = test1

Item 2 : Field x = test2

Item 3 : Field x = test4

Item 4 : Field x = test1

Performing required query would return a list of: test1, test2, test4

Answer 1

I once used Lucene 2.9.2 and there I used the approach with the FieldCache as described in the book "Lucene in Action" by Manning:

String[] fieldValues = FieldCache.DEFAULT.getStrings(indexReader, fieldname);

The array fieldValues contains all values in the index for field fieldname (Example: ["NY", "NY", "NY", "SF"]), so it is up to you now how to process the array. Usually you create a HashMap<String,Integer> that sums up the occurrences of each possible value, in this case NY=3, SF=1.

Maybe this helps. It is quite slow and memory consuming for very large indexes (1.000.000 documents in index) but it works.

Answer 2

I think a WildcardQuery searching on field 'x' and value of '*' would do the trick.

Answer 3

I've implemented this before as an extension method:

public static class ReaderExtentions
{
    public static IEnumerable<string> UniqueTermsFromField(
                                          this IndexReader reader, string field)
    {
        var termEnum = reader.Terms(new Term(field));

        do
        {
            var currentTerm = termEnum.Term();

            if (currentTerm.Field() != field)
                yield break;

            yield return currentTerm.Text();
        } while (termEnum.Next());
    }
}

You can use it very easily like this:

var allPossibleTermsForField = reader.UniqueTermsFromField("FieldName");

That will return you what you want.

EDIT: I was skipping the first term above, due to some absent-mindedness. I've updated the code accordingly to work properly.

Answer 4

You can use facets to return the first N values of a field if the field is indexed as a string or is indexed using KeywordTokenizer and no filters. This means that the field is not tokenized but just saved as it is.

Just set the following properties on a query:

facet=true
facet.field=fieldname
facet.limit=N //the number of values you want to retrieve

Answer 5

TermEnum te = indexReader.Terms(new Term("fieldx"));
do
{
    Term t = te.Term();
    if (t==null || t.Field() != "fieldx") break;
    Console.WriteLine(t.Text());
} while (te.Next());