Lucene not null query?

Question

How can we construct a query to search for particular field to be not null?

field_name:* is not working. I tried field_name:[a* to z*] this works fine for English, but does not cover all languages.

Any alternative suggestions?

Answer 1

Have a look at org.apache.lucene.search.FieldValueQuery:

A Query that matches documents that have a value for a given field as reported by LeafReader#getDocsWithField(String).

please note, that it only works with DocValues, so you would need to change a way you create your document:

document.add(new StringField("field-name", "field-value", Field.Store.YES));
document.add(new SortedDocValuesField("field-name", new BytesRef("field-value")));

here I added two fields - you still need regular StringField to get value. You may choose to use BinaryDocValues#get() for older versions of Lucene, but as I see, it is removed in v7. Not sure, what is a proper way to retrieve a value now - please check this

Answer 2

I have just started to play around with lucene (via logstash elastic search) and find that this seems to work from the kibana UI. I am not sure yet if this is some intelligence in elastic search or kibana, i just know that elastic search borrows from the lucene syntax.

application:unit-test && !exception

will return all results from my unit tests which have not had an exception

application:unit-test && exception

returns those which have a non null exception indexed. so you might try just

field

or

!field

Answer 3

This is currently not supported by Lucene. See this for a discussion.

An alternative option may be to store some pre-defined string (like nullnullnullnull) as the field value if it is null. Then you can use a negative filter to remove these records. (I don't like this much, but can't think of a better option)

Answer 4

Try field:[* TO *] or field:["" TO *]. But it's probably better to use a filter for this though.

Answer 5

I was having the same problem but there's a property you can set on the query parser which lets you have wildcard characters at the start of a search term.

queryParser.setAllowLeadingWildcard(true);

This solved the problem for me

Please see Wildcard at the Beginning of a searchterm -Lucene

Answer 6

I found this to work in some cases field:([0 TO 9] [a TO z])