Efficient way to query all records where a field i

Question

I want to query an indexed field efficiently to retrieve all records where the indexed field is not null (present in the index). The field to be queried contains a Ref<T> to another entity, in case this is relevant.

What I could do is an inequality search, like .filter/.filterKey("fieldname >=", "a"), where a is the smallest ASCII that I want to grab.

But is this efficient? Or can I do an equality search somehow, which returns all records that are present in the index?

--

This is how my data looks like: I want to filter all records where the "overlay" column has a key, and omit those where the field is not set. I would like to use an equality filter if possible, so I don't need a composite index (as I am filtering on other fields at the same time).

--

I can use this to test for != null

.filter("user >", "\uFFFD");

and this to test for == null

.filter("user <", "\uFFFD");

I guess that's not the way it should be done. Is there a way to solve this problem using equality instead of inequality?

Answer 1

If you want to query for entities which have a null value for an indexed field:

ofy().load().type(Thing.class).filter("fieldname !=", null)

However, this is not the same thing as an equality filter. Under the covers, GAE treats != as a pair of filters (> and <) and this brings with it the limitations of inequality filters.

If you need an equality filter on "not null", create a synthetic indexed field in your entity which is populated using an @OnSave method. You can use a partial index on the 'true' value to limit the cost of indexing this extra data (ie, @Index(IfTrue.class))

Answer 2

If your property is String, a more efficient way than "!= null" is to use:

ofy().load().type(Thing.class).filter("fieldname >=", "");

Similarly, you can use > 0 if your property is a positive number, etc.