I encountered a problem that elasticsearch could not return the count of unique documents by just using terms aggregation on a nested field.
Here is an example of our model:
{
...,
"location" : [
{"city" : "new york", "state" : "ny"},
{"city" : "woodbury", "state" : "ny"},
...
],
...
}
I want to do aggregation on the state field, but this document will be counted twice in the 'ny' bucket since 'ny' appears twice in the document.
So I'm wondering if where is a way to grab the count of distinct documents.
mapping:
people = {
:properties => {
:location => {
:type => 'nested',
:properties => {
:city => {
:type => 'string',
:index => 'not_analyzed',
},
:state => {
:type => 'string',
:index => 'not_analyzed',
},
}
},
:last_name => {
:type => 'string',
:index => 'not_analyzed'
}
}
}
the query is pretty simple:
curl -XGET 'http://localhost:9200/people/_search?pretty&search_type=count' -d '{
"query" : {
"bool" : {
"must" : [
{"term" : {"last_name" : "smith"}}
]
}
},
"aggs" : {
"location" : {
"nested" : {
"path" : "location"
},
"aggs" : {
"state" : {
"terms" : {"field" : "location.state", "size" : 10}
}
}
}
}
}'
The response:
{
"took" : 104,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1248513,
"max_score" : 0.0,
"hits" : [ ]
},
"aggregations" : {
"location" : {
"doc_count" : 2107012,
"state" : {
"buckets" : [ {
"key" : 6,
"key_as_string" : "6",
"doc_count" : 214754
}, {
"key" : 12,
"key_as_string" : "12",
"doc_count" : 168887
}, {
"key" : 48,
"key_as_string" : "48",
"doc_count" : 101333
} ]
}
}
}
}
The doc_count is much larger than the total in hit. So there must be duplicates.
Thanks!