Going to random page at once with ES (Elasticsearc

2019-08-27 17:14发布

问题:

In interface we have list of rows [logs] shown by Yii1 CGridView widget. DataProvider is ES (ElasticSearch). If user just enters index page, he/she sends request to fetch top 30 results [size = 30, from = 0] with descending order by date and id

'sort' => [
    'date' => [
        'order' => 'DESC'
     ],
     'id' => [
        'order' => 'DESC'
     ],
],
'from' => $pageFromPrevious * $this->getSize(),
'size' => $this->getSize()

And it's working. Now if I click last page buttom [CGridView widget allows/shows that button], ES gives me error

Result window is too large, from + size must be less than or equal to: [10000] but was [26160]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting

I understand the error above, and looked for alternatives like search_after and scan-scroll, but they seem not suitable for me. If a user click random page, or edit url [page number] or go to the last page at once? What are the correct ways to solve this problem. Any advice?

== EDITED == P.S: ES v.6.2

== EDITED ==

It's duplicate in case of message "

...from + size must be less than or equal to...

"

But I wanted to find answer (

I understand the error above, and looked for alternatives like search_after and scan-scroll, but they seem not suitable for me

)

to "

How to solve problem, when user chooses to go to last page at once or at middle of thousandth page

"

And Adam T's answer about "google" seems to be correct.

回答1:

If you have sufficient memory, a simple option is to increase the index setting index.max_result_window from 10000 to the number you require (in this case around 27000).

See https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#dynamic-index-settings

However, this will not scale indefinitely - you'll run out of memory if you set it too big. It might be worth looking at alternatives to jumping to the last page. For example, changing the sort order in your example to 'ASC' would effectively give you the last page without having to page through all the others first.