in solr is there an way to fetch all results from

Question

as far as I know this is not possible as the solr wiki. do you guys have any work around?

Answer 1

If you need to get everything out, you can either set the number of rows ridiculously high (as indicated above, with the caveat that, well, it won't work because you'll run out of memory) or iterate through your results using "rows" and "start"

Pseudocode:

numresults = <do an initial query and find out how many results there are>
rows = 100 (or 1000 or 100,000 or whatever you can handle)
start = 0
while (start) <= numresults
 url = "http://.../?q=<your query>&rows=<rows>&start=<start>&...
 result = <do the call with the url>
 <process the docs>
 start += rows
endwhile

See http://wiki.apache.org/solr/CommonQueryParameters for use of "start"

Also remember that when you're grabbing gobs of documents, use the 'fl' parameter to only pull back what you're actually gonna use.

Answer 2

The only workaround is to set the rows value large enough to return all documents.

However, I wouldn't recommend this for anything larger than about 1000 documents. If the number of documents you are fetching is large enough, you will run into memory or timeout issues with the XML you have to generate and parse. For example, if there are 2-3 million documents in your index, do you really want all of that in a single response? It's paginated for a reason. You should probably leverage it.

Of secondary concern... Why are you doing this to begin with? What's the point of putting a bunch of data into a search index, if you are just going to pull it ALL out? You may be better off using your original data source at that point.