How do you push 10,000 results into an array befor

问题:

I have a Parse.com query and a function where I want to send the data from the Parse.com query. I need to sum all of the results from the table and it returns more than 1,000 Parse.com objects. I'm sure there are many ways to deal with this issue but here is the approach I've taken for similar tasks:

I will usually send the first 1,000 results on in the program while looping through to get 10,000 or more results.

This time my problem is different because I need to send an array of all the results at once to the next function so they can be processed at once. I've looked for an informative example on how to do something like this but I have yet to understand the examples I've seen.

Over the months I've scoured Stackoverflow, Github, Parse docs and the Parse forum as well as the general web looking for a better understanding. Please don't waste our time by sending a link without some additional insight as it's likely I already read it and I don't understand it or can't apply it as is.

1) If I try to put the query inside a function so that I could recursively collect the results I can't figure out how to return the result from outside of the parse.com query. When I do things this way, I just pass the result to another function called inside the first function. To deal with results greater than 1,000 in number I'll loop through recursively passing results to the function called within the function until all results from Parse.com are processed. Since I need to sum everything at once (I think? I'm sure there are other ways to do this and that's a solution to what I'm asking)

2) If I pass the first thousand results on to the next function I don't know how to make things pause so the loop can complete sending all of the functions so it can all be processed at once.

There are probably other ways to deal with this but this is what I was thinking.

Here is a small sample of something I'd like to accomplish:

var query = new Parse.Query("Table");
query.find().then(function(results){
//results.length > 1000 and I need to process all of the results in $scope.step
$scope.step = _.filter(results, function(n){
return moment(h.createdAt).isAfter(moment([2015]))})

$scope.step is being called for display in the front end HTML. I'm sure there is a clever way to work with promises or callbacks with loops or arrays or something to make this happen. I suppose I could repeat the query 10 times skipping 1,000 results each time but that would very messy and a nightmare to maintain.

I've since tried the following examples to try and get something to work.

Attempt 1:

var array = [];
var agentQueryAll = new Parse.Query(Parse.User);
query.limit(1000)
query.find().then(function (results) {
       var increment = 0;
       while(results > 1000){
                var query1 = new Parse.Query(Parse.User);
                query1.limit(1000);
                query1.skip(increment*1000);
                query1.find().then(function (results) {
                    array.push(results);
                    increment++
                })}
            console.log(array);//this is an array with only the first 1,000 results

Attempt2:

var array = [];
            var SO = function (increment) {
                var agentQueryAll = new Parse.Query(Parse.User);
                agentQueryAll.limit(1000);
                agentQueryAll.skip(increment*1000);
                agentQueryAll.find().then(function (results) {
                    array.push(results);
                        if(results.length > 1000){
                            increment++
                            SO(increment);
                        } else {
                            return array;
                        }
                    });
            };
SO();

If I console.log the array inside function SO I can see the correct results accumulate in the array on my Javascript Console. When I console.log(SO()) I get an array with empty arrays. If I console.log(array) outside function SO() I get an empty array [].

回答1:

Instead of recursion, you can use an array.reduce(...) pattern to fetch your results in batches.

As there's no naturally occurring array to reduce, you need to synthesise one, with one element per batch. The array elements could be anything you like but it's convenient for them to be the batch-size, with any remainder as the final element. The values in the array will be the limit passed to query.limit().

function findInBatches(N, batchSize) {
    //First apportion N into an array of limits, 
    //each of bachSize, and finally any remainder.
    var limits = [];
    for(var nn = N; nn > 0; nn -= batchSize) {
        limits.push(Math.min(nn, batchSize));
    }

    // Now, the limits array can be reduced to build a promise chain,
    // which progressively accumulates results in an array as the query.find() promises settle.
    var accumulator = [];
    return limits.reduce(function(promise, limit, i) {
        var query = new Parse.Query(Parse.User);
        query.limit(limit);
        query.skip(i * batchSize);
        return promise.then(function() {
            return query.find().then(function (results) {
                if (results && results.length) {
                    accumulator = accumulator.concat(results);
                } else {
                    return Parse.Promise.error('Premature end of results');
                }
            });
        });
    }, Parse.Promise.as()) // Parse.Promise.as() gets the chain started.
    .then(function() {
        return accumulator;
    }, function (e) { // error handler
        console.log(e);
        if (accumulator.length) {
            return accumulator; //return any results fetched before the error occurred.
        } else {
            return Parse.Promise.error('No results found');//explain what went wrong.
        }
    });
}

Call as follows :

findInBatches(10000, 1000).then(function(results) {
    //use `results` here 
}, function(e) {
    //an error occurred. 'No results found' or something unpredicted
});

回答2:

So basically you need to create an array outside of the function that loops over the Parse.com and then push the results into it in order to fetch all the records. Then you need to execute the next function which will sum or do the final calculation. Notice that I don't know anything abou how parse.com works, this is simple javascript and it should work if you can adequate this solution to your specific problem:

var new_array = [], total_records = 35000, query = new Parse.Query("Table");

while (total_records > 10000){
    query.find().then(function(results){
        new_array.push(results);
    });
}

last_function(new_array);