backbone.js cache collections and refresh

Question

I have a collection that can potentially contain thousands of models. I have a view that displays a table with 50 rows for each page.

Now I want to be able to cache my data so that when a user loads page 1 of the table and then clicks page 2, the data for page 1 (rows #01-50) will be cached so that when the user clicks page 1 again, backbone won't have to fetch it again.

Also, I want my collection to be able to refresh updated data from the server without performing a RESET, since RESET will delete all the models in a collection, including references of existing model that may exist in my app. Is it possible to fetch data from the server and only update or add new models if necessary by comparing the existing data and the new arriving data?

Answer 1

Backbone.Collection.fetch has an option {add:true} which will add models into a collection instead of replacing the contents.

myCollection.fetch({add:true})

So, in your first scenario, the items from page2 will get added to the collection.

As far as your 2nd scenario, there's currently no built in way to do that.

According to Jeremy that's something you're supposed to do in your App, and not part of Backbone.

Backbone seems to have a number of issues when being used for collaborative apps where another user might be updating models which you have client side. I get the feeling that Jeremy seems to focus on single-user applications, and the above ticket discussion exemplifies that.

In your case, the simplest way to handle your second scenario is to iterate over your collection and call fetch() on each model. But, that's not very good for performance.

For a better way to do it, I think you're going to have to override collection._add, and go down the line dalyons did on this pull request.

Answer 2

In my app, I addressed the reset question by adding a new method called fetchNew:

app.Collection = Backbone.Collection.extend({

    // fetch list without overwriting existing objects (copied from fetch())
    fetchNew: function(options) {
        options = options || {};
        var collection = this,
            success = options.success;
        options.success = function(resp, status, xhr) {
            _(collection.parse(resp, xhr)).each(function(item) {
                // added this conditional block
                if (!collection.get(item.id)) {
                    collection.add(item, {silent:true});
                }
            });
            if (!options.silent) {
                collection.trigger('reset', collection, options);
            }
            if (success) success(collection, resp);
        };
        return (this.sync || Backbone.sync).call(this, 'read', this, options);
    }

});

This is pretty much identical to the standard fetch() method, except for the conditional statement checking for item existence, and using add() by default, rather than reset. Unlike simply passing {add: true} in the options argument, it allows you to retrieve sets of models that may overlap with what you already have loaded - using {add: true} will throw an error if you try to add the same model twice.

This should solve your caching problem, assuming your collection is set up so that you can pass some kind of page parameter in options to tell the server what page of options to send back. You'll probably want to add some sort of data structure within your collection to track which pages you've loaded, to avoid doing unnecessary requests, e.g.:

app.BigCollection = app.Collection.extend({

    initialize: function() {
        this.loadedPages = {};
    },

    loadPage: function(pageNumber) {
        if (!this.loadedPages[pageNumber]) {
            this.fetchNew({ 
                page: pageNumber,
                success: function(collection) {
                    collection.loadedPages[pageNumber] = true;
                }
            })
        }
    }

}); 

Answer 3

I managed to get update in Backbone 0.9.9 core. Check it out as it's exactly what you need http://backbonejs.org/#Collection-update.