Best practise of using localstorage to store a lar

Question

Currently I'm experimenting with localStorage to store a large amount of objects of same type, and I am getting a bit confused.

One way of thinking is to store all the object in an array. But then for each read/write of a single object I need to deserialise/serialise the whole array.

The other way is to directly store each object with its key in the localStorage. This will make accessing each object much easier but I'm worried of the amount of objects that will be stored (tens of thousands). Also, getting all the objects will require iterating the whole localStorage!

I'm wondering which way will be better in your experience? Also, would it be worthwhile to try on more sophisticated client side database like PouchDB?

Answer 1

If you do not want to have a lot of keys, you can:

• concat row JSONs with \n and store them as a single key
• build and update an index(es) stored under separate keys, each linking some key with a particular row number.

In this case parsing rows is just .split('\n') that is ~2 orders of magnitude faster, then JSON.parse.

Please, notice, that you possibly need special effort to syncronize simultaneously opened tabs. It can be a challenge in complex cases.

localStorage has both good and bad parts.

Good parts:

• syncronous;
• extremely fast, both read and write are just memcpy – it‘s 100+Mb/s throughput even on weak devices (for example JSON.stringify is in general 5-20 times slower than localStorage.setItem);
• thoroughly tested and reliable.

Bad news:

• no transactions, so you need an engineering effort to sync tabs;
• think you have not more than 2Mb (cause there exist systems with this limit);
• 2Mb of storage actually mean 1M chars you can save.

These points show borders of localStorage applicability as a DB. LS is good for tasks, where you need syncronicity and speed, and where you can trim you DB to fit into quota.

So localStorage is good for caches and logs. Not more.

Answer 2

I hadn't personally used localStorage to manage so many elements.

However, the pattern I usually use to manage data is to load the complete info database into a javascript object, manage it on memory during the proccess and saving it again to localStorage when the proccess is finished.

Of course, this pattern may not be a good approach to your needings, depending on your project specifications.

If you need to save data constantly, data access could become a problem, and thus probably using some type of small database access is a better option.

If your data volume is exceptionally high it also could be a problem to manage it on memory, however, depending on data model, you'd be able to build it to efficient structures that would allow you to load and save data just when it's needed.

Answer 3

If you want something simple for storing a large amount of key/values, and you don't want to have to worry about the types, then I recommend LocalForage. You can store strings, numbers, arrays, objects, Blobs, whatever you want. It uses IndexedDB and WebSQL where available, so the storage limits are much higher than LocalStorage.

PouchDB works too, but the API is more complex, and it's better-suited for when you want to sync data with CouchDB on the server.