Key-Value Database

问题:

I need to store short strings (50+ characters), and quickly look them up. At first I wanted to use SQLite for this. I created a single table with a single indexed TEXT column. Using all kinds of low-level tricks I could fill the database with 1 million strings in 10 seconds.

The problem was that if the PC was rebooted, adding an additional 10.000 rows took 30 seconds, which is nowhere in line with 1M rows in 10s. The reason for this is that SQLite has to read a very large part of the existing index from disk, before it can add the new rows. Another problem is that the database doubles in size, because all strings are stored twice (once in the regular table, once in the index table) because of the b-tree mechanism.

Is there a simple alternative? Which uses hash-based lookups for example? I know about the various NoSQL solutions, but they are focussed at decentral and distributed storage, while I just need a simple embedded 'flat-file' database?

回答1:

How critical is it to persist the latest values to disk?

You can use an in-memory collection such as Dictionary or List and persist it to disk by serializing it to a local file in an ansynchronous call at regular intervals.

Then when your application starts you can deserialize it back into memory.

This would be a far more responsive way to accomplish what you are after as you dont have to keep hammering the disk all the time with read/write efforts.