Embedded non-relational (nosql) data store [closed

Question

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.

Closed 6 years ago.

I'm thinking about using/implementing some kind of an embedded key-value (or document) store for my Windows desktop application. I want to be able to store various types of data (GPS tracks would be one example) and of course be able to query this data. The amount of data would be such that it couldn't all be loaded into memory at the same time.

I'm thinking about using sqlite as a storage engine for a key-value store, something like y-serial, but written in .NET. I've also read about FriendFeed's usage of MySQL to store schema-less data, which is a good pointer on how to use RDBMS for non-relational data. sqlite seems to be a good option because of its simplicity, portability and library size.

My question is whether there are any other options for an embedded non-relational store? It doesn't need to be distributable and it doesn't have to support transactions, but it does have to be accessible from .NET and it should have a small download size.

UPDATE: I've found an article titled SQLite as a Key-Value Database which compares sqlite with Berkeley DB, which is an embedded key-value store library.

Answer 1

You can try this one https://github.com/mdsoftware/mData. Small, free and quite unusual. Lisp-like data query language, expression compiler, high-performance binary serialization, all included.

Answer 2

Personally I would go for SQLite with NHibernate (and Fluent NHibernate). NHibernate can generate the database schema automatically for your classes, so you just need to specify what classes you want to persist, and that's quite easy with Fluent NHibernate. Furthermore, you can search for specific objects and you don't need to load all data to memory.

Answer 3

Could you create a simple sqlite database with two columns:

==documents==
id|data

and data would be json data.

You could also create a key table which would be:

==keys==
keyname|keyvalue|id

that would be indexed on keyname and keyvalue for quick lookups.

A single db file could be a collection, and you could create multiple db files for multiple collections.

You could use folders as "dbs" to match mongodb's hierarchy of db->collection->document

Answer 4

This is an old question, but I thought I'd add an answer in case anyone stumbles on it. My company just released an open source embedded XML database for the .NET platform called Nxdb. It's under the Apache 2.0 license and has been in development and use internally for several years. It's basically a binding to a cross-compiled (using IKVM) version of BaseX (a fantastic Java XML database) along with extra functionality for the embedded use case and the .NET environment. The project page is here: https://dracorp.assembla.com/spaces/nxdb

XML works well for this type of data store given that as long as the content you're trying to store is serializable to text you can store complex hierarchical trees. In fact, if you access the database directly, you don't ever even have to touch "XML". It can also be queried with XQuery, a powerful and complete query language.

Answer 5

Take a look at RavenDB. It looks as though it can be embedded and is schemaless and works with .NET

From the website:

• Scalable infrastructure: Raven builds on top of existing, proven and scalable infrastructure
• Simple Windows configuration: Raven is simple to setup and run on windows as either a service or IIS7 website
• Transactional: Raven support System.Transaction with ACID transactions. If you put data in it, that data is going to stay there
• Map/Reduce: Easily define map/reduce indexes with Linq queries
• .NET Client API: Raven comes with a fully functional .NET client API which implements Unit of Work and much more
• RESTful: Raven is built around a RESTful API

Answer 6

Windows has a built-in embedded non-relational store. It is called ESENT and is used by several Windows applications, including the Active Directory and Windows Desktop Search.

http://blogs.msdn.com/windowssdk/archive/2008/10/23/esent-extensible-storage-engine-api-in-the-windows-sdk.aspx

If you want .NET access you can use the ManagedEsent layer on CodePlex.

http://managedesent.codeplex.com/

That project has a PersistentDictionary class that implements a key-value store that implements the IDictionary interface, but is backed by a database.