Full-text search in NoSQL databases

Question

• Has anyone here have any experience deploying a real online system that had a full text search in any of the NoSQL databases?
• For example, how does the full-text search compare in MongoDB, Riak and CouchDB?
• Some of the metric that I am looking for is ease of deployment and maintaince and of course speed.
• How mature are they? Are they any replacement for the Lucene infrastructure?

Thanks.

Answer 1

I've just finished completion of this using data that is stored in MongoDB while having my Fulltext engin in Sphinx Search. I know mongo has a votable issue for adding fulltext to a future release; however at this point they don't have it.

There are several ways of inserting your Mongo information into sphinx; however the one I've found the most luck with (and has been extremely easy) is through xmlpipe2. It took me a bit to fully understand how to use this; however this article: Sphinx xmlpipe2 in PHP has an outstanding walk through which shows (at least in PHP) how to build the document, then how to insert it into sphinx.

Essentially my config ends up looking like this:

source my_source {
     type = xmlpipe
     xmlpipe_command = /usr/bin/php /www/generateSphinXml.php identifierForMyTable
}

with my index then looking like this:

index my_index {
     source = my_source
     path = /usr/local/sphinx/var/data/my_index
     docinfo = extern
     min_word_len = 1
     mlock = 0
     morphology = stem_en
     charset_type = utf-8 //<----- This is q requirement however.
     enable_star = 1
     html_strip = 0
     min_prefix_len = 2
}

I've had excellent success with this; hopefully you can find this as useful.

Answer 2

MarkLogic has better options for text search, if I recall. Here is a discussion on the topic, though it is on their blog, from their writers.

Answer 3

If you are using PHP there is a great solution for fulltext search in No-SQL database MongoDB named as MongoLantern. http://sourceforge.net/projects/mongolantern/

Previously I was using Sphinx+MongoDB to perform fulltext search, the performance was great but result quality was very poor. With MongoLantern my current search improved a lot.

MongoLantern is also listed in MongoDB site.

Please let me know if you try it of your own.

Answer 4

Solr could be used with 10gen's Mongo Connector, which allows to push data there (among others)

https://github.com/10gen-labs/mongo-connector/tree/master/mongo-connector

From their example:

python mongo_connector.py -m localhost:27217 -t http://localhost:8080/solr

Answer 5

cLunce project. Also xapian not mentioned above. I use Sphinx and it's very good but somewhat clumsy to set up. I actually prefer piping data from Mongo into Sphinx via XMLPIPE2, instead of using Sphinx' SQL in sphinx.conf file.

Answer 6

Here's the details on Riak Search http://wiki.basho.com/Riak-Search.html and a presentation on it as well