Is it possible to control order of replication?

Question

I have a huge master CouchDB database and slave read-only CouchDB database, that synchronizes with master database.
Because rate of changes is quick, and channel between servers is slow and unstable, I want to set order/priority to define what documents come first. I need to ensure that the documents with highest priority are definitely of the latest version, and I can ignore documents in the end of list.

SORTING, not FILTERING

If it is not possible, what solution could be?

Resource I have already looked at:
http://wiki.apache.org/couchdb/Replication
http://couchapp.org/page/index

UPDATE: the master database is actually Node.js NPM registry, and order is list of Most Depended-upon Packages. I am trying to make proxy, because cloning 50G always fails after a while. But the fact is "we don't need 90% of those modules, but quick & reliable access to those we depend on."

Answer 1

The short answer is no.

The long answer is that CouchDB provides ACID guarantees at the individual document level only, by design. The replicator will update each document atomically when it replicates (as can anyone, the replicator is just using the public API) but does not guarantee ordering, this is mostly because it uses multiple http connections to improve throughput. You can configure that down to 1 if you like and you'll get better ordering, but it's not a panacea.

After the bigcouch merge, all bets are off, there will be multiple sources and multiple targets with no imposed total order.

Answer 2

CouchDB, out of the box, does not provide you with any options to control the order of replication. I'm guessing you could piece something together if you keep documents with different priorities in different databases on the master, though. Then, you could replicate the high-priority master database into the slave database first, replicate lower-priority databases after that, etc.

Answer 3

You could setup filtered replication or named document replication:

• http://wiki.apache.org/couchdb/Replication#Filtered_Replication
• http://wiki.apache.org/couchdb/Replication#Named_Document_Replication

Both of these are alternatives to replicating an entire database. You could do the replication in smaller batch sizes, and order the batches to match your priorities.