Techniques in dealing with Eventual Consistency of

问题:

How does existing persistence framework for the GAE, like Objectify, Datanucleus, Twig, etc. deal with the "eventual consistent" nature of the datastore?

I'm working with the datastore in the DatastoreService layer (I'm not using such persistence framework right now).

During my unit test I get the correct count of object sometimes and sometimes not. Which is expected.

This is my JUnit helper configuration:

private final LocalServiceTestHelper helper =
    new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig()
        .setDefaultHighRepJobPolicyUnappliedJobPercentage(50)); 

Now, even if I have placed the put/get methods in a do-while loop with retries (with retry count of 4), it still misses to put/get entities.

So right now, for those specified frameworks, were they able to assure that if I put 3 object I will get 3 objects? If so how was it implemented?

The only thing I can think of is to use Memcache, that is, my methods will try to get the entity from it first, and if it misses that's the only time it will check the datastore. Is this a sound approach, or there is a better approach or right approach?

回答1:

If you want strong consistency of your queries then there are two choices:

1. ) Use ancestor queries.

2. ) Use datastore get operations instead of queries, they are strongly consistent.

It is often possible to use gets instead of queries if you you structure your key names carefully. Your other choice, if you need queries to "appear" consistent for a given user is to apply a little trick: inject the users changes into the results. So, if they change their name and you redirect to their profile page, inject their change into the results before sending them.

回答2:

Objectify does not deal with eventual consistency in any way, shape or form. You, the developer, have to keep that in mind when using Objectify, or any other library for that matter.

A do/while loop around the query, or adding a wait() does nothing to guarantee strong consistency. At best you increase the probability of getting the latest results, but it's nowhere near certain.

Memcache doesn't really offer any guarantees here either, because it's unreliable storage. It's perfectly legal for the memcache put() function to be a NO-OP, i.e. do nothing.

To address your core issue, you would need to design your app in a way that uses strong consistency. So you can either use strongly consistent datastore read operations like gets or ancestor queries. Or you can pass around some data via HTTP.

回答3:

If you require strong consistency in Objectify then you can perform an ancestor query, see this article for an explanation and examples. However, this is more expensive than an eventually consistent query, so favor eventually consistent queries where possible.