NDB HRD transactions, which ancestor determines th

Question

Is it the closest or the most distant parent relative of the entity being written, which determines the entity group? (Question 1) For, if I have,

two simultaneous requests to write two different entities, in this example, both having immediate parent the Data entity (with key '2'), and having subsequent ancestors of:

Person:9 > Collection:3 > Script:4 > Data:2 > Record of Tom Cruise
Person:9 > Collection:3 > Script:4 > Data:2 > Record of Shia La Boef

In either case they both belong to the same entity group, either anchored at entity Person:9, or at entity Data:2. Which is the correct determiner of the entity group, Person:9 or Data:2? Also if there are two kinds of entities descended from Data:2, say Record and Scheme, will all the Record and Scheme entities belong to the same entity group, anchored by Data:2, or, by virtue of being different kinds, belong to separate entity groups? (Question 2)

Incidentally, if it is Person:9 which determines the entity group, and different kinds under a parent do not form different entity groups under that parent, then everything descended from Person:9 is the same entity group and is going to have to be written in serial, the horror

Since in this example, I am performing these writes of the same kind of entity to the same entity group concurrently, they will be applied serially, and therefore take 'double the time.', than if they could be applied concurrently.

What is a good solution for this 'doubling' of time taken? (Question 3 -- optional!)

I have thought of the following:

Since I know that the two separate writes must be initiated by two separate client instances, I can add a further ancestor to the chain, which represents the client instance doing the writing, like so:

Person:9 > Collection:3 > Script:4 > Data:2 > **Client:92** > Record of Tom Cruise
Person:9 > Collection:3 > Script:4 > Data:2 > **Client:37** > Record of Shia La Boef

This way the writes will belong to different entity groups (so long as the hypothesis of Person:9 anchoring the group is wrong), and therefore can always be performed concurrently. Can an AppEngineer/expert weigh in on this? (Question 4)

Further since I enforce the restriction that separate clients can only make serial requests to the datastore, and I can guarantee without any performance impact that any writes made by a single client never need to occur more than 1 time per second, the above method, if it works, will mean there is zero performance impact and as long as I have enough separate Clients (and, he, enough quota) I can make as many writes to the datastore as fast as the HTTP can carry them. Can an AppEngineer/expert weigh in on this? (Question 5)

The only issue I see with this group splitting approach is that querying for the Record entities under the Data:2 parent, is now complicated by the fact that, even though the records are related semantically, they are separated by the different clients. So in order to collect all records, I need to first collect all clients, and then collect all there records. Can anyone see whether this would create a stupendously horrible performance impact, doing this kind of "query all the children of the children you just queried" query...? Can an AppEngineer/expert weigh in on this? (Question 6)

Answer 1

You have some misconceptions here.

Firstly, the documentation is quite explicit on what constitutes an entity group: it is everything under a root entity.

However I don't know where you got the idea that writes within an entity group are in some way more "serial" than ones outside. The documentation doesn't say that, or imply it. The only thing that it does say about this is that writes to a single entity group take place at no more than 1 per second.

The rest of your questions make no sense at all: adding another element to the chain doesn't change the root entity.

I'm not sure why you need such deep entity group chains in the first place. The documentation's advice on scaling is to keep entity groups small. If each leaf entity will only ever be written to by one client, it sounds like the client itself should be the root, and the rest of the path should not be part of the ancestry at all: perhaps you could use a ReferenceProperty to refer to one or more of those entities via its key.