Google app engine: how to handle concurrency (raci

Question

I am trying to solve the racing problem based on this to prevent duplicate user registrations. So if the account exists or the email has been used, no entity will be created.

@ndb.transactional
def get_or_insert2(account, email):
    accountExists, emailExists = False, False
    entity = Member.get_by_id(account)
    if entity is not None:
        accountExists = True
    if Member.query(Member.email==email).fetch(1):
        emailExists = True
    if not accountExists and not emailExists:
        entity = Member(id=account)
        entity.put()
    return (entity, accountExists, emailExists)

My questions:

1. I got an error message: BadRequestError: Only ancestor queries are allowed inside transactions. what was the problem?

2. Is the code correct? I mean, can it really solve the racing problem?

Thanks.

Answer 1

Transactions work on entity groups, and you can include up to 5 entity groups in a cross group transaction. An entity group is handled by a single server (or group, replicated), which means it is able to have consistent internal state when checking data or doing ancestor queries within the entity group.

Regular queries are global, on indexes with eventual consistency. You don't know when all changes from all nodes have been included in an index. You can't lock up the entire datastore to get consistent snapshot state for your transaction. This is a key difference from a regular RDBMS if you're used to consistent index for queries.

For 1), the problem is that you're doing a regular query inside a transaction, which doesn't work as explained above. The answer to 2) then becomes no, query can't solve racing problem, you need explicit gets.

You will need a Model for Member, Email and SSN. This is a quick untested example that hopefully gets you going:

class Member(ndb.Model):
    email = ndb.KeyProperty()
    ssn = ndb.KeyProperty()
    # More user properties goes here...

class Email(ndb.Model):
    member = ndb.KeyProperty()

class SSN(ndb.Model):
    member = ndb.KeyProperty()

@ndb.tasklet
def get_or_insert2(account, email, ssn):
    created = False
    member_key = ndb.Key(Member, account)
    email_key = ndb.Key(Email, email)
    ssn_key = ndb.Key(SSN, ssn)
    member_obj, email_obj, ssn_obj = yield ndb.get_multi_async([member_key, email_key, ssn_key])

    if member_obj is None and email_obj is None and ssn_obj is None:
        member_obj = Member(key=member_key, email=email_key, ssn=ssn_key))
        email_obj = Email(key=email_key, member=member_key)
        ssn_obj = SSN(key=ssn_key, member=member_key)
        yield ndb.put_multi_async([member_obj, email_obj])
        created = True

    raise ndb.Return([created, member_obj, email_obj, ssn_obj])

outcome = ndb.transaction(lambda: get_or_insert2(account, email, ssn), xg=True)

I'm not sure if it works to combine @ndb.tasklet and @ndb.transactional(xg=True) decorators, and if so, which order, just try it out.

If you need to query User based on email or ssn, you could for example rename the KeyProperties to *_ref and make something like

@ndb.ComputedProperty
def email(self):
    return self.email_ref.id()

While this ends up being more lines of code than you anticipated, it is conceptually simple and straight forward, and you can easily figure out what's going on when you get back to it later.