App Engine (Python) Datastore Precall API Hooks

Question

Background

So let's say I'm making app for GAE, and I want to use API Hooks.

BIG EDIT: In the original version of this question, I described my use case, but some folks correctly pointed out that it was not really suited for API Hooks. Granted! Consider me helped. But now my issue is academic: I still don't know how to use hooks in practice, and I'd like to. I've rewritten my question to make it much more generic.

---

Code

So I make a model like this:

class Model(db.Model):
    user = db.UserProperty(required=True)
    def pre_put(self):
        # Sets a value, raises an exception, whatever.  Use your imagination

And then I create a db_hooks.py:

from google.appengine.api import apiproxy_stub_map

def patch_appengine(): 
    def hook(service, call, request, response):
        assert service == 'datastore_v3'
        if call == 'Put':
            for entity in request.entity_list():
                entity.pre_put()

    apiproxy_stub_map.apiproxy.GetPreCallHooks().Append('preput',
                                                        hook,
                                                        'datastore_v3')

Being TDD-addled, I'm making all this using GAEUnit, so in gaeunit.py, just above the main method, I add:

import db_hooks
db_hooks.patch_appengine()

And then I write a test that instantiates and puts a Model.

---

Question

While patch_appengine() is definitely being called, the hook never is. What am I missing? How do I make the pre_put function actually get called?

Answer 1

I don't think that Hooks are really going to solve this problem. The Hooks will only run in the context of your AppEngine application, but the user can change their nickname outside of your application using Google Account settings. If they do that, it won't trigger any logic implement in your hooks.

I think that the real solution to your problem is for your application to manage its own nickname that is independent of the one exposed by the Users entity.

Answer 2

The issue here is that within the context of the hook() function an entity is not an instance of db.Model as you are expecting.

In this context entity is the protocol buffer class confusingly referred to as entity (entity_pb). Think of it like a JSON representation of your real entity, all the data is there, and you could build a new instance from it, but there is no reference to your memory-resident instance that is waiting for it's callback.

Monkey patching all of the various put/delete methods is the best way to setup Model-level callbacks as far as I know†

Since there doesn't seem to be that many resources on how to do this safely with the newer async calls, here's a BaseModel that implements before_put, after_put, before_delete & after_delete hooks:

class HookedModel(db.Model):

    def before_put(self):
        logging.error("before put")

    def after_put(self):
        logging.error("after put")

    def before_delete(self):
        logging.error("before delete")

    def after_delete(self):
        logging.error("after delete")

    def put(self):
        return self.put_async().get_result()

    def delete(self):
        return self.delete_async().get_result()

    def put_async(self):
        return db.put_async(self)

    def delete_async(self):
        return db.delete_async(self)

Inherit your model-classes from HookedModel and override the before_xxx,after_xxx methods as required.

Place the following code somewhere that will get loaded globally in your applicaiton (like main.py if you use a pretty standard looking layout). This is the part that calls our hooks:

def normalize_entities(entities):
    if not isinstance(entities, (list, tuple)):
        entities = (entities,)
    return [e for e in entities if hasattr(e, 'before_put')]

# monkeypatch put_async to call entity.before_put
db_put_async = db.put_async
def db_put_async_hooked(entities, **kwargs):
    ents = normalize_entities(entities)
    for entity in ents:
        entity.before_put()
    a = db_put_async(entities, **kwargs)
    get_result = a.get_result
    def get_result_with_callback():
        for entity in ents:
            entity.after_put()
        return get_result()
    a.get_result = get_result_with_callback
    return a
db.put_async = db_put_async_hooked


# monkeypatch delete_async to call entity.before_delete
db_delete_async = db.delete_async
def db_delete_async_hooked(entities, **kwargs):
    ents = normalize_entities(entities)
    for entity in ents:
        entity.before_delete()
    a = db_delete_async(entities, **kwargs)
    get_result = a.get_result
    def get_result_with_callback():
        for entity in ents:
            entity.after_delete()
        return get_result()
    a.get_result = get_result_with_callback
    return a
db.delete_async = db_delete_async_hooked

You can save or destroy your instances via model.put() or any of the db.put(), db.put_async() etc, methods and get the desired effect.

†would love to know if there is an even better solution!?

Answer 3

Hooks are a little low level for the task at hand. What you probably want is a custom property class. DerivedProperty, from aetycoon, is just the ticket.

Bear in mind, however, that the 'nickname' field of the user object is probably not what you want - per the docs, it's simply the user part of the email field if they're using a gmail account, otherwise it's their full email address. You probably want to let users set their own nicknames, instead.