Datastore write limit tests - trying to break app

Question

We´re trying to test the write limit exceptions mentioned to be about 1 write / second to prep our code for it (https://developers.google.com/appengine/docs/python/datastore/exceptions -> Timeout)

So I´m creating a item and updating it with the loop count 10k times via tasks and 10k times via a loop... It doesn´t seem to trigger a exception although the writes per second should be high enough (I remember something like more than one write per second gets critical).

Always the same: things don´t break when your´re you want them to ;).

class Message(ndb.Model):
    text = ndb.StringProperty()
    count = ndb.IntegerProperty()

#defined in seperate file
class DeferredClass(object):
    def put(self, id, x):
        msg = Message.get_by_id(id)
        msg.count = x
        try:
            msg.put()
        except:
            logging.error("error putting the Message")
            logging.error(sys.exc_info()[0])


msg = Message(text="TestGreeting", count=0)
key = msg.put()
id = key.id()
test = DeferredClass()

for x in range(10000):
    deferred.defer(test.put, id, x)
for x in range(10000):
    msg.count = x
    try:
        msg.put()
    except:
        logging.error("error putting the Message")
        logging.error(sys.exc_info()[0])

self.response.out.write("done")

PS: We´re aware that the docs are for db and the code is ndb... the basic limitations should still exist... Also: Docs on ndb Exceptions would be great! Anyone?

Answer 1

Using a non-default TaskQueue with a increased rate limit of 350/tasks/sec led to 20 instances being fired up and plenty of Timeout Exceptions... Thanks Mr. Steinrücken!

The Exception is google.appengine.api.datastore_errors.Timeout, which is the same as documented for the db package - so no ndb extras there.

PS: Our idea is to catch the exception in our cache handling class as a sign of datastore overload and automatically set up shading for that item... monitoring the quests a minute and diable shading again when not needed...