Recursive delete in google app engine

Question

I'm using google app engine with django 1.0.2 (and the django-helper) and wonder how people go about doing recursive delete. Suppose you have a model that's something like this:

class Top(BaseModel):
    pass

class Bottom(BaseModel):
    daddy = db.ReferenceProperty(Top)

Now, when I delete an object of type 'Top', I want all the associated 'Bottom' objects to be deleted as well.

As things are now, when I delete a 'Top' object, the 'Bottom' objects stay and then I get data that doesn't belong anywhere. When accessing the datastore in a view, I end up with:

Caught an exception while rendering: ReferenceProperty failed to be resolved.

I could of course find all objects and delete them, but since my real model is at least 5 levels deep, I'm hoping there's a way to make sure this can be done automatically.

I've found this article about how it works with Java and that seems to be pretty much what I want as well.

Anyone know how I could get that behavior in django as well?

Answer 1

Actually that behavior is GAE-specific. Django's ORM simulates "ON DELETE CASCADE" on .delete().

I know that this is not an answer to your question, but maybe it can help you from looking in the wrong places.

Answer 2

If your hierarchy is only a small number of levels deep, then you might be able to do something with a field that looks like a file path:

daddy.ancestry = "greatgranddaddy/granddaddy/daddy/"
me.ancestry = daddy.ancestry + me.uniquename + "/"

sort of thing. You do need unique names, at least unique among siblings.

The path in object IDs sort of does this already, but IIRC that's bound up with entity groups, which you're advised not to use to express relationships in the data domain.

Then you can construct a query to return all of granddaddy's descendants using the initial substring trick, like this:

query = Person.all()
query.filter("ancestry >", gdaddy.ancestry + "\U0001")
query.filter("ancestry <", gdaddy.ancestry + "\UFFFF")

Obviously this is no use if you can't fit the ancestry into a 500 byte StringProperty.

Answer 3

You need to implement this manually, by looking up affected records and deleting them at the same time as you delete the parent record. You can simplify this, if you wish, by overriding the .delete() method on your parent class to automatically delete all related records.

For performance reasons, you almost certainly want to use key-only queries (allowing you to get the keys of entities to be deleted without having to fetch and decode the actual entities), and batch deletes. For example:

db.delete(Bottom.all(keys_only=True).filter("daddy =", top).fetch(1000))

Answer 4

Reconsider the data structure. If the relationship will never change on the record lifetime, you could use "ancestors" feature of GAE:

class Top(db.Model): pass
class Middle(db.Model): pass
class Bottom(db.Model): pass

top = Top()
middles = [Middle(parent=top) for i in range(0,10)]
bottoms = [Bottom(parent=middle) for i in range(0,10) for middle in middles]

Then querying for ancestor=top will find all the records from all levels. So it will be easy to delete them.

descendants = list(db.Query().ancestor(top))
# should return [top] + middles + bottoms