I'm developing a directory app on app-engine (python) and I've run into trouble with too many write ops. The first issue is that I have a .NET script that goes through an excel file and posts the data to a page in my app. When I ran it, it got through around 700 records and I had already used 75% of my write ops quota. The same thing happened when I wrote a script to update all of my models to have a search field for each property. I went from 75% of the quota filled to 96% in around 20 seconds and I got a temporary quota limit for too many writes in a short amount of time. I think the issue might be related to indexes but I'm sort of a newb when it comes to python and appengine. Here are my models :
class AlumniEntry(db.Model):
"""Models an entry for a single alumni"""
author = db.UserProperty()
entered = db.DateTimeProperty(auto_now_add=True)
title = db.StringProperty()
first_name = db.StringProperty(required=True)
first_name_search = db.StringProperty()
maiden_name = db.StringProperty()
maiden_name_search = db.StringProperty()
spouse_name = db.StringProperty()
spouse_name_search = db.StringProperty()
grad_year = db.StringProperty(required=True)
elementary = db.StringProperty(choices=('yes', 'no', 'idk'), default='idk')
class LastName(db.Model):
entry = db.ReferenceProperty(AlumniEntry, collection_name='last_names')
last_name = db.StringProperty(required=True)
last_name_search = db.StringProperty()
class PhoneNumber(db.Model):
entry = db.ReferenceProperty(AlumniEntry, collection_name='phone_numbers')
number = db.PhoneNumberProperty(default=None)
class Email(db.Model):
entry = db.ReferenceProperty(AlumniEntry, collection_name='emails')
email = db.EmailProperty(default=None)
email_search = db.EmailProperty(default=None)
class Address(db.Model):
entry = db.ReferenceProperty(AlumniEntry, collection_name='addresses')
street = db.StringProperty()
street_search = db.StringProperty()
city = db.StringProperty()
city_search = db.StringProperty()
state = db.StringProperty()
state_search = db.StringProperty()
zip_code = db.StringProperty()
class UserAuth(db.Model):
added_by = db.StringProperty(required=True)
user_id = db.StringProperty(required=True)
Unless you target a property in a search (or use it for ordering), making them unindexed saves index writes. Each property is indexed twice (once ascending, once descending) unless either the property type is inherently unindexed, or you set
indexed=False
.See http://code.google.com/appengine/docs/python/datastore/propertyclass.html#Property
In your case, if
street_search
is a normalized form ofstreet
used for searching, then markingstreet
asindexed=False
will save 2 writes.