When does Django look up the primary key of foreig

Question

I have two simple models, one representing a movie an the other representing a rating for a movie.

class Movie(models.Model):
    id = models.AutoField(primary_key=True)

    title = models.TextField()

class Rating(models.Model):
    id = models.AutoField(primary_key=True)

    movie = models.ForeignKey(Movie)
    rating = models.FloatField()

My expectation is that I would be able to first create a Movie and a Review referencing that movie then commit them both to the database, as long as I committed the Movie first so that it was given a primary key for the Review to refer to.

the_hobbit = Movie(title="The Hobbit")
my_rating = Rating(movie=the_hobbit, rating=8.5)
the_hobbit.save()
my_rating.save()

To my surprise it still raised an IntegrityError complaining that I was trying to specify a null foreign key, even the Movie had been committed and now had a primary key.

IntegrityError: null value in column "movie_id" violates not-null constraint

I confirmed this by adding some print statements:

print "the_hobbit.id =", the_hobbit.id           # None
print "my_rating.movie.id =", my_rating.movie.id # None
print "my_rating.movie_id =", my_rating.movie_id # None

the_hobbit.save()

print "the_hobbit.id =", the_hobbit.id           # 3
print "my_rating.movie.id =", my_rating.movie.id # 3
print "my_rating.movie_id =", my_rating.movie_id # None

my_rating.save()                                 # raises IntegrityError

The .movie attribute is referring to a Movie instance which does have a non-None .id, but .movie_id is holding into the value None that it had when the Movie instance was crated.

I expected Django to look up .movie.id when I tried to commit the Review, but apparently that's not what it's doing.

---

Aside

In my case, I've dealt this this behaviour by overriding the .save() method on some models so that they look up the primary keys of foreign keys again before saving.

def save(self, *a, **kw):
    for field in self._meta.fields:
        if isinstance(field, ForeignKey):
            id_attname = field.attname
            instance_attname = id_attname.rpartition("_id")[0]
            instance = getattr(self, instance_attname)
            instance_id = instance.pk
            setattr(self, id_attname, instance_id)

    return Model.save(self, *a, **kw)

This is hacky, but it works for me so I am not really looking for a solution to this particular problem.

---

I am looking for an explanation of Django's behaviour. At what points does Django look up the primary key for foreign keys? Please be specific; references to the Django source code would be best.

Answer 1

As stated by the docs:

The keyword arguments are simply the names of the fields you’ve defined on your model. Note that instantiating a model in no way touches your database; for that, you need to save().

Add a classmethod on the model class:

class Book(models.Model):
    title = models.CharField(max_length=100)

    @classmethod
    def create(cls, title):
        book = cls(title=title)
        # do something with the book
        return book

book = Book.create("Pride and Prejudice")

Add a method on a custom manager (usually preferred):

class BookManager(models.Manager):
    def create_book(self, title):
        book = self.create(title=title)
        # do something with the book
        return book

class Book(models.Model):
    title = models.CharField(max_length=100)

    objects = BookManager()

book = Book.objects.create_book("Pride and Prejudice")

origin: https://docs.djangoproject.com/en/dev/ref/models/instances/?from=olddocs#creating-objects

When you assign the_hobbit, you are assigning an instance of Movie, thus not hitting the database. Once you call 'save' the database does fill up, however your variable is still pointing to the object in memory, not aware of the sudden database change.

That said, changing the order of your sequence should also effectively create the objects:

the_hobbit = Movie(title="The Hobbit")
the_hobbit.save()
my_rating = Rating(movie=the_hobbit, rating=8.5)
my_rating.save()

Answer 2

Looking in the Django source, the answer lies in some of the magic Django uses to provide its nice API.

When you instantiate a Rating object, Django sets (though with some more indirection to make this generic) self.movie to the_hobbit. However, self.movie isn't a regular property, but is rather set through __set__. The __set__ method (linked above) looks at the value (the_hobbit) and tries to set the property movie_id instead of movie, since it's a ForeignKey field. However, since the_hobbit.pk is None, it just sets movie to the_hobbit instead. Once you try to save your rating, it tries to look up movie_id again, which fails (it doesn't even try to look at movie.)

Interestingly, it seems this behaviour is changing in Django 1.5.

Instead of

setattr(value, self.related.field.attname, getattr(
    instance, self.related.field.rel.get_related_field().attname))
# "self.movie_id = movie.pk"

it now does

    related_pk = getattr(instance, self.related.field.rel.get_related_field().attname)
    if related_pk is None:
        raise ValueError('Cannot assign "%r": "%s" instance isn\'t saved in the database.' %
                            (value, instance._meta.object_name))

which in your case would result in a more helpful error message.

Answer 3

My opinion is that after you call the save() method on your hobbit object, that object is saved. but the local reference that is present in your my_rating object doesnt really know it has to update itself with values that are present in the database.

So when you call my_rating.movie.id django doesnt recognize the need for a db query on the movie object again and hence you get None, which is the value that the local instance of that object contains.

but my_rating.movie_id doesnt depend on what data is present on your local instances - that is an explicit way of asking django to look into the database and see what information is there through the foreign key relationship.

Answer 4

The main issue has to do with side effects that are wanted or not. And with variables really being pointers to objects in Python.

When you create an object out of a model, it doesn't have a primary key yet as you haven't saved it yet. But, when saving it, should Django have to make sure it updates attributes on the already-existing object? A primary key is logical, but it would also lead you to expect other attributes being updated.

An example for that is Django's unicode handling. Whatever charset you give the text you put into a database: Django gives you unicode once you get it out again. But if you create an object (with some non-unicode attribute) and save it, should Django modify that text attribute on your existing object? That already sounds a little bit more dangerous. Which is (probably) why Django doesn't do any on-the-fly updating of objects you ask it to store in the database.

Re-loading the object from database gives you a perfect object with everything set, but it also makes your variable point to a different object. So that would not help in your example in case you already gave the Rating a pointer at your "old" Movie object.

The Movie.objects.create(title="The Hobbit") mentioned by Hedde is the trick here. It returns a movie object from the database, so it already has an id.

the_hobbit = Movie.objects.create(title="The Hobbit")
my_rating = Rating(movie=the_hobbit, rating=8.5)
# No need to save the_hobbit, btw, it is already saved.
my_rating.save()

(I had problems with the difference between my objects and objects from the database, too, when my newly created object didn't output unicode. The explanation I put on my weblog is the same as above, but worded a bit differently.)

Answer 5

Just to complete, as I am not able to comment...

You may also (but rather not in this case) be willing to change the behaviour on the database side. This may be useful for running some tests that may lead to similar problems (since they are done in a commit and rollbacked). Sometimes it may be better to use this hacky command in order to keep the tests as close as possible to the real behaviour of the app rather than packing them in a TransactionalTestCase :

It has to do with the properties of the constraints... Executing the following SQL command will also solve the problem (PostgreSQL only):

SET CONSTRAINTS [ALL / NAME] DEFERRABLE INITIALLY IMMEDIATE;