Django: “Soft” ForeignField without database integ

Question

I have a Django project that has multiple django "apps". One of them has models to represent data coming from an external source (I do not control this data).

I want my other apps to be able to have references to this "external app" but I want to avoid all the fuzz of the database integrity checks. I don't want the db to have any constraints on these "soft foreign keys".

Do you know how I can code a custom field that will emulate a real Django ForeignKey without creating a hard constraint on the database?

Maybe this already exists, but I didn't have any luck on Google.

Thanks in advance for the help :-)

NB: I'm aware of the generic relations system with the content_types. But I don't want generic relations. I want specific relations to identified models only without hard integrity constraints.

EDIT:

I found related links:

• Django ForeignKey which does not require referential integrity?
• Understanding / mySQL aka tricking ForeignKey relationships in Django

But I didn't find a proper answer to my question. :(

EDIT 2012, june 4:

I've looked deep into django's code to find what needs to be done, but I think that simply subclassing ForeignKey will not be enough. Could you give me some directions on how to do this?

NB: I use South for managing my database schema, so I figure I'll need to do something about that too. But it may be out of the subject here :)

Answer 1

I tried something similar to Izz ad-Din Ruhulessin's suggestion but it didn't work because I have columns other than the "fake FK" column. The code I tried was:

class DynamicPkg(models.Model):
    @property
    def cities(self):
        return City.objects.filter(dpdestinations__dynamic_pkg=self)


class DynamicPkgDestination(models.Model):
    dynamic_pkg = models.ForeignKey(DynamicPkg, related_name='destinations')
    # Indexed because we will be joining City.code to
    # DynamicPkgDestination.city_code and we want this to be fast.
    city_code = models.CharField(max_length=10, db_index=True)


class UnmanagedDynamicPkgDestination(models.Model):
    dynamic_pkg = models.ForeignKey(DynamicPkg, related_name='destinations')
    city = models.ForeignKey('City', db_column='city_code', to_field='code', related_name='dpdestinations')

    class Meta:
        managed = False
        db_table = DynamicPkgDestination._meta.db_table


class City(models.Model):
    code = models.CharField(max_length=10, unique=True)

and the errors I got were:

Error: One or more models did not validate:
travelbox.dynamicpkgdestination: Accessor for field 'dynamic_pkg' clashes with related field 'DynamicPkg.destinations'. Add a related_name argument to the definition for 'dynamic_pkg'.
travelbox.dynamicpkgdestination: Reverse query name for field 'dynamic_pkg' clashes with related field 'DynamicPkg.destinations'. Add a related_name argument to the definition for 'dynamic_pkg'.
travelbox.unmanageddynamicpkgdestination: Accessor for field 'dynamic_pkg' clashes with related field 'DynamicPkg.destinations'. Add a related_name argument to the definition for 'dynamic_pkg'.
travelbox.unmanageddynamicpkgdestination: Reverse query name for field 'dynamic_pkg' clashes with related field 'DynamicPkg.destinations'. Add a related_name argument to the definition for 'dynamic_pkg'.

However I did come up with a working solution by using a proxy model. I did still have to hack around some Django validation that prevents fields from being included in proxy models:

class DynamicPkg(models.Model):
    @property
    def cities(self):
        return City.objects.filter(dpdestinations__dynamic_pkg=self)



def proxify_model(new_class, base):
    """
    Like putting proxy = True in a model's Meta except it doesn't spoil your
    fun by raising an error if new_class contains model fields.
    """
    new_class._meta.proxy = True
    # Next 2 lines are what django.db.models.base.ModelBase.__new__ does when
    # proxy = True (after it has done its spoil-sport validation ;-)
    new_class._meta.setup_proxy(base)
    new_class._meta.concrete_model = base._meta.concrete_model


class DynamicPkgDestination(models.Model):
    dynamic_pkg = models.ForeignKey(DynamicPkg, related_name='destinations')
    # Indexed because we will be joining City.code to
    # DynamicPkgDestination.city_code and we want this to be fast.
    city_code = city_code_field(db_index=True)


class ProxyDynamicPkgDestination(DynamicPkgDestination):
    city = models.ForeignKey('City', db_column='city_code', to_field='code', related_name='dpdestinations')


proxify_model(ProxyDynamicPkgDestination, DynamicPkgDestination)


class City(models.Model):
    code = models.CharField(max_length=10, unique=True)

Answer 2

Yo guys,

I managed to make what I wanted.

First, I created a new field:

from django.db.models.deletion import DO_NOTHING
from django.db.models.fields.related import ForeignKey, ManyToOneRel

class SoftForeignKey(ForeignKey):
    """
    This field behaves like a normal django ForeignKey only without hard database constraints.
    """
    def __init__(self, to, to_field=None, rel_class=ManyToOneRel, **kwargs):
        ForeignKey.__init__(self, to, to_field=to_field, rel_class=rel_class, **kwargs)
        self.on_delete = DO_NOTHING

    no_db_constraints = True

Since I use South to manage my database schema, I had to add this:

from south.modelsinspector import add_introspection_rules
add_introspection_rules([], [r'^ecm\.lib\.softfk\.SoftForeignKey'])

Then, I had to monkey patch south so that it takes the no_db_constraints parameter into account. There were two functions involved in the creation of FK constraints:

from django.db.models.deletion import DO_NOTHING
from django.db.models.fields.related import ForeignKey, ManyToOneRel
from django.core.management.color import no_style
from south.db.generic import DatabaseOperations, invalidate_table_constraints, flatten

def column_sql(self, table_name, field_name, field, tablespace='', with_name=True, field_prepared=False):
    """
    Creates the SQL snippet for a column. Used by add_column and add_table.
    """

    # If the field hasn't already been told its attribute name, do so.
...
...
...

        if field.rel and self.supports_foreign_keys:
            # HACK: "soft" FK handling begin
            if not hasattr(field, 'no_db_constraints') or not field.no_db_constraints:
                self.add_deferred_sql(
                    self.foreign_key_sql(
                        table_name,
                        field.column,
                        field.rel.to._meta.db_table,
                        field.rel.to._meta.get_field(field.rel.field_name).column
                    )
                )
            # HACK: "soft" FK handling end

    # Things like the contrib.gis module fields have this in 1.1 and below
    if hasattr(field, 'post_create_sql'):
        for stmt in field.post_create_sql(no_style(), ta
....
....

# monkey patch South here
DatabaseOperations.column_sql = column_sql

And:

from django.db.models.deletion import DO_NOTHING
from django.db.models.fields.related import ForeignKey, ManyToOneRel
from django.core.management.color import no_style
from south.db.generic import DatabaseOperations, invalidate_table_constraints, flatten

@invalidate_table_constraints
def alter_column(self, table_name, name, field, explicit_name=True, ignore_constraints=False):
    """
    Alters the given column name so it will match the given field.
    Note that conversion between the two by the database must be possible.
    Will not automatically add _id by default; to have this behavour, pass
    explicit_name=False.

    @param table_name: The name of the table to add the column to
    @param name: The name of the column to alter
    @param field: The new field definition to use
    """

    if self.dry_run:
        if self.debug:
...
...
    if not ignore_constraints:
        # Add back FK constraints if needed
        if field.rel and self.supports_foreign_keys:
            # HACK: "soft" FK handling begin
            if not hasattr(field, 'no_db_constraints') or not field.no_db_constraints:
                self.execute(
                    self.foreign_key_sql(
                        table_name,
                        field.column,
                        field.rel.to._meta.db_table,
                        field.rel.to._meta.get_field(field.rel.field_name).column
                    )
                )
            # HACK: "soft" FK handling end

# monkey patch South here
DatabaseOperations.alter_column = alter_column

This is really ugly but I didn't find another way.

Now you can use the SoftForeignKey field exactly like a normal ForeignKey except that you won't have any referencial integrity enforcement.

See here for the complete monkey-patch : http://eve-corp-management.org/projects/ecm/repository/entry/ecm/lib/softfk.py

Answer 3

I solved this by using a GenericForeignKey:

thing_content_type = models.ForeignKey(ContentType, on_delete=models.CASCADE, blank=True, null=True)
thing_object_id = models.UUIDField(default=uuid.uuid4, blank=True, null=True)

thing = GenericForeignKey(ct_field='thing_content_type', fk_field='thing_object_id')

On the plus side, it's out-of-the-box Django

On the negative side, you have three additional attributes in your model.

Additionally, reverse relations don't automatically work, but in my case, I'm okay with that.

Answer 4

You could try using an unmanaged model:

from django.db import models


class ReferencedModel(models.Model):
    pass


class ManagedModel(models.Model):
    my_fake_fk = models.IntegerField(
        db_column='referenced_model_id'
    )


class UnmanagedModel(models.Model):
    my_fake_fk = models.ForeignKey(
        ReferencedModel, 
        db_column='referenced_model_id'
    )

    class Meta:
        managed = False
        db_table = ManagedModel._meta.db_table

Specifying managed=False in a Model Meta class will not create a db table for it. However, it will behave exactly like other models.

Answer 5

If you just want to disable the ForeignKey constraint check on a field, then just add db_constraint=False to that field.

user = models.ForeignKey('User', db_constraint=False)

See also: Django - How to prevent database foreign key constraint creation

Answer 6

Piggybacking off of marianobianchi's comment, one of the options for ForeignKey.on_delete is

DO_NOTHING: Take no action. If your database backend enforces referential integrity, this will cause an IntegrityError unless you manually add a SQL ON DELETE constraint to the database field (perhaps using initial sql).

This combined with disabling foreign key constraints at the db level should do the trick. From what I can tell, there are two ways of doing this. You could disable fk constraints entirely like this:

from django.db.backend.signals import connection_created
from django.dispatch import receiver

@receiver(connection_created)
def disable_constraints(sender, connection):
    connection.disable_constraint_checking()

It looks like the django db backends offer a constraint_checks_disabled context manager, too, so you could wrap the relevant db accesses in code like this to avoid disabling the checks throughout:

from django.db import connection
with connection.constraint_checks_disabled():
    do_stuff()