Move models between Django (1.8) apps with require

Question

This is an extension to this question: How to move a model between two Django apps (Django 1.7)

I need to move a bunch of models from old_app to new_app. The best answer seems to be Ozan's, but with required foreign key references, things are bit trickier. @halfnibble presents a solution in the comments to Ozan's answer, but I'm still having trouble with the precise order of steps (e.g. when do I copy the models over to new_app, when do I delete the models from old_app, which migrations will sit in old_app.migrations vs. new_app.migrations, etc.)

Any help is much appreciated!

Answer 1

After work was done I tried to make new migration. But I facing with following error: ValueError: Unhandled pending operations for models: oldapp.modelname (referred to by fields: oldapp.HistoricalProductModelName.model_ref_obj)

If your Django model using HistoricalRecords field don't forget add additinal models/tables while following @Nostalg.io answer.

Add following item to database_operations at the first step (4.a):

    migrations.AlterModelTable('historicalmodelname', 'newapp_historicalmodelname'),

and add additional Delete into state_operations at the last step (4.d):

    migrations.DeleteModel(name='HistoricalModleName'),

Answer 2

In case you need to move the model and you don't have access to the app anymore (or you don't want the access), you can create a new Operation and consider to create a new model only if the migrated model does not exist.

In this example I am passing 'MyModel' from old_app to myapp.

class MigrateOrCreateTable(migrations.CreateModel):
    def __init__(self, source_table, dst_table, *args, **kwargs):
        super(MigrateOrCreateTable, self).__init__(*args, **kwargs)
        self.source_table = source_table
        self.dst_table = dst_table

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        table_exists = self.source_table in schema_editor.connection.introspection.table_names()
        if table_exists:
            with schema_editor.connection.cursor() as cursor:
                cursor.execute("RENAME TABLE {} TO {};".format(self.source_table, self.dst_table))
        else:
            return super(MigrateOrCreateTable, self).database_forwards(app_label, schema_editor, from_state, to_state)


class Migration(migrations.Migration):

    dependencies = [
        ('myapp', '0002_some_migration'),
    ]

    operations = [
        MigrateOrCreateTable(
            source_table='old_app_mymodel',
            dst_table='myapp_mymodel',
            name='MyModel',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('name', models.CharField(max_length=18))
            ],
        ),
    ]

Answer 3

I've built a management command to do just that - move a model from one Django app to another - based on nostalgic.io's suggestions at https://stackoverflow.com/a/30613732/1639699

You can find it on GitHub at alexei/django-move-model

Answer 4

Migrating a model between apps.

The short answer is, don't do it!!

But that answer rarely works in the real world of living projects and production databases. Therefore, I have created a sample GitHub repo to demonstrate this rather complicated process.

I am using MySQL. (No, those aren't my real credentials).

The Problem

The example I'm using is a factory project with a cars app that initially has a Car model and a Tires model.

factory
  |_ cars
    |_ Car
    |_ Tires

The Car model has a ForeignKey relationship with Tires. (As in, you specify the tires via the car model).

However, we soon realize that Tires is going to be a large model with its own views, etc., and therefore we want it in its own app. The desired structure is therefore:

factory
  |_ cars
    |_ Car
  |_ tires
    |_ Tires

And we need to keep the ForeignKey relationship between Car and Tires because too much depends on preserving the data.

The Solution

Step 1. Setup initial app with bad design.

Browse through the code of step 1.

Step 2. Create an admin interface and add a bunch of data containing ForeignKey relationships.

View step 2.

Step 3. Decide to move the Tires model to its own app. Meticulously cut and paste code into the new tires app. Make sure you update the Car model to point to the new tires.Tires model.

Then run ./manage.py makemigrations and backup the database somewhere (just in case this fails horribly).

Finally, run ./manage.py migrate and see the error message of doom,

django.db.utils.IntegrityError: (1217, 'Cannot delete or update a parent row: a foreign key constraint fails')

View code and migrations so far in step 3.

Step 4. The tricky part. The auto-generated migration fails to see that you've merely copied a model to a different app. So, we have to do some things to remedy this.

You can follow along and view the final migrations with comments in step 4. I did test this to verify it works.

First, we are going to work on cars. You have to make a new, empty migration. This migration actually needs to run before the most recently created migration (the one that failed to execute). Therefore, I renumbered the migration I created and changed the dependencies to run my custom migration first and then the last auto-generated migration for the cars app.

You can create an empty migration with:

./manage.py makemigrations --empty cars

Step 4.a. Make custom old_app migration.

In this first custom migration, I'm only going to perform a "database_operations" migration. Django gives you the option to split "state" and "database" operations. You can see how this is done by viewing the code here.

My goal in this first step is to rename the database tables from oldapp_model to newapp_model without messing with Django's state. You have to figure out what Django would have named your database table based on the app name and model name.

Now you are ready to modify the initial tires migration.

Step 4.b. Modify new_app initial migration

The operations are fine, but we only want to modify the "state" and not the database. Why? Because we are keeping the database tables from the cars app. Also, you need to make sure that the previously made custom migration is a dependency of this migration. See the tires migration file.

So, now we have renamed cars.Tires to tires.Tires in the database, and changed the Django state to recognize the tires.Tires table.

Step 4.c. Modify old_app last auto-generated migration.

Going back to cars, we need to modify that last auto-generated migration. It should require our first custom cars migration, and the initial tires migration (that we just modified).

Here we should leave the AlterField operations because the Car model is pointing to a different model (even though it has the same data). However, we need to remove the lines of migration concerning DeleteModel because the cars.Tires model no longer exists. It has fully converted into tires.Tires. View this migration.

Step 4.d. Clean up stale model in old_app.

Last but not least, you need to make a final custom migration in the cars app. Here, we will do a "state" operation only to delete the cars.Tires model. It is state-only because the database table for cars.Tires has already been renamed. This last migration cleans up the remaining Django state.

Answer 5

Nostalg.io's way worked in forwards (auto-generating all other apps FKs referencing it). But i needed also backwards. For this, the backward AlterTable has to happen before any FKs are backwarded (in original it would happen after that). So for this, i split the AlterTable in to 2 separate AlterTableF and AlterTableR, each working only in one direction, then using forward one instead of the original in first custom migration, and reverse one in the last cars migration (both happen in cars app). Something like this:

#cars/migrations/0002...py :

class AlterModelTableF( migrations.AlterModelTable):
    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        print( 'nothing back on', app_label, self.name, self.table)

class Migration(migrations.Migration):                                                         
    dependencies = [
        ('cars', '0001_initial'),
    ]

    database_operations= [
        AlterModelTableF( 'tires', 'tires_tires' ),
        ]
    operations = [
        migrations.SeparateDatabaseAndState( database_operations= database_operations)         
    ]           


#cars/migrations/0004...py :

class AlterModelTableR( migrations.AlterModelTable):
    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        print( 'nothing forw on', app_label, self.name, self.table)
    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        super().database_forwards( app_label, schema_editor, from_state, to_state)

class Migration(migrations.Migration):
    dependencies = [
        ('cars', '0003_auto_20150603_0630'),
    ]

    # This needs to be a state-only operation because the database model was renamed, and no longer exists according to Django.
    state_operations = [
        migrations.DeleteModel(
            name='Tires',
        ),
    ]

    database_operations= [
        AlterModelTableR( 'tires', 'tires_tires' ),
        ]
    operations = [
        # After this state operation, the Django DB state should match the actual database structure.
       migrations.SeparateDatabaseAndState( state_operations=state_operations,
         database_operations=database_operations)
    ]   

Answer 6

Just now moved two models from old_app to new_app, but the FK references were in some models from app_x and app_y, instead of models from old_app.

In this case, follow the steps provided by Nostalg.io like this:

• Move the models from old_app to new_app, then update the import statements across the code base.
• makemigrations.
• Follow Step 4.a. But use AlterModelTable for all moved models. Two for me.
• Follow Step 4.b. as is.
• Follow Step 4.c. But also, for each app that has a newly generated migration file, manually edit them, so you migrate the state_operations instead.
• Follow Step 4.d But use DeleteModel for all moved models.

Notes:

• All the edited auto-generated migration files from other apps have a dependency on the custom migration file from old_app where AlterModelTable is used to rename the table(s). (created in Step 4.a.)
• In my case, I had to remove the auto-generated migration file from old_app because I didn't have any AlterField operations, only DeleteModel and RemoveField operations. Or keep it with empty operations = []

• To avoid migration exceptions when creating the test DB from scratch, make sure the custom migration from old_app created at Step 4.a. has all previous migration dependencies from other apps.

  old_app
    0020_auto_others
    0021_custom_rename_models.py
      dependencies:
        ('old_app', '0020_auto_others'),
        ('app_x', '0002_auto_20170608_1452'),
        ('app_y', '0005_auto_20170608_1452'),
        ('new_app', '0001_initial'),
    0022_auto_maybe_empty_operations.py
      dependencies:
        ('old_app', '0021_custom_rename_models'),
    0023_custom_clean_models.py
      dependencies:
        ('old_app', '0022_auto_maybe_empty_operations'),
  app_x
    0001_initial.py
    0002_auto_20170608_1452.py
    0003_update_fk_state_operations.py
      dependencies
        ('app_x', '0002_auto_20170608_1452'),
        ('old_app', '0021_custom_rename_models'),
  app_y
    0004_auto_others_that_could_use_old_refs.py
    0005_auto_20170608_1452.py
    0006_update_fk_state_operations.py
      dependencies
        ('app_y', '0005_auto_20170608_1452'),
        ('old_app', '0021_custom_rename_models'),
  

BTW: There is an open ticket about this: https://code.djangoproject.com/ticket/24686