Duplicating model instances and their related obje

Question

I've models for Books, Chapters and Pages. They are all written by a User:

from django.db import models

class Book(models.Model)
    author = models.ForeignKey('auth.User')

class Chapter(models.Model)
    author = models.ForeignKey('auth.User')
    book = models.ForeignKey(Book)

class Page(models.Model)
    author = models.ForeignKey('auth.User')
    book = models.ForeignKey(Book)
    chapter = models.ForeignKey(Chapter)

What I'd like to do is duplicate an existing Book and update it's User to someone else. The wrinkle is I would also like to duplicate all related model instances to the Book - all it's Chapters and Pages as well!

Things get really tricky when look at a Page - not only will the new Pages need to have their author field updated but they will also need to point to the new Chapter objects!

Does Django support an out of the box way of doing this? What would a generic algorithm for duplicating a model look like?

Cheers,

John

---

Update:

The classes given above are just an example to illustrate the problem I'm having!

Answer 1

I haven't tried it in django but python's deepcopy might just work for you

EDIT:

You can define custom copy behavior for your models if you implement functions:

__copy__() and __deepcopy__()

Answer 2

If there's just a couple copies in the database you're building, I've found you can just use the back button in the admin interface, change the necessary fields and save the instance again. This has worked for me in cases where, for instance, I need to build a "gimlet" and a "vodka gimlet" cocktail where the only difference is replacing the name and an ingredient. Obviously, this requires a little foresight of the data and isn't as powerful as overriding django's copy/deepcopy - but it may do the trick for some.

Answer 3

Django does have a built-in way to duplicate an object via the admin - as answered here: In the Django admin interface, is there a way to duplicate an item?

Answer 4

I had no luck with any of the answers here with Django 2.1.2, so I created a generic way of performing a deep copy of a database model that is heavily based on the answers posted above.

The key differences from the answers above is that ForeignKey no longer has an attribute called rel, so it has to be changed to f.remote_field.model etc.

Furthermore, because of the difficulty of knowing the order the database models should be copied in, I created a simple queuing system that pushes the current model to the end of the list if it is unsuccessfully copied. The code is postet below:

import queue
from django.contrib.admin.utils import NestedObjects
from django.db.models.fields.related import ForeignKey

def duplicate(obj, field=None, value=None, max_retries=5):
    # Use the Nested Objects collector to retrieve the related models
    collector = NestedObjects(using='default')
    collector.collect([obj])
    related_models = list(collector.data.keys())

    # Create an object to map old primary keys to new ones
    data_snapshot = {}
    model_queue = queue.Queue()
    for key in related_models:
        data_snapshot.update(
            {key: {item.pk: None for item in collector.data[key]}}
        )
        model_queue.put(key)

    # For each of the models in related models copy their instances
    root_obj = None
    attempt_count = 0
    while not model_queue.empty():
        model = model_queue.get()
        root_obj, success = copy_instances(model, related_models, collector, data_snapshot, root_obj)

        # If the copy is not a success, it probably means that not
        # all the related fields for the model has been copied yet.
        # The current model is therefore pushed to the end of the list to be copied last
        if not success:

            # If the last model is unsuccessful or the number of max retries is reached, raise an error
            if model_queue.empty() or attempt_count > max_retries:
                raise DuplicationError(model)
            model_queue.put(model)
            attempt_count += 1
    return root_obj

def copy_instances(model, related_models, collector, data_snapshot, root_obj):

# Store all foreign keys for the model in a list
fks = []
for f in model._meta.fields:
    if isinstance(f, ForeignKey) and f.remote_field.model in related_models:
        fks.append(f)

# Iterate over the instances of the model
for obj in collector.data[model]:

    # For each of the models foreign keys check if the related object has been copied
    # and if so, assign its personal key to the current objects related field
    for fk in fks:
        pk_field = f"{fk.name}_id"
        fk_value = getattr(obj, pk_field)

        # Fetch the dictionary containing the old ids
        fk_rel_to = data_snapshot[fk.remote_field.model]

        # If the value exists and is in the dictionary assign it to the object
        if fk_value is not None and fk_value in fk_rel_to:
            dupe_pk = fk_rel_to[fk_value]

            # If the desired pk is none it means that the related object has not been copied yet
            # so the function returns unsuccessful
            if dupe_pk is None:
                return root_obj, False

            setattr(obj, pk_field, dupe_pk)

    # Store the old pk and save the object without an id to create a shallow copy of the object
    old_pk = obj.id
    obj.id = None

    if field is not None:
        setattr(obj, field, value)

    obj.save()

    # Store the new id in the data snapshot object for potential use on later objects
    data_snapshot[model][old_pk] = obj.id

    if root_obj is None:
        root_obj = obj

return root_obj, True

I hope it is of any help :)

The duplication error is just a simple exception extension:

class DuplicationError(Exception):
    """
    Is raised when a duplication operation did not succeed

    Attributes:
        model -- The database model that failed
    """

    def __init__(self, model):
        self.error_model = model

    def __str__(self):
        return f'Was not able to duplicate database objects for model {self.error_model}'

Answer 5

Using the CollectedObjects snippet above no longer works but can be done with the following modification:

from django.contrib.admin.util import NestedObjects
from django.db import DEFAULT_DB_ALIAS

and

collector = NestedObjects(using=DEFAULT_DB_ALIAS)

instead of CollectorObjects

Answer 6

I think you'd be happier with a simpler data model, also.

Is it really true that a Page is in some Chapter but a different book?

userMe = User( username="me" )
userYou= User( username="you" )
bookMyA = Book( userMe )
bookYourB = Book( userYou )

chapterA1 = Chapter( book= bookMyA, author=userYou ) # "me" owns the Book, "you" owns the chapter?

chapterB2 = Chapter( book= bookYourB, author=userMe ) # "you" owns the book, "me" owns the chapter?

page1 = Page( book= bookMyA, chapter= chapterB2, author=userMe ) # Book and Author aggree, chapter doesn't?

It seems like your model is too complex.

I think you'd be happier with something simpler. I'm just guessing at this, since I don't your know entire problem.

class Book(models.Model)
    name = models.CharField(...)

class Chapter(models.Model)
    name = models.CharField(...)
    book = models.ForeignKey(Book)

class Page(models.Model)
    author = models.ForeignKey('auth.User')
    chapter = models.ForeignKey(Chapter)

Each page has distinct authorship. Each chapter, then, has a collection of authors, as does the book. Now you can duplicate Book, Chapter and Pages, assigning the cloned Pages to the new Author.

Indeed, you might want to have a many-to-many relationship between Page and Chapter, allowing you to have multiple copies of just the Page, without cloning book and Chapter.