Django rest framework M2M serializer save nested o

Question

I have two models: Task and Scenario. Tasks are created beforehand and Scenarios are created afterwards by including few of the existing tasks. Important thing is in a scenario, tasks must be arranged in a specific sequence and not according to their ids.

class Task(models.Model):
    stakeholder = models.ForeignKey(User, related_name='tasks', blank=True, )
    project = models.ForeignKey(Project, related_name='project_tasks' )
    title = models.CharField(max_length=50, blank=True, null = True, )
    ...

class Scenario(models.Model):
    stakeholder = models.ForeignKey(User, related_name='scenarios', blank=True,)
    tasks = models.ManyToManyField(Task, blank=True)

Serializers:

class TaskSerializer(serializers.ModelSerializer):
    id = serializers.IntegerField()
    class Meta:
        model = Task
        fields = '__all__'

class ScenarioSerializer(serializers.ModelSerializer):
    tasks = TaskSerializer(many=True, required=False)
    class Meta:
        model = Scenario
        fields = '__all__'

    def get_or_create_task(self, data):
        qs  = Task.objects.filter(pk=data.get('id'))
        if qs.exists():
            return qs.first()
        task = Task.objects.create(**data)
        return task

    def add_tasks(self, instance, tasks):
        for task_data in tasks:
            task = self.get_or_create_task(task_data)
            instance.tasks.add(task)

    def create(self, validated_data):
        tasks = validated_data.pop('tasks')
        instance = Scenario.objects.create(**validated_data)
        self.add_tasks(instance, tasks)
        return instance

    def update(self, instance, validated_data):
        tasks = validated_data.pop('tasks', [])
        instance = super().update(instance, validated_data)
        self.add_tasks(instance, tasks)
        return instance

Few requirements I have:

While retrieving a scenario/s I wanted to retrieve corresponding task objects and not just their ids, that's why the line id = serializers.IntegerField() exists in TaskSerializer.

With this code a new Task is successfully created, but Task update fails at e.g./api/tasks/1 as it also expects id in the request body. Without id = serializers.IntegerField() both task creation and update is successful.

Also, without id = serializers.IntegerField() in TaskSerializer Scenario creation is successful and included tasks are returned in the correct order, say task3, task1, Task5. However, whichever tasks are included while scenario creation, those are created again in the database (of course with successive ids).

Uncommenting id = serializers.IntegerField() leads to successful scenario creation and no extra tasks are automatically created (which is good), but the scenario returns tasks in the oder of their ids i.e. if a scenario was created with Task3, task1, task5 then when you GET a scenario you get back task1, task3, task5.

Why is it happening?

Extra Note: I do not intend to create a task while scenario creation, tasks will always be created beforehand, and scenario will be created afterwards by attaching a few existing tasks to it

Answer 1

you can try to set tasks as read_only and get data from context, and to save the created order you may use extra select

class ScenarioSerializer(serializers.ModelSerializer):
    tasks = serializers.SerializerMethodField()

    class Meta:
        model = Scenario
        fields = '__all__'

    def get_tasks(self, obj):
        qs = obj.tasks.extra(
            select={'creation_seq': 'scenario_scenario_tasks.id'}
            ).order_by("creation_seq")
        return TaskSerializer(qs, many=True).data

    def get_or_create_task(self, data):
        qs = Task.objects.filter(pk=data.get('id'))
        if qs.exists():
            return qs.first()
        serializer = TaskSerializer(data=data)
        serializer.is_valid(raise_exception=True)
        task = serializer.save()
        return task

    def add_tasks(self, instance, data):
        tasks = data.get('tasks', [])
        for task_data in tasks:
            task = self.get_or_create_task(task_data)
            instance.tasks.add(task)

    def create(self, validated_data):
        instance = super().create(validated_data)
        # for python 2
        # instance = super(ScenarioSerializer, self).create(validated_data)
        data = self.context.get('request').data
        #      HERE ^^^^^^^^^^
        self.add_tasks(instance, data)
        return instance

    def update(self, instance, validated_data):
        instance = super().update(instance, validated_data)
        # for python 2
        # instance = super(ScenarioSerializer, self).update(instance, validated_data)
        data = self.context.get('request').data
        #      HERE ^^^^^^^^^^
        self.add_tasks(instance, data)
        return instance

and here drfdoc-demo you can look on full example with class based views for drfdoc