Django API list with pagination - Page is not JSON

Question

I am trying to create a Django API for a list with pagination but have this error

TypeError: Object of type 'Page' is not JSON serializable

The following is my API code:

@api_view(['POST'])
def employee_get_list_by_page(request):
    # ----- YAML below for Swagger -----
    """
    description: employee_get_list_by_page
    parameters:
      - name: token
        type: string
        required: true
        location: form       
      - name: page
        type: string
        required: true
        location: form   
      - name: page_limit
        type: string
        required: true
        location: form                   
    """
    token = request.POST['token']
    try:  
        auth_employee = AuthEmployeeSessionToken.objects.get(token=token) 

    except AuthEmployeeSessionToken.DoesNotExist:   
        return Response("Invalid Token", status=status.HTTP_406_NOT_ACCEPTABLE)

    employee_list = Employee.objects.filter(company = auth_employee.employee.company.id) 


    page = request.GET.get('page', request.POST['page'])
    paginator = Paginator(employee_list, request.POST['page_limit'])

    try:
        employees = paginator.page(page)
    except PageNotAnInteger:
        employees = paginator.page(request.POST['page'])
    except EmptyPage:
        employees = paginator.page(paginator.num_pages)

    return Response(employees,status=status.HTTP_200_OK) <-- passing employees probably cause this error that employees as Page is not JSON serializable.

This is my model https://gist.github.com/axilaris/89b2ac6a7762f428ad715f4916f43967. as_dict notice I have this .as_dict() to create my json for response. I use it for single request but for list Im not sure how it is done.

1. How do an API query for this list with pagination support ?
2. Side issue which I want the JSON to list just the fields that I want.

Answer 1

I just had a go at recreating this, and I see a few ways to fix it.

Firstly, JSON will not be able to parse either the Page object, nor the QuerySet underlying the page.object_list property. It will say "Object of type 'Employee' is not JSON serialisable".

So to solve this, I'd try:

employee_list = Employee.objects.filter(company = auth_employee.employee.company.id).values().order_by('id') 

page = request.GET.get('page', request.POST['page'])
paginator = Paginator(employee_list, request.POST['page_limit'])

try:
    employees = paginator.page(page)
except PageNotAnInteger:
    employees = paginator.page(request.POST['page'])
except EmptyPage:
    employees = paginator.page(paginator.num_pages)

return Response(list(employees) ,status=status.HTTP_200_OK)

Firstly, we use .values() on the employees queryset because the resulting ValuesQuerySet of this call can be parsed with list(employees). Inside the Page class, they evaluate the object list inside the instance this way before returning any results.

Lastly, because JSON can't serialise the Page class, we simply call list(Page) to return a list. This works because Page implements __getitem__ and returns the underlying object_list.

Also, you may find that some datatypes will throw JSON serialization errors (values() will return DateTime objects for DateTime fields). In my testing I had an issue with Object of type 'datetime' is not JSON serializable. If this happens, you need to use a different JSON Encoder or extend your own. The DjangoJSONEncoder can be found in django.core.serializers.json.DjangoJSONEncoder and handles datetimes, uuids, decimals, and other common datatypes in django.

Edit:: You've mentioned your model code as:

class Employee(models.Model):

    user = models.OneToOneField(User, on_delete=models.CASCADE, related_name='employee')
    company = models.ForeignKey(Company)
    username = models.CharField(max_length=30, blank=False)
    email = models.CharField(max_length=30, blank=False)
    first_name = models.CharField(max_length=30, blank=False)
    last_name = models.CharField(max_length=30, blank=False)

    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    def __str__(self):
        return self.user.username

    def as_dict(self):

        return {"id": "%d" % self.id,
                "company": self.company.as_dict(),
                "username": self.username if self.username else "",
                "email": self.email if self.email else "",
                "first_name": self.first_name if self.first_name else "",
                "last_name": self.last_name if self.last_name else "",
                "tel":self.tel if self.tel else "",               
                "created_at":self.created_at.strftime('%Y-%m-%d %H:%M'),
                "updated_at":self.updated_at.strftime('%Y-%m-%d %H:%M')}

Because you have this as_dict method, we could use this to render the representation of your employees instead of relying on .values(). Here is how:

employee_list = Employee.objects.filter(company = auth_employee.employee.company.id).order_by('id') 

page = request.GET.get('page', request.POST['page'])
paginator = Paginator(employee_list, request.POST['page_limit'])

try:
    employees = paginator.page(page)
except PageNotAnInteger:
    employees = paginator.page(request.POST['page'])
except EmptyPage:
    employees = paginator.page(paginator.num_pages)
#  Here we map a lambda function over the queryset of Models to return the dictionary representation for each element in the list
employees_page = list(
    map(lambda employee: employee.as_dict(), list(employees))
)
return Response(employees_page ,status=status.HTTP_200_OK)