Foreign Keys on Scrapy

Question

im doing an scrap with scrapy and my model on django is:

class Creative(models.Model):
    name = models.CharField(max_length=200)
    picture = models.CharField(max_length=200, null = True)

class Project(models.Model):
    title = models.CharField(max_length=200)
    description = models.CharField(max_length=500, null = True)
    creative = models.ForeignKey(Creative)

class Image(models.Model):
    url = models.CharField(max_length=500)
    project = models.ForeignKey(Project)

And my scrapy model:

from scrapy.contrib.djangoitem import DjangoItem
from app.models import Project, Creative

class ProjectItems(DjangoItem):
    django_model = Project

class CreativeItems(DjangoItem):
    django_model = Creative

So when i save:

creative["name"]  = hxs.select('//*[@id="owner"]/text()').extract()[0]
picture  = hxs.select('//*[@id="owner-icon"]/a/img/@src').extract()
if len(picture)>0:
    creative["picture"] = picture[0]
creative.save()


# Extract title and description of the project
project["title"] = hxs.select('//*[@id="project-title"]/text()').extract()[0]
description = hxs.select('//*[@class="project-description"]/text()').extract()
if len(description)>0:
    project["description"] = description[0]
project["creative"] = creative
project.save()

I got the error:

Project.creative" must be a "Creative" instance.

So, how can i add a foreing key value on scrapy?

Answer 1

Like it's been done here, put the ID of your creative directly in creative_id, it should work I think:

 project["creative_id"] = creative.id

It will specify the foreign key, without bother you with the object missing (because you're in a Scrapy environment where you don't directly touch the model objects...).

Answer 2

This can be done by assigning the return value of the creative.save() to the value at project['creative'] So for instance in the following example we use the djangoCreativeItem variable to pass this information to the project:

creative["name"]  = hxs.select('//*[@id="owner"]/text()').extract()[0]
picture  = hxs.select('//*[@id="owner-icon"]/a/img/@src').extract()   
if len(picture)>0:
    creative["picture"] = picture[0]
djangoCreativeItem = creative.save()

# Extract title and description of the project
project["title"] = hxs.select('//*[@id="project-title"]/text()').extract()[0]
description = hxs.select('//*[@class="project-description"]/text()').extract()
if len(description)>0:
    project["description"] = description[0]
project["creative"] = djangoCreativeItem
project.save()