Import csv data into database in Django Admin

Question

I've tried to import a csv file into a database by tweaking the modelform inside the admin doing this:

models.py:

class Data(models.Model):
    place = models.ForeignKey(Places)
    time = models.DateTimeField()
    data_1 = models.DecimalField(max_digits=3, decimal_places=1)
    data_2 = models.DecimalField(max_digits=3, decimal_places=1)
    data_3 = models.DecimalField(max_digits=4, decimal_places=1)

Forms.py:

import csv
class DataImport(ModelForm):
    file_to_import = forms.FileField()

    class Meta:
        model = Data
        fields = ("file_to_import", "place")

    def save(self, commit=False, *args, **kwargs):
        form_input = DataImport()
        self.place = self.cleaned_data['place']
        file_csv = request.FILES['file_to_import']
        datafile = open(file_csv, 'rb')
        records = csv.reader(datafile)
        for line in records:
            self.time = line[1]
            self.data_1 = line[2]
            self.data_2 = line[3]
            self.data_3 = line[4]
            form_input.save()
        datafile.close()

Admin.py:

class DataAdmin(admin.ModelAdmin):
    list_display = ("place", "time")
    form = DataImport

admin.site.register(Data, DataAdmin)

But i'm stuck trying to import the file i put in "file_to_import" field. Getting AttributeError in forms.py : 'function' object has no attribute 'FILES'.

What i'm doing wrong?

Answer 1

In the save() method, you don't have any access to the request object - you can see that it's not passed in. Normally you would expect to have a NameError there, but I suspect that you've got a function elsewhere in the file called request().

At the point of saving, all the relevant data should be in cleaned_data: so you should be able to do

file_csv = self.cleaned_data['file_to_import']

At that point you'll have another problem, which is when you get to open - you can't do that, as file_to_import is not a file on the server filesystem, it's an in-memory file that has been streamed from the client. You should be able to pass file_csv directly to csv.reader.

Answer 2

After a long search i found an answer: Create a view inside the admin using a standard form

Form:

class DataInput(forms.Form):
    file = forms.FileField()
    place = forms.ModelChoiceField(queryset=Place.objects.all())

    def save(self):
        records = csv.reader(self.cleaned_data["file"])
        for line in records:
            input_data = Data()
            input_data.place = self.cleaned_data["place"]
            input_data.time = datetime.strptime(line[1], "%m/%d/%y %H:%M:%S")
            input_data.data_1 = line[2]
            input_data.data_2 = line[3]
            input_data.data_3 = line[4]
            input_data.save()

The view:

@staff_member_required
def import(request):
    if request.method == "POST":
        form = DataInput(request.POST, request.FILES)
        if form.is_valid():
            form.save()
            success = True
            context = {"form": form, "success": success}
            return render_to_response("imported.html", context,
            context_instance=RequestContext(request))
    else:
        form = DataInput()        
        context = {"form": form}
        return render_to_response("imported.html", context,
        context_instance=RequestContext(request)) 

The rest is part of this post: http://web.archive.org/web/20100605043304/http://www.beardygeek.com/2010/03/adding-views-to-the-django-admin/

Answer 3

Take a look at django-admin-import, it does more or less exactly what you want -- you can upload a XLS (not a CSV, but that should not matter) and lets you assign columns to model fields. Default values are also supported.

http://pypi.python.org/pypi/django-admin-import/0.2.1

Additionally, it does not take away the possibility to modify individual records by hand because you don't have to replace the default model form used in the administration.