Django Multilingual Text Field using JSON

Question

I recently ask this question Custom Django MultilingualTextField model field but I found no good reason why I should not do this, so I create a model Field that support multilingual text, auto return text in current language. This basically is the field that store custom Language object to database in json format. Here is the code:

Github: https://github.com/james4388/django-multilingualfield

Ussage:

from django.db import models
from multilingualfield import MLTextField, MLHTMLField

class MyModel(models.Model):
    text = MLTextField()
    html = MLHTMLField()

Used it like normal text field, translation is auto bases on system language (translation.get_language)

>>>from django.utils import translation
>>>translation.active('en')

>>>m = MyModal.objects.create(text='Hello world',html='<b>Hello world</b>');
>>>m.text
Hello world
>>>translation.active('fr')
>>>m.text       #Auto fallback to first language (if any).
Hello world
>>>m.text.value('Bonjour')
>>>m.text.value('Ciao','es')
>>>m.text
Bonjour
>>>m.save()
>>>m.text.get_available_language()
['en', 'fr', 'es']
>>>m.text.remove_language('en')

Field.py

from __future__ import unicode_literals

from django.core.exceptions import ValidationError
from django.conf import settings
from django.db import models, DatabaseError, transaction
from django.utils.translation import ugettext_lazy as _, get_language
from django.utils import six

try:
    import json
except ImportError:
    from django.utils import simplejson as json

def get_base_language(lang):
    if '-' in lang:
        return lang.split('-')[0]
    return lang

def get_current_language(base=True):
    l = get_language()
    if base:
        return get_base_language(l)
    return l

from .widgets import MultilingualWidget, MultilingualHTMLWidget
from .forms import MultilingualTextFormField, MultilingualHTMLFormField
from .language import LanguageText

class MultilingualTextField(six.with_metaclass(models.SubfieldBase, models.Field)):
    """
    A field that support multilingual text for your model
    """
    default_error_messages = {
        'invalid': _("'%s' is not a valid JSON string.")
    }
    description = "Multilingual text field"

    def __init__(self, *args, **kwargs):
        self.lt_max_length = kwargs.pop('max_length',-1)
        self.default_language = kwargs.get('default_language', get_current_language())
        super(MultilingualTextField, self).__init__(*args, **kwargs)


    def formfield(self, **kwargs):
        defaults = {
            'form_class': MultilingualTextFormField,
            'widget': MultilingualWidget
        }
        defaults.update(**kwargs)
        return super(MultilingualTextField, self).formfield(**defaults)

    def validate(self, value, model_instance):
        if not self.null and value is None:
            raise ValidationError(self.error_messages['null'])
        try:
            self.get_prep_value(value)
        except:
            raise ValidationError(self.error_messages['invalid'] % value)

    def get_internal_type(self):
        return 'TextField'


    def db_type(self, connection):
        return 'text'

    def to_python(self, value):
        if isinstance(value, six.string_types):
            if value == "" or value is None:
                if self.null:
                    return None
                if self.blank:
                    return ""
            try:
                valuejson = json.loads(value)
                Lang = LanguageText(max_length=self.lt_max_length,default_language=self.default_language)
                Lang.values = valuejson
                return Lang
            except ValueError:
                try:
                    Lang = LanguageText(value,language=None,max_length=self.lt_max_length,default_language=self.default_language)
                    return Lang
                except:
                    msg = self.error_messages['invalid'] % value
                    raise ValidationError(msg)
        return value

    def get_db_prep_value(self, value, connection=None, prepared=None):
        return self.get_prep_value(value)

    def get_prep_value(self, value):
        if value is None:
            if not self.null and self.blank:
                return ""
            return None
        if isinstance(value, six.string_types):
            value = LanguageText(value,language=None,max_length=self.lt_max_length,default_language=self.default_language)
        if isinstance(value, LanguageText):
            value.max_length = self.lt_max_length
            value.default_language = self.default_language
            return json.dumps(value.values)
        return None

    def get_prep_lookup(self, lookup_type, value):
        if lookup_type in ["exact", "iexact"]:
            return self.to_python(self.get_prep_value(value))
        if lookup_type == "in":
            return [self.to_python(self.get_prep_value(v)) for v in value]
        if lookup_type == "isnull":
            return value
        if lookup_type in ["contains", "icontains"]:
            if isinstance(value, (list, tuple)):
                raise TypeError("Lookup type %r not supported with argument of %s" % (
                    lookup_type, type(value).__name__
                ))
                # Need a way co combine the values with '%', but don't escape that.
                return self.get_prep_value(value)[1:-1].replace(', ', r'%')
            if isinstance(value, dict):
                return self.get_prep_value(value)[1:-1]
            return self.to_python(self.get_prep_value(value))
        raise TypeError('Lookup type %r not supported' % lookup_type)

    def value_to_string(self, obj):
        return self._get_val_from_obj(obj)

Forms.py

from django import forms
from django.utils import simplejson as json

from .widgets import MultilingualWidget, MultilingualHTMLWidget
from .language import LanguageText

class MultilingualTextFormField(forms.CharField):

    widget = MultilingualWidget

    def __init__(self, *args, **kwargs):
        kwargs['widget'] = MultilingualWidget
        super(MultilingualTextFormField, self).__init__(*args, **kwargs)

    def clean(self, value):
        """
        The default is to have a TextField, and we will decode the string
        that comes back from this. However, another use of this field is
        to store a list of values, and use these in a MultipleSelect
        widget. So, if we have an object that isn't a string, then for now
        we will assume that is where it has come from.
        """
        value = super(MultilingualTextFormField, self).clean(value)
        if not value:
            return value
        if isinstance(value, basestring):
            try:
                valuejson = json.loads(value)
                Lang = LanguageText()
                Lang.values = valuejson
                return Lang
            except ValueError:
                try:
                    Lang = LanguageText(value,language=None)
                    return Lang
                except:
                    raise forms.ValidationError(
                        'JSON decode error: %s' % (unicode(exc),)
                    )
        else:
            return value

Language object in language.py

from __future__ import unicode_literals

from django.core.exceptions import ValidationError
from django.conf import settings
from django.db import models, DatabaseError, transaction
from django.utils.translation import ugettext_lazy as _, get_language

try:
    import json
except ImportError:
    from django.utils import simplejson as json

def get_base_language(lang):
    if '-' in lang:
        return lang.split('-')[0]
    return lang

def get_current_language(base=True):
    l = get_language()
    if base:
        return get_base_language(l)
    return l


class LanguageText(object):
    '''
        JSON text field blah blah blah
    '''
    values = {}
    default_language = None
    max_length = -1

    def __init__(self, value=None, language=None, default_language=None, max_length=-1):
        self.max_length = max_length
        self.default_language = default_language
        self.values = {}
        if value is not None:
            self.value(value,language)

    def __call__(self, value=None, language=None):
        self.value(value,language)
        return self

    def get_available_language(self):
        return self.values.keys()

    def get_current_language(self, base=False):
        return get_current_language(base)

    def remove_language(self, lang):
        try:
            return self.values.pop(lang)
        except:
            pass

    def has_language(self, lang):
        return self.values.has_key(lang)

    def get(self, language=None, fallback=True):
        if language is None:
            curr_lang = get_current_language(False)
        else:
            curr_lang = language
        curr_lang_base = get_current_language(True)
        if curr_lang in self.values:
            return self.values[curr_lang]
        if not fallback:
            return None
        if curr_lang_base in self.values:
            return self.values[curr_lang_base]
        if self.default_language in self.values:
            return self.values[self.default_language]
        try:
            first_lang = self.values.keys()[0]
            return self.values[first_lang]
        except:
            pass
        return None

    def value(self, value=None, language=None):
        if value is None:   #Get value
            return self.get(language)
        else: #Set value
            if language is None:
                language = get_current_language(False)
            if self.max_length != -1:
                value = value[:self.max_length]
            self.values[language] = value
            return None

    def __unicode__(self):
        return self.value()

    def __str__(self):
        return unicode(self.value()).encode('utf-8')

    def __repr__(self):
        return unicode(self.value()).encode('utf-8')

widgets.py

from django import forms
from django.utils import simplejson as json
from django.conf import settings
from .language import LanguageText
from django.template import loader, Context

class MultilingualWidget(forms.Textarea):

    def __init__(self, *args, **kwargs):
        forms.Widget.__init__(self, *args, **kwargs)

    def render(self, name, value, attrs=None):
        if value is None: #New create or edit none
            vjson = '{}'
            aLang = []
            Lang = '[]'
            Langs = json.dumps(dict(settings.LANGUAGES))
            t = loader.get_template('multilingualtextarea.html')
            c = Context({"data":value,"vjson":vjson,"lang":Lang,"langs":Langs,"langobjs":settings.LANGUAGES,"fieldname":name})
            return t.render(c)
        if isinstance(value, LanguageText):
            vjson = json.dumps(value.values)
            aLang = value.get_available_language()
            Lang = json.dumps(aLang)
            Langs = json.dumps(dict(settings.LANGUAGES))
            t = loader.get_template('multilingualtextarea.html')
            c = Context({"data":value,"vjson":vjson,"lang":Lang,"langs":Langs,"langobjs":settings.LANGUAGES,"fieldname":name})
            return t.render(c)
        return "Invalid data '%s'" % value

So I would like to know is this a good approach? Why shouldn't I do this? Plz help

Answer 1

Code looks good to me.

The only thing that could impact performance is the frequent json encoding/decoding... yet, that shouldn't have a major impact unless you are facing thousands of users on a server with minimal resources.

The previous question you linked to contains some comments noting that adding additional languages might be easier using other means. But in the end - that's a mixture between personal preferences and maintainability. If it fits your project goals, I can't see any reason not to do it the way you've coded it.

Providing proof that your implementation is the best is near to impossible. That is, unless you prove it yourself by creating a different, non-json based implementation and benchmark both on your production server. You'll notice differences will be rather minimal on regular machines. Yet, only the individual numbers will provide actual proof and can help you decide if it's "tuned" and "resource -friendly" enough for your project's purposes. I think it will fit your needs... but that's only my 2 cents.