Storing a binary hash value in a Django model fiel

Question

I have a twenty byte hex hash that I would like to store in a django model. If I use a text field, it's interpreted as unicode and it comes back garbled.

Currently I'm encoding it and decoding it, which really clutters up the code, because I have to be able to filter by it.

def get_changeset(self):
    return bin(self._changeset)

def set_changeset(self, value):
    self._changeset = hex(value)

changeset = property(get_changeset, set_changeset)

Here's an example for filtering

Change.objects.get(_changeset=hex(ctx.node()))

This is the approach that was recommended by a django developer, but I'm really struggling to come to terms with the fact that it's this ugly to just store twenty bytes.

Maybe I'm too much of a purist, but ideally I would be able to write

Change.objects.get(changeset=ctx.node())

The properties allow me to write:

change.changeset = ctx.node()

So that's as good as I can ask.

Answer 1

I'm assuming if you were writing raw SQL you'd be using a Postgres bytea or a MySQL VARBINARY. There's a ticket with a patch (marked "needs testing") that purportedly makes a field like this (Ticket 2417: Support for binary type fields (aka: bytea in postgres and VARBINARY in mysql)).

Otherwise, you could probably try your hand at writing a custom field type.

Answer 2

You could also write your own custom Model Manager that does the escaping and unescaping for you.

Answer 3

Starting with 1.6, Django has BinaryField allowing to store raw binary data. However, for hashes and other values up to 128 bits it's more efficient (at least with the PostgreSQL backend) to use UUIDField available in Django 1.8+.

Answer 4

If this issue is still of interest, Disqus' django-bitfield fits the bill:

https://github.com/disqus/django-bitfield

... the example code on GitHub is a little confusing at first w/r/t the modules' actual function, because of the asinine variable names -- generally I am hardly the sort of person with either the wherewithal or the high ground to take someone elses' goofy identifiers to task... but flaggy_foo?? Srsly, U guys.

If that project isn't to your taste, and you're on Postgres, you have a lot of excellent options as many people have written and released code for an assortment of Django fields that take advantage of Postgres' native type. Here's an hstore model field:

https://github.com/jordanm/django-hstore -- I have used this and it works well.

Here's a full-text search implementation that uses Postgres' termvector types:

https://github.com/aino/django-pgindex

And while I cannot vouch for this specific project, there are Django bytea fields as well:

https://github.com/aino/django-arrayfields

Answer 5

"I have a twenty byte hex hash that I would like to store in a django model."

Django does this. They use hex digests, which are -- technically -- strings. Not bytes.

Do not use someHash.digest() -- you get bytes, which you cannot easily store.

Use someHash.hexdigest() -- you get a string, which you can easily store.

Edit -- The code is nearly identical.

See http://docs.python.org/library/hashlib.html