zc.lockfile.LockError in ZODB

Question

I am trying to use ZODB 3.10.2 on my web server which is running Debian and Python 2.7.1. It seems like every time I try to access the same database from 2 different processes, I get a mysterious exception. I tried accessing a database from an interactive Python session and everything seemed to work fine:

>>> import ZODB
>>> from ZODB.FileStorage import FileStorage
>>> storage = FileStorage("test.db")
>>> 

But then I tried the same series of commands from another session running at the same time and it didn't seem to work:

>>> import ZODB
>>> from ZODB.FileStorage import FileStorage
>>> storage = FileStorage("test.db")
    No handlers could be found for logger "zc.lockfile"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/ZODB3-3.10.2-py2.7-linux-x86_64.egg/ZODB/FileStorage/FileStorage.py", line 125, in __init__
    self._lock_file = LockFile(file_name + '.lock')
  File "/usr/local/lib/python2.7/site-packages/zc.lockfile-1.0.0-py2.7.egg/zc/lockfile/__init__.py", line 76, in __init__
    _lock_file(fp)
  File "/usr/local/lib/python2.7/site-packages/zc.lockfile-1.0.0-py2.7.egg/zc/lockfile/__init__.py", line 59, in _lock_file
    raise LockError("Couldn't lock %r" % file.name)
zc.lockfile.LockError: Couldn't lock 'test.db.lock'
>>>

Why is this happening? What can be done about it?

Answer 1

You can not access the same database files from two processes at the same time (which is obvious). That's why you get this error. If you need to perform actions on the same data.fs file from two or more processes: use ZEO.

Answer 2

The ZODB does not support multi-process access. This is why you get the lock error; the ZODB file storage has been locked by one process to prevent other processes altering it.

There are several ways around this. The easiest option is to use ZEO. ZEO extends the ZODB machinery to provide access to objects over a network, and you can easily configure your ZODB to access a ZEO server instead of a local FileStorage file:

<zodb>
    <zeoclient>
    server localhost:9100
    </zeoclient>
</zodb>

Another option is to use RelStorage, which stores the ZODB data in a relational database. RelStorage supports PostgreSQL, Oracle and MySQL backends. RelStorage takes care of concurrent access from different ZODB clients. Here is an example configuration:

<zodb>
  <relstorage>
    <postgresql>
      # The dsn is optional, as are each of the parameters in the dsn.
      dsn dbname='zodb' user='username' host='localhost' password='pass'
    </postgresql>
  </relstorage>
</zodb>

RelStorage requires more up-front setup work but can outperform ZEO in many scenarios.