How to lock a directory between python processes i

I have two (or more) python processes running and want to create a concept similar to an exclusion mutex for a shared resource. The 'shared resource' in this case is a directory. How might I most easily/standardly/etc implement a mutex? A hidden .lock file that each process agrees to check and, if exists, appends their PID as a new row and then pops their PID when they have access to the file?

I basically just want to clear a directory and make sure no other process tries to read or write to it while I'm clearing it.

Is there a standard linux way of doing this? Maybe something I can just execute with a shell line from python?

Linux

There are two standard types of locking in Linux: advisory locking (specified in POSIX) and mandatory locking (Linux-specific).

However, both of them may be applied only on files, but not directories. So yes, you need a lock-file. It assumes that all users should know about the lock-file and acquire the lock before accessing directory. Hence, mandatory locking will not help here, and you need advisory locking.

There are three kinds of advisory file locks in Linux:

flock(2) (specified in POSIX);
POSIX record locks, see "Advisory record locking" section in fcntl(2) and also lockf(3) wrapper (both specified in POSIX);
Open file description locks, see fcntl(2) (Linux-specific, available in recent kernels).

Python

In Python, flock(), lockf() and fcntl() functions are available through fcntl module. There is also flock module that adds context-manager support to fcntl.flock function.

Here is an example:

import flock

with open('/my/dir/lockfile', 'w') as fp:
    with flock.Flock(fp, flock.LOCK_EX) as lock:
        pass # exclusive lock is acquired here

PS.

With this approach, you can not prevent a random process from accessing your directory, if it don't know about your lock-file. It is probably possible to implement a filesystem using FUSE that will support mandatory directory locks, but I'm not aware of such implementations.