how to monitor a complete directory tree for chang

问题:

How can I monitor a whole directory tree for changes in Linux (ext3 file system)?

Currently the directory contains about half a million files in about 3,000 subdirectories, organized in three directory levels.

Those are mostly small files (< 1kb, some few up to 100 kb). It's a sort of queue and I need to know when files are being created, deleted or their content modified within 5-10 seconds of that happening.

I know there is inotify and sorts, but AFAIK they only monitor a single directory, which means I would need 3,000 inotify handles in my case - more than the usual 1024 handles allowed for a single process. Or am I wrong?

In case the Linux system can't tell me what I need: perhaps there is a FUSE project that simulates a file system (replicating all file accesses on a real file system) and separately logs all modifications (couldn't fine one)?

回答1:

To my knowledge, there's no other way than recursively setting an inotify watch on each directory.

That said, you won't run out of file descriptors because inotify does not have to reserve an fd to watch a file or a directory (its predecessor, dnotify, did suffer from this limitation). inotify uses "watch descriptors" instead.

According to the documentation for inotifywatch, the default limit is 8192 watch descriptors, and you can increase it by writing the new value to /proc/sys/fs/inotify/max_user_watches.

回答2:

I've done something similar using the inotifywait tool:

#!/bin/bash
while true; do

inotifywait -e modify,create,delete -r /path/to/your/dir && \
<some command to execute when a file event is recorded>

done

This will setup recursive directory watches on the entire tree and allow you to execute a command when something changes. If you just want to view the changes, you can add the -m flag to put it into monitor mode.

回答3:

$ inotifywait -m -r /path/to/your/directory

This command is enough to watch the directory recursively for all events such as access, open, create, delete ...

回答4:

Wasn't fanotify supposed to provide that capability eventually? Quoting LWN:

“fanotify has two basic 'modes' directed and global. [...] fanotify global instead indicates that it wants everything on the system and then individually marks inodes that it doesn't care about.”

I lost track what its latest status was, though.

回答5:

Use inotifywait from inotify-tools:

sudo apt install inotify-tools

Now create a script myscript.sh that includes hidden files and folders too:

#!/bin/bash
while true; do

inotifywait -e modify,create,delete,move -r $1

done

Make the script executable with chmod +x myscript.sh

Run it with ./myscript.sh /folder/to/monitor

If you don't provide argument it will use the working directory by default.

Also, you can run several commands adding && \ at the end of the previous command to add the next one:

#!/bin/bash
while true; do

inotifywait -e modify,create,delete,move -r $1 && \
echo "event" && \
echo "event 2"

done

If you don't want to execute any command on events, just run the command directly with the -m modifier so doesn't close:

inotifywait -e modify,create,delete,move -m -r /path/to/your/dir

回答6:

inotify is the best option when you have many subdirectories but if not I am used to using this command below:

watch -d find <<path>>