How to purge disk I/O caches on Linux

Question

I need to do it for more predictable benchmarking.

Answer 1

Short good enough answer: (copy paste friendly)

DISK=/dev/sdX # <===ADJUST THIS===
sync
echo 3 > /proc/sys/vm/drop_caches
blockdev --flushbufs $DISK
hdparm -F $DISK

Explanation:

sync: From the man page: flush file system buffers. Force changed blocks to disk, update the super block.

echo 3 > /proc/sys/vm/drop_cache: from the kernel docs this will cause the kernel to drop clean caches

blockdev --flushbufs /dev/sda: from the man page: call block device ioctls [to] flush buffers.

hdparm -F /dev/sda: from the man page: Flush the on-drive write cache buffer (older drives may not implement this)

Although the blockdev and hdparm commands look similar according to an answer above they issue different ioctls to the device.

Long probably better way:

(I'll assume that you have formatted the disk but you can adapt these commands if you want to write directly to the disk)

Run this only once before the 1st benchmark:

MOUNT=/mnt/test # <===ADJUST THIS===
dd if=/dev/urandom of=$MOUNT/temp-hddread.tmp bs=64M count=16

Run this every time you want to empty the caches:

DISK=/dev/sdX # <===ADJUST THIS===
MOUNT=/mnt/test # <===AND THIS===
dd if=/dev/urandom of=$MOUNT/temp-hddwrite.tmp bs=64M count=16
rm $MOUNT/temp-hddwrite.tmp
sync
echo 3 > /proc/sys/vm/drop_caches
blockdev --flushbufs $DISK
hdparm -F $DISK
dd if=$MOUNT/temp-hddread.tmp of=/dev/null

Run this when you're done.

MOUNT=/mnt/test # <===ADJUST THIS===
rm $MOUNT/temp-hddread.tmp

Explanation:

The HDD may have H/W caches that will not be cleared by the above commands. I'm writing and reading pseudo-random data hopping to fill them with garbage. How much data depends on how large the HDD cache may be. I'm using /dev/urandom because it's fast and we don't care about true randomness. I'm creating /mnt/test/temp-hddread.tmp from the start and use it every time I want to read enough random data. I'm creating and deleting /mnt/test/temp-hddwrite.tmp each time I want to write enough random data.

Credits

I've wrote this answer based on the best parts of the existing answers.

Answer 2

Unmounting and re-mounting the disk under test will reset all caches and buffers.

Answer 3

Sounds like you want the sync command, or the sync() function.

If you want disk cache flushing: echo 3 | sudo tee /proc/sys/vm/drop_caches

Answer 4

Disk cache purging: echo 3 | sudo tee /proc/sys/vm/drop_caches

Command documentation: https://www.kernel.org/doc/Documentation/sysctl/vm.txt

Writing to this will cause the kernel to drop clean caches, dentries and inodes from memory, causing that memory to become free.

To free pagecache:

echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation, and dirty objects are not freeable, the user should run "sync" first in order to make sure all cached objects are freed.

Answer 5

You can do it like this:

# sync # (move data, modified through FS -> HDD cache) + flush HDD cache
# echo 3 > /proc/sys/vm/drop_caches # (slab + pagecache) -> HDD (https://www.kernel.org/doc/Documentation/sysctl/vm.txt)
# blockdev --flushbufs /dev/sda
# hdparm -F /dev/sda

# NEXT COMMAND IS NOT FOR BENCHMARKING:
# should be run before unplug, flushes everything possible guaranteed.
# echo 1 > /sys/block/sdX/device/delete

You may use strace to see that these are three different syscalls

Also, it may be desirable to turn off HDD cache using hdparm, not sure what thing you benchmarking.

In any way, you cannot prevent HDD to cache last 64/32/16 MB of recently used data. In order to kill that cache, just write some amount of zeroes (and flush) + read some unrelated place from HDD. This is required since cache may be divided to read-part and write-part. After that you can benchmark HDD.