How can I monitor disk I/O in a particular directory?

Question

I've got a few processes with a known name that all write to files in a single directory. I'd like to log the number of disk block reads and writes over a period (not just file access) to test whether a parameter change reduces the amount of I/O significantly. I'm currently using iostat -d -p, but that is limited to the whole partition.

related: How can I have activity in a folder logged? and monitoring activity on my computer. — Tobias Kienzler, Mar 18 '11 at 09:20
This is different from those questions, because I'm not interested in the files themselves but rather the amount of I/O. — l0b0, Mar 18 '11 at 09:42

score 10 · Accepted Answer · edited Jan 24 '12 at 03:29

I realize this is going to sound both simplistic and absurd, but if you have control over the apps in question (maybe in a test environment) you could mount ONLY that directory on a partition of its own, then iostat, etc. would tell you only about it, and nothing else on that spot.

If there are physical drives involved you could fake it up with a loopback mount à la

dd if=/dev/zero of=/bigdisk/LOOPFILE bs=1024m count=1024m # 1gb loopback file
mke2fs -j /bigdisk/LOOPFILE
mkdir /tmpcopy
mount -o loop /tmpcopy /bigdisk/LOOPFILE
cp -r -p $SPECIALDIR2MONITOR /tmpcopy
umount /tmpcopy
mount -o loop $SPECIALDIR2MONITOR  /bigdisk/LOOPFILE,

That would not completely remove all competing disk I/O, but I'm pretty sure iostat's output would be more specific to your need.

Tobias Kienzler · Answer 2 · 2011-03-18T09:39:50.773

3

You can use inotifywait -m DIRNAME from the inotify-tools.

edited Mar 18 '11 at 09:39

answered Mar 18 '11 at 09:24

Tobias Kienzler

9,374

-d doesn't exist in inotifywait 3.13. Do you mean -m? – l0b0 Mar 18 '11 at 09:35
@l0b0: oh, I use 3.14 from the git repo. sorry. But the manpage states it's the same as -m with the exception of running in the background and requiring an outfile. – Tobias Kienzler Mar 18 '11 at 09:38
also -r might be useful – Tobias Kienzler Mar 18 '11 at 09:40
1

Tested it. I'm sorry for the misunderstanding, but I'm not interested in which files (or even how many files) were accessed, but rather the amount of I/O. – l0b0 Mar 18 '11 at 09:41
Oh I see. Maybe inotifywatch does what you want, but I don't know much about it – Tobias Kienzler Mar 18 '11 at 10:17
2

Note, this is Linux-specific. BSDs have kqueue and pnotify system calls, but I don't know if they do exactly what the author is requesting. – Shawn J. Goff Mar 18 '11 at 12:03

score 3 · Answer 3 · answered Mar 18 '11 at 20:55

I don't think there's a direct way. One way to get the data you want would be to access the directory tree through a virtual filesystem that logs accesses. Loggedfs is one such filesystem, though I don't know if it can show all the data you're interested in. (If not it would probably be a modest coding effort to that data.)

mkdir /tmp/replica
loggedfs /path/to/directory /tmp/replica
mycommand --root=/tmp/replica
fusermount -u /tmp/replica

How can I monitor disk I/O in a particular directory?

3 Answers3

Linked

Related