From what I see in the kernel source, inotify does only fire up after a write is completed (i.e. your guess is wrong). After the notification is triggered, only two more things happen in sys_write
, the function that implements the write
syscall: setting some scheduler parameters, and updating the position on the file descriptor. This code has been similar as far back as 2.6.14. By the time the notification fires, the file already has its new size.
Check for things that may go wrong:
- Maybe the reader is getting old notifications, from the previous write.
- If the reader calls
stat
and then calls read
or vice versa, something might happen in between. If you keep appending to the file, calling stat
first guarantees that you'll be able to read that far, but it's possible that more data has been written by the time the reader calls read
, even if it hasn't yet received the inotify notification.
- Just because the writer calls
write
doesn't mean that the kernel will write the requested number of characters. There are very few circumstances where atomic writes are guaranteed up to any size. Each write
call is guaranteed atomic, however: at some point the data isn't written yet, and then suddenly n bytes have been written, where n is the return value of the write
call. If you observe a partially-written file, it means that write
returned less than its size argument.
Useful tools to investigate what's going on include:
strace -tt
- the auditd subsystem
I think your approach is correct, and tracking the cookie is a robust way of doing this.
However, the only place in the source of inotify-tools (3.14) that cookie
is referenced is in the header defining the struct
to match the kernel API.
If you like living on the edge, this patch (issue #72) applies cleanly to 3.14 and adds a %c
format specifier for the event cookie in hex:
--- libinotifytools/src/inotifytools.c.orig 2014-10-23 18:05:24.000000000 +0100
+++ libinotifytools/src/inotifytools.c 2014-10-23 18:15:47.000000000 +0100
@@ -1881,6 +1881,12 @@
continue;
}
+ if ( ch1 == 'c' ) {
+ ind += snprintf( &out[ind], size-ind, "%x", event->cookie);
+ ++i;
+ continue;
+ }
+
if ( ch1 == 'e' ) {
eventstr = inotifytools_event_to_str( event->mask );
strncpy( &out[ind], eventstr, size - ind );
This change modifies libinotifytools.so
, not the inotifywait
binary. To test before installation:
LD_PRELOAD=./libinotifytools/src/.libs/libinotifytools.so.0.4.1 \
inotifywait --format="%c %e %f" -m -e move /tmp/test
Setting up watches.
Watches established.
40ff8 MOVED_FROM b
40ff8 MOVED_TO a
Assuming that MOVED_FROM always occurs before MOVED_TO (it does, see fsnotify_move()
, and it's an ordered queue, though independent moves might get interleaved) in your script you cache the details when you see a MOVED_FROM line (perhaps in an associative array indexed by ID), and run your processing when you see a MOVED_TO with the matching half of the information.
declare -A cache
inotifywait --format="%c %e %f" -m -e move /tmp/test |
while read id event file; do
if [ "$event" = "MOVED_FROM" ]; then
cache[$id]=$file
fi
if [ "$event" = "MOVED_TO" ]; then
if [ "${cache[$id]}" ]; then
echo "processing ..."
unset cache[$id]
else
echo "mismatch for $id"
fi
fi
done
(With three threads running to shuffle a pair of files each 10,000 times, I never saw a single out of order event, or event interleaving. It may depend on filesystem and other conditions of course.)
Best Answer
What happens if you
mount -o remount
the CIFS filesystem?This sounds like it is a bug in the CIFS implementation, in that directories are not triggering the notify events correctly.
I could find no references regarding event injection into an existing inotify stream. I suppose it could be done using SystemTap, but that's not practical. A better solution is to file a bug report on
ReadyMedia
(current name for the project at SourceForge.net).I had a quick look at the code for
notify.c
at SF.net and it looks good, but just 30 seconds made it clear there was a memory leak in the code that adds/removes watches. :( However, it looks like modifying the code to properly support notify on directories wouldn't be tough.