Will incron’s IN_CREATE event give me a blank file

inotify

I want to monitor a directory for new files, which will be created by another process. I plan to use incrod to do this.

I am not interested in reacting to changes in files, as the files should never be updated. Therefore the IN_CREATE event seems the sensible option to listen for. However, I am unsure, if I listen for that event, could I end up with an empty file (before the other process has written any information to it)? Should I use the IN_WRITE_CLOSE event instead?

Best Answer

You can observe which events are passed to your directory with inotifywait to check how does it behave when your process is running:

$ inotifywait -m . 
Setting up watches.
Watches established.

For example, after running touch file in same directory:

$ inotifywait -m .
Setting up watches.
Watches established.
./ CREATE file
./ OPEN file
./ ATTRIB file
./ CLOSE_WRITE,CLOSE file

Since there is delay between creating file and closing it, you'll have empty file after CREATE event, as you suspected. Observe events coming after running:

from time import sleep

with open("somefile", 'w') as somefile:
    sleep(15)
    somefile.write("...")

Therefore using CLOSE_WRITE event sounds reasonable.

Related Solutions

Linux – Using Inotifywait for Large Number of Files in a Directory

Is there a particular reason you are using:

 | awk '($2=="CREATE"){print $3}' > ~/output.file

instead inotifywait options like --format and --outfile ?

If I run:

inotifywait -m --format '%f' -e create /home/don/folder/ --outfile /home/don/output.file

then open another tab, cd to ~/folder and run:

time seq -w 00001 50000 | parallel touch {}

real    1m44.841s
user    3m22.042s
sys     1m34.001s

(so I get much more than 500 files per minute) everything works fine and output.file contains all the 50000 file names that I just created.
Once the process has finished writing the files to disk you can append them to your test.out (assuming you are always in ~/folder):

xargs < /home/don/output.file cat >> final.file

Or use read if you want to process files as they are created. So, while in ~/folder you could run:

inotifywait -m --format '%f' -e create ~/folder | while read file; do cat -- "$file" >> ~/test.out; done

Note that in inotifywait stable, -m and -t cannot be used together. Support for usage of both switches has been recently added so if you build inotify-tools from git you should be able to use monitor with timeout (to specify how long it has to wait for an appropriate event to occur before exiting). I've tested the git version on my system (exit if no create events occur within 2 seconds) and it works fine:

inotifywait -m -t 2 --format '%f' -e create ~/folder | while read file; do cat -- "$file" >> ~/test.out; done

Bash – Execute command after inotifywait established watches

When inotifywait outputs the string "Watches established." is secure to make changes in the watched inodes, so you should wait to the string to appears on standard error before touching the files.

As an example,this code should to that,

inotifywait -r -m -e close_write "somedir" \
2> >(while :;do read f; [ "$f" == "Watches established." ] && break;done;\
find "somedir" -type f -exec touch {} ";")\
| while read f; do echo "$f hi";done

Best Answer

Related Solutions

Linux – Using Inotifywait for Large Number of Files in a Directory

Bash – Execute command after inotifywait established watches

Related Question