This question arises from another one that I had posed on Stackoverflow. I am using Watcher – the same issues apply to Incron – to monitor a folder and its child folders for changes and silently squirrel away those changes to Dropbox.
I monitor the write_close event – IN_CLOSE_WRITE – for the purpose. Originally I was watching the modify event, i.e. IN_MODIFY. While this worked I found that when writing large files it would fire more than once. That sounded fair so I switched to IN_CLOSE_WRITE since I felt that it was reasonably fair to assume that for a given file it would only occur once.
However, that is not the case. Even for a very small text file – just one character – created in Nano the event occurs two times. At best this can result in unnecessary traffic when the same file is synchronized on Dropbox two times. In my own case it leads to disaster since on the first event I perform the synchronization and then delete the server side file. The result – on the second event the Dropbox side file becomes a 0 byte file.
I am dealing with this for now by making my synchronization script sleep for 10s
before I do anything else and then I check that the file in question still exists prior to attempting Dropbox sync. This works because on the second iteration the file is missing and the script just terminates.
This sounds hackish at best. Perhaps not a bad hack but I would prefer to understand – just why does even the IN_CLOSE_WRITE event occur more than one time?
Some additional information
- Check to ensure that there aren't multiple instances of watcher running.
Output from ps ax|grep watcher.py
23880 ? Sl 0:01 python /usr/local/bin/watcher.py restart
24977 pts/0 S+ 0:00 grep --color=auto watcher.py
The file system is ext4. I should mention that I have encountered precisely the same issue with Incron. I start the Watcher daemon up from a batch script executed via /etc/rc2.d. Incron OTH starts up without any messing about by me via its default apt-get install incron installation.
The essence of my watcher.ini file is shown below.
[DEFAULT]
logfile=/var/log/watcher.log
pidfile=/var/run/watcher.pid
[job1]
watch=/path/to/watch
events=write_close
excluded=
recursive=true
autoadd=true
command=/home/datastore.php $filename
I reduced the datastore.php script to the bare essentials to verify that it is fired up twice without any of my messy Dropbox upload + source delete code.
#! /usr/bin/php
<?php
file_put_contents('/tmp/watcher',$argv[1],FILE_APPEND);
?>
I then created a little file at the path in question and then examined /tmp/watcher. The problem still persists – the file still has two successive entries for $argv[1].
Best Answer
I've just had this using IN_CLOSE_WRITE and finally tracked down the problem. My original script was reading in mail from stdin line by line and then writing each line to the file. This was causing incron to fire on every line write. I saw this based on the number of incron events and how many lines were in the file.
I then changed the original script to read all of stdin in before writing the file. Once this was in place I no longer saw multiple incron executions on the same file.