Linux Fu: Watch That Filesystem

The UNIX Way™ is to cobble together different, single-purpose programs to get the effect you want, for instance in a Bash script that you run by typing its name into the command line. But sometimes you want the system to react to changes in the system without your intervention. For example, you might like to watch a directory and kick off some program automatically when a file appears from a completed FTP transaction, without having to sit there and refresh the directory yourself.

The simple but ugly way to do this just scans the directory periodically. Here’s a really dumb shell script:

while true
   for I in `ls`
    do cat $I; rm $I
 sleep 10

Just for an example, I dump the file to the console and remove it, but in real life, you’d do something more interesting. This is really not a good script because it executes all the time and it just isn’t a very elegant solution. (If you think I should use for I in *, try doing that in an empty directory and you’ll see why I use the ls command instead.)

Increase Elegance

Honestly, you want something more elegant right? Modern kernels (2.6.13 and later) have filesystem notifications in the form of an interface called inotify. You can use these calls programmatically with the sys/inotify.h header file. There is also a set of command-line programs you can install, usually packaged as inotify-tools.

One of those tools is inotifywait and it makes for a nicer script. For example:

while true
   if FN=`inotifywait –e close_write,moved_to --format %f .`
    cat $FN
    rm $FN

That’s better, I think. It doesn’t wake up frequently, only when something has changed. I figure any sane program putting something in the directory will either open the file for writing and close it, or it will move it. Either way will work and the %f tells the command to report the file name. There are other events you can wait for as well, of course.

If you are wondering why the move case is necessary, think about how most text editors and network download software works. Usually, a new file doesn’t have the final name until it is complete. For example, Chrome will download the file test.txt as test.txt.crdownload or something like that. Only when the file is done will it rename (move) the file to test.txt.

If you want to try the command without a script so you can see the effect, just open up two terminal windows like this:

In the lower terminal, issue the inotifywait command. Don’t forget the period at the end which tells it to monitor the current directory. Then in the other terminal create a file in the same directory. The name of the file will appear in the first terminal and the program will exit. The script just takes advantage of this behavior to set the FN variable, takes action, and then relaunches inotifywait. You can ask the program not to quit, by the way, but that makes scripting a little more difficult. However, it also removes the problem of a file changing while you are doing your processing.

The other command line, inotifywatch, also outputs file change events but it watches for a certain amount of time and then gives you a summary of changes. I won’t talk about it any further. If you think you need that capability, you can read the man page.

A New Cron

The script is still less than ideal, though. Presumably, a system might have lots of different directories it wants to monitor. You really don’t want to repeat this script, or a variation of it, for each case.

There is another program for that, called incron (you will almost surely have to install this one). The incron program is like cron but instead of time-based events, the events are based on file notifications. Once you install it, you will probably have to change /etc/incron.allow and /etc/incron.deny if you want to actually use it, especially as a normal user.

Suppose you want to run a script when a file appears in the hexfiles directory. You can use the command incrontab -e to edit your incron table. The format is very picky (it wants spaces, not tabs, for example). Here’s a line from the file that will do the job:

/home/alw/Downloads/hexfiles IN_CLOSE_WRITE,IN_MOVED_TO /home/alw/bin/program_cpu $@/$#

The $@/$# at the end provides a full path to the file affected. You can also grab the vent time as text ($%) or a number ($&). You can monitor all the usual events and also set options to do things like not dereference symbolic links. You can find it all in the incron man pages.


I’m not a big fan of GUI editors, but I know I’m in the minority. If you like, there’s a Java-based incrontab editor available. There isn’t much documentation, but you can import your incrontab — if it exists — from /var/spool/incron/your_user_id. If you look at the image below, you can see it offers a form that builds the incron table line for you.

You can find the system files in /etc/incron.d, usually. All the locations can be set by the /etc/incron.conf file, so if you aren’t sure where to look or you want to change the location for the table files, start there.

Go Forth and Watch

Using incron is quite elegant. A system program does all the waiting and our script only runs when necessary. It is easy to look and see all the things you have notifications set for. You can do a lot with these tools, and not just in the embedded space. How are you going to use them?


Read More here.