Linux Server – How to Download File Once Created

downloadlinux

I recently started to work with Linux server, I am very new. My CUDA/C++ program solves 2D differential equation and writes down output every, say, 1000 time steps. It happens roughly every minute.

Is it possible to automatically download files to my PC once they are generated on the Linux server, or save them directly to my PC?

This would significantly accelerate my work since now I have to wait for my program to finish all the calculations and then download it manually. I also typically use 6 GPUS at the same time, they produce output in different specified folders on the LINUX server (say, folders 0, 1,2,3,4,5).

Best Answer

Yes, you can use inotify-wait a command which is part of the inotify-tools package. Create a file, called for instance my_monitor, with content

#!/bin/bash
while true # will loop forever!
do 
   inotify-wait -r -e modify,attrib,close_write,move,create,delete /path/to/dir/or/file/to/monitor && /path/to/script
done

inotify-wait monitors a directory or a file (and, if you add the flag -r, it will monitor the whole directory tree recursively) for changes. I have selected some typical changes to watch for, and you can find an exhaustive list here; inotify-wait terminates as soon as one event in the list above occurs, and the command above then executes some bash script (which you have to write) that will copy the files you need to your other machine. As soon as the file transfer completes, the cycle starts again.

You can start the script above with

nohup /path/to/my_monitor

which means that, even if you log out from this pc, the script will not stop working: basically, it will run forever (i.e., until reboot).

That's all.

Related Question