0

I recently started to work with Linux server, I am very new. My CUDA/C++ program solves 2D differential equation and writes down output every, say, 1000 time steps. It happens roughly every minute.

Is it possible to automatically download files to my PC once they are generated on the Linux server, or save them directly to my PC?

This would significantly accelerate my work since now I have to wait for my program to finish all the calculations and then download it manually. I also typically use 6 GPUS at the same time, they produce output in different specified folders on the LINUX server (say, folders 0, 1,2,3,4,5).

Kamil Maciorowski
  • 74,068
Mikhail Genkin
  • 105
  • "Is it possible to automatically download files to my PC once they generated on the Linux server" - Yes; You can write code that will do this. – Ramhound Dec 02 '16 at 21:02

2 Answers2

1

You can run a simple server on the Linux machine that serves the generated files, e.g. cd to the desired directory and run:

python -m SimpleHTTPServer

Then on the home PC you can use wget to periodically download the new stuff (as achieved with -N):

while true; do
    wget -rN http://server/
    sleep 60
done

If you are running Windows at home, you can get Wget for Windows and write the above as a BAT script, or use a different downloader that does that kind of mirroring natively.

timotree
  • 1,118
user4815162342
  • 313
1

Yes, you can use inotify-wait a command which is part of the inotify-tools package. Create a file, called for instance my_monitor, with content

#!/bin/bash
while true # will loop forever!
do 
   inotify-wait -r -e modify,attrib,close_write,move,create,delete /path/to/dir/or/file/to/monitor && /path/to/script
done

inotify-wait monitors a directory or a file (and, if you add the flag -r, it will monitor the whole directory tree recursively) for changes. I have selected some typical changes to watch for, and you can find an exhaustive list here; inotify-wait terminates as soon as one event in the list above occurs, and the command above then executes some bash script (which you have to write) that will copy the files you need to your other machine. As soon as the file transfer completes, the cycle starts again.

You can start the script above with

nohup /path/to/my_monitor

which means that, even if you log out from this pc, the script will not stop working: basically, it will run forever (i.e., until reboot).

That's all.

MariusMatutiae
  • 47,503
  • 12
  • 81
  • 131
  • Can you provide an example? Say, on Linux server my file is created in the directory: share-home/genkin/0, and I want to download it to my PC to the folder C:\data\0 – Mikhail Genkin Dec 06 '16 at 20:10