2

I can not access the directory on my remote kernel these days. Therefore, I want to export the output of the computations to my local machine.

Some details:

  • My remote kernel is on a Tyan TX46 with 4 Opteron 2.2Ghz cpus and 32GB of ram and my local machine has a Intel(R)Core(TM)2CPU 6600 @2.40 GHz processor and 4gb ram.

  • I don't use parallel remote kernels.

  • I installed my remote kernel by some settings on Evaluation>Kernel Configuration Options.

Is there a way to do that?

cesm
  • 449
  • 3
  • 12
  • 1
    I think the question has potential but you should add more details. Are you using parallel remote kernels? What are your OS? How do you open a remote connection? Etc.. Everything which could be important to a potential answerer. – halirutan Feb 11 '13 at 02:50
  • @halirutan Thank you. I have added some details. Please see my edit. – cesm Feb 11 '13 at 15:18
  • What do you mean "you can't access the directory"? How can you launch a remote kernel, e.g. through ssh, without being able to scp the file? Is this something you need to do once, or every time you run your Mathematica program? How much data are we talking about? – Joel Klein Feb 11 '13 at 15:34
  • @JoelKlein When I accessed to remote kernel, it generates a directory for me on network. I use this directory in order to write some data and import this data to Mathematica. I launch the remote kernel by ssh. I should write every computed variable, nearly 16 variables. Some variables use 5-10 GBs of Ram and some variables use 1-2 GBs of Ram. – cesm Feb 11 '13 at 16:05
  • 1
    I think you probably have to share a directory either on your or on the remote computer. Then you can export directly to the shared directory. Is this windows or linux? – Ajasja Feb 11 '13 at 17:49
  • I bet you could define a custom Stream type which would stream the data over the network to a local kernel, which would save the data to a file. This would involve either MathLink or the Parallel package. That said, it's probably less of a hassle to Save the data and either scp it from the remote machine or, if for some reason this is impossible, invoke scp from mathematica on the remote machine to push the data to your local machine. – Tobias Hagge Feb 11 '13 at 19:47
  • @Ajasja Yes,however, the problem is that I can't access to this directory. Because of this, I try to find alternative ways. In addition, the OS is windows. – cesm Feb 11 '13 at 19:53
  • @cesm Do you mean that every time you log in using ssh, a new directory is created for you, and that you can't access the directory of one ssh session from another one? – Szabolcs Feb 11 '13 at 20:04
  • @Szabolcs No, the first time I accessed it, it generated a directory for me .So, I have solely one directory. – cesm Feb 11 '13 at 20:23
  • 2
    @cesm Could you explain why you can't access the files on the remote machine then? Is there any chance you could figure out how to access them? I think this would be more convenient than trying to find a solution within Mathematica. Even if we did implement a MathLink based solution, it would be very very slow and unreliable for transferring gigabytes over the network. – Szabolcs Feb 11 '13 at 20:34
  • @Szabolcs There is a problem with network. Because of this I can't access it. I am also struggling to solve this issue. In order to go further, I think there may be a way to import the data onto local machine. I understood, thank you for your interest. – cesm Feb 11 '13 at 20:57
  • @cesm Here is what I'd do: Create a folder on your local computer and share it with everybody. Then export to that folder from mathematica. For example write to \\NAME-OR-IP-OF-MY-COMPUTER\MY-SHARED-FOLDER – Ajasja Feb 11 '13 at 21:33
  • I cannot test this on Windows now but my MSFTP package, mentioned here: http://mathematica.stackexchange.com/a/19015 should do what you want. – Rolf Mertig Feb 11 '13 at 21:48

1 Answers1

1

In an effort to reduce unanswered questions:

I would create a folder on your computer (example C:\tmp) and give everyone read/write permissions

Shared

Then you can access the folder from the remote kernel by doing

d=RandomReal[{-1, 1}, {3, 2000}];
Export[Export["\\\\YOUR-COMPUTER-NAME_OR-IP\\tmp\\d.dat", d]

There are also other options:

  1. Set up a sftp server on your computer and connect to it using Rolf Mertig package.
  2. See if you can connect using scp.
  3. Read some papers while the administrators fix your network.
Community
  • 1
Ajasja
  • 13,634
  • 2
  • 46
  • 104
  • Hello, I've tried your proposal under Linux environment but I couldn't make it work. Is there two nested Export functions : Export[Export[...]] ? I tried as well something with ssh but with no success : Export["!ssh user@machineip cat d.dat", d] – sekisushai Jun 16 '15 at 09:45