5

I'm running a simulation with parallel, and I want the results from each parameter set exported to a file whenever it's done (because the parameter space is huge and my computer doesn't have enough memory to keep all the results). This is the code I tried:

output = OpenWrite["file.txt", PageWidth -> Infinity];
WriteString[output, "certain text"]; 
WriteString[output, "\n"]; 
ParallelDo[
 myarray=myfunction[i,j,k];
 Export[output, myarray // N, "CSV"];
 WriteString[output, "\n"],
{i,5},{j,4},{k,3}
]

Unfortunately I keep getting this error message which I don't understand:

CharacterEncoding is not a known option for OutputStream[file.txt,82].

The problem does not occur when I use Do instead of ParallelDo. So I'm guessing it's got something to do with parallelization. What am I missing here?

--------edit----------

Apparently when I use PutAppend I don't get any error message, but I can't figure out how to remove the quotation marks and the line brakes in the middle of the data.

J Yeh
  • 105
  • 4

1 Answers1

3

The output variable lives on the main kernel. This variable is special, it represents an open file. It cannot be used on the parallel kernels if the file was opened on the main kernel.

Is it still possible to write to the same file in parallel from multiple different kernels? Yes, but to do it robustly you must use an advanced parallel programming concept: locks and CriticalSection. This is not only complicated, but it can also have detrimental effect on performance in Mathematica.

So why do you want to write to the same file from all subkernels? I would write to a different file from each subkernel, then merge those files at the end if necessary. To build a unique file name for each subkernel, you can use their $KernelID.

Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263