Save and call matrices with large output in Mathematica

Question

If we have the large matrix

A=RandomReal[{-10, 10}, {1000, 1000}];

and I need it for further calculation, how can I save it on hard disk and then call for further calculation? Output is very large and Mathematica can not present it, so I can not copy/paste it? Is it possible to print it and then call from hard disk?

Thank you in advance!

Related answer about saving definitions that also extends to data: http://mathematica.stackexchange.com/a/2008/121 — Mr.Wizard, Apr 23 '12 at 06:39

score 10 · Answer 1 · answered Apr 23 '12 at 02:14

10

Export["A.wdx", A];

will save matrix A in file "A.wdx" on disk. WDX format is binary, compressed, and cross-platform.

Restart Mathematica and evaluate

A = Import["A.wdx"];

This will read "A" from disk file "A.wdx" and store it in symbol A.

Proceed with your computation.

answered Apr 23 '12 at 02:14

Vince

161
1
3

yes, but look this problem, if the matirx is in function of variables, when I call after I can not change the variables In[3]:= A[w1_, w2_] := ( { {2 w1[1]w1[3], 4 w2[5]w1[6]}, {3 w1[1]w2[3], 7 w1[1]w2[3]} } );
In[4]:= Export["A.wdx", A[w1, w2]];

In[5]:= A[w1_, w2_] := Import["A.wdx"];

In[7]:= A[z, z]

Out[7]= {{2 w1[1] w1[3], 4 w1[6] w2[5]}, {3 w1[1] w2[3], 7 w1[1] w2[3]}}
– George Mills May 01 '12 at 13:46
Yes, my answer was limited and was biased towards saving simple data. Still you might get what you want with replacement rules, awkwardly. Say, With[{ia=Import@"A.wdx"}, A[w1_,w2_] := ReplaceAll[ia, {Globalw1 -> w1, Globalw2 -> w2}]; or some such thing. – Vince May 07 '12 at 03:27

score 5 · Answer 2 · answered Apr 22 '12 at 14:29

5

If you enter:

A >> "savedA.m"

then you can get it back with:

B=<<savedA`

You will get a window, telling you, that a large output was generated. But B now is the same matrix as the A you saved.

answered Apr 22 '12 at 14:29

Peter Breitfeld

5,182
1
24
32

2

If you include a semi-colon after B=<<savedA` then the output will be suppressed. – rcollyer Apr 22 '12 at 17:28
@rcollyer can you give small example? – George Mills Apr 22 '12 at 19:17
@GeorgeMills sure, but there isn't much beyond what Peter said. Try this, generate a matrix A = RandomReal[1, {5,5}], save it A >> "saved.m", and read it back in B=<<saved`;. Note, by adding the semi-colon after saved`, I suppressed its output, and you can verify that it is the same matrix by, in this case, enter B in another cell, or if they're to large, A==B returns true. – rcollyer Apr 22 '12 at 19:39
@rcollyer thank you for many comments. 1) If I want to use A >> "saved.m" how to save somewhere else, for example in another folder and then call from there? 2) It is not necessary to put A==B, I can call with A=<<saved`; after exit from kernel. – George Mills Apr 22 '12 at 21:08
@GeorgeMills 1) You can specify a directory directly in the filename, just like any other program, or specify the directory before hand using SetDirectory. 2) Yes, after you exit the kernel you can then use A=<<saved` to reload A. – rcollyer Apr 22 '12 at 21:57
@rcollyer yes but how can I set directory to save on another partition for example D? – George Mills May 02 '12 at 14:07
@GeorgeMills how would you normally specify a directory? – rcollyer May 02 '12 at 14:16
@rcollyer mathematica is installed on C, but I want to Export on D partition, for example D:\George – George Mills May 02 '12 at 14:23
@GeorgeMills per the docs, SetDirectory["D:\\George"] where the \\ is necessary to generate \ in the filename. I'd look at the file naming tutorial, specifically pay attention to the Composing section. – rcollyer May 02 '12 at 14:28
@rcollyer thanks a lot – George Mills May 02 '12 at 14:29
@GeorgeMills it appears that the \ was being escaped incorrectly. See my edits above. – rcollyer May 02 '12 at 14:33
@rcollyer I see, it is clear, thank you very much. – George Mills May 02 '12 at 15:53
@rcollyer I used command SetDirectory["D:\George"] but I got message that I dont have enough space on C during calculation? why? here is location of big file C:\Users\user\AppData\Local\Temp – George Mills May 02 '12 at 17:02
@rcollyer you can see here output but I need to use just E, not C
In[27]:= ResetDirectory[]

Out[27]= "C:\Users\user\Documents"

In[29]:= DirectoryStack[]

Out[29]= {}

In[30]:= SetDirectory["E:\"]

Out[30]= "E:\"

In[32]:= DirectoryStack[]

Out[32]= {"E:\", "C:\Users\user\Documents"}
– George Mills May 02 '12 at 17:18
@GeorgeMills you're misinterpreting what DirectoryStack is telling you. The current working directory is on top of the stack, e.g. "E:\" in your case, and will be the one used. On the other hand, I don't know why your getting the out of space error. – rcollyer May 02 '12 at 17:24

score 5 · Answer 3 · edited Apr 13 '17 at 12:56

5

Yes, this is possible: use DumpSave. Warning: not cross-architecture friendly. For your example:

In[18]:= A = RandomReal[{-10, 10}, {1000, 1000}];

(* for demonstration, save the same matrix A to the variable B for double checking *)
B = A;


(* Dump the variable definition, which includes the variable name, to the file *)
In[20]:= DumpSave[FileNameJoin[{$TemporaryDirectory, "a.mx"}], A];


(* Clear the variable A and it's contents from Mathematica's memory. 
Now, the only remnants of A are in the file "a.mx" and in the copied variable B *)
In[21]:= Clear[A]

(* Import and define the variable A again *)
Get[FileNameJoin[{$TemporaryDirectory, "a.mx"}]]


(* Check that our important definition of A matches what we defined at the beginning *)
In[23]:= A == B

Out[23]= True

You can now use A as if you never quit MMA or cleared the variable. So, you can try to calculate the inverse (Inverse[A]) or any other legal MMA calculation.

Save and call matrices with large output in Mathematica

3 Answers3

Linked