9

Context

I run long Mathematica sessions; sometimes I have Output up to Out[3000+]. Hence I would like to be able to clear out some previous outputs which do use lots of memory.

Attempt

I am aware of clearing some outputs via unprotecting Out as follows

In[1]:= MemoryInUse[]/10^9.

Out[1]= 0.0179439

In[2]:= a = Table[1, {10^8}];
In[3]:= MemoryInUse[]/10^9.

Out[3]= 0.419424

In[4]:= Unprotect[Out];
In[5]:= Out[2] =.
In[6]:= Protect[Out];
In[7]:= MemoryInUse[]/10^9.

Out[7]= 0.0297147

My problem is if I have 3000+ outputs, which one are using a lot of memory so I can decide if it is safe to clear them.

Question

What I am after is a way of finding out which amongst the previous Out are using most of the memory? I would like to have a function, say MemoryHog[n] which would return a list of numbers corresponding to the nth sorted Outputs using most memory.

chris
  • 22,860
  • 5
  • 60
  • 149

2 Answers2

12

This should work

ClearAll[MemoryHog];
MemoryHog[n_] := Module[{list},
   list = {#, ByteCount[Out[#]]} & /@ Range[$Line];
   list=SortBy[list, -Last[#] &];
   list[[1 ;; n]]
];

Now lets test it

(*Some sample input -- evaluate in three cells *)
(*In[3]*)
Range[1000];
(*In[4]*)    
Range[10000];
(*In[5]*)    
Range[100000];

(*  get the hogs*)
MemoryHog[3]
(* ==>
{{5, 400168}, {4, 40168}, {3, 4168}}
*)
Ajasja
  • 13,634
  • 2
  • 46
  • 104
  • You remembered $Line before I did, but you seem to have forgotten about SortBy. – Mr.Wizard Aug 20 '12 at 11:06
  • True, true, list=SortBy[list, -Last[#] &] would work as well... – Ajasja Aug 20 '12 at 11:16
  • Actually better; time it. :-) – Mr.Wizard Aug 20 '12 at 11:18
  • Sorry, no time for that now :) I'll take your word for it. – Ajasja Aug 20 '12 at 11:19
  • @Ajasja I understand that Instring/@ Transpose[MemoryHog[3]][[1]] would allow me to review the corresponding Inputs. On the other hand its not very readable. Would you know how to print it out as the equivalent of ??In but restricted to the hogs? It would help to decide if they can be cleared without pain. – chris Aug 20 '12 at 11:55
  • As a quick hack this works DisplayForm@ToExpression@InString[#] & /@ MemoryHog[3][[All, 1]] – Ajasja Aug 20 '12 at 12:08
8

Why do you have so many Out[] values assigned? This seems like a very poor practice if memory is a concern. I suggest setting something reasonable like $HistoryLength = 3 and finding another way to store your results.

Nevertheless you can find the approximate memory used for each value like this:

Array[{#, ByteCount@Out@#} &, $Line - 1]

This gives output such as:

{{1, 56}, {2, 184}, {3, 6520}}

which are a pairs: {line-number, bytes-used}

It is probably academic now that you know about $HistoryLength, but here is a way to write your "memory hog" function that has some features you may like. It respects $HistoryLength and $Line such that it will not error if you request more Out values than are available, and it returns a Short summary of each Out value.

memoryHog[n_] :=
  Reverse@#[[# ~Ordering~ -Min[n, $HistoryLength, $Line - 1]]] & @ 
    Table[
      {ByteCount@Out@i, i, Short@Out@i},
      {i, Max[1, $Line - $HistoryLength], $Line - 1}
    ] // Column
Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371
  • @Wizard I have many Out[] because I do research ;-) i.e. I don't necessarily know where I am going. Or if you prefer I am not a good programmer. Thanks for your answer. – chris Aug 20 '12 at 11:33
  • @chris That does not require you to keep all output, does it? Perhaps you should look into a logging method that will write each Out to a file? – Mr.Wizard Aug 20 '12 at 11:36
  • I didn't know about the $HistoryLength though. It might do the trick for me. – chris Aug 20 '12 at 11:37
  • @Wizard No; typically I don't use any of the outputs. – chris Aug 20 '12 at 11:38
  • @chris in that case, certainly set $HistoryLength to a low or zero value in your user init.m file. Ask if you need help with that. – Mr.Wizard Aug 20 '12 at 11:39
  • @Wizard Thanks. Your version of memory hog assumes $HistoryLength is set it seems. I would be after input rather than output. I found that InString[5] gives me this input but in a strange format. I am sure you know how to make it human readable ? – chris Aug 21 '12 at 09:01
  • @chris sorry, yes this assumes that $HistoryLength is set, but I forgot to state it explicitly. Try: DisplayForm @ ToExpression @ InString[5] – Mr.Wizard Aug 21 '12 at 09:11
  • Thanks; I ll modify your function accordingly. The problem I see with the current form is that Short[ something big] takes a while to process. – chris Aug 21 '12 at 10:50
  • @chris if the input line is descriptive enough you could use DisplayForm @ ToExpression @ InString @ i in the place of Short @ Out @ i -- perhaps you already are doing this. – Mr.Wizard Aug 21 '12 at 10:53