1

How do I redirect all printed text to a file instead of the FrontEnd? This question was answered in Temporarily redirect the output of Print[ ] to a second file, except that this does not capture text printed in subkernels as I encountered when I replaced a Do with a ParellelDo. These prints are not saved in my log and are outputted in the FrontEnd. How do I fix this?

P.S.: The text that is printed by the subkernels does not necessarily have to be printed to the same file (might be problematic with kernels that want to write to the same file simultaneously), as long as it is logged somewhere.

Possible relevant posts are: Local logfiles for remote kernels and How to Export from ParallelDo?, but they are not enough for me to help me construct a solution to this problem.

The texts printed are diagnostics on what computations are being done: e.g. what is the computation, how much time is it taking and how much memory is it using. Not all the code is even written by myself. Thus replacing Print with some other function to collect and export the data is not really feasible. (Except if you mean to do something like Block[{Print=NewPrint}...], or another way where I don't actually need to alter the code. )

Why is it useful to Print to file instead of to the frontend? Firstly, a large amount of printed text output will slow down or freeze the frontend. Dedicated text editors are much better suited for dealing with thousands of lines of text. In addition this makes it much easier to save and efficiently archive (semi-permanent) logs of everything you have run.

Kvothe
  • 4,419
  • 9
  • 28
  • This seems like an example of fighting the paradigm, turning an easy problem into something hard. Print is not terribly useful: it's almost always easier to collect your results in a List and Export that. – John Doty Jun 06 '18 at 15:12
  • @John, It is not really a result that I'm looking to export. I want a log of diagnostic that are Print'ed by my codes. – Kvothe Jun 06 '18 at 15:14
  • Did you try to distribute the $Output definition to the other kernels? Also, @JohnDoty is right, why does it have to be Print? If it's really debug information, it should be Printed in the first place, right? You can always define your own debugPrint and swap out it's definition depending on the circumstances – Lukas Lang Jun 06 '18 at 15:32
  • @Mathe, can you expand on that (or write an answer)? What do you mean did I try to distribute $Output. For the second part I don't really understand. You actually agree that it should be Printed right? Do you mean why I don't want it to be Printed in the FrontEnd, yes maybe I should have answered that (note that the previous asker of the similar question also assumed that the intent would be clear). (cont'ed) – Kvothe Jun 06 '18 at 15:39
  • @Mathe, There are very good reasons: The output can become quite overwhelming. It slows down the FrontEnd, it is annoying how it keeps shifting the focus when adding lines from the next Print. Also the large amount of text lines are actually much more readable in a normal text editor. (The MMA frontend is not actually that well suited for a very large amount of printed lines.) In addition, this way you have permanent logs (that I automatically clearly label by date and which program was running) instead of some text hidden away in a notebook as output, that will be replaced next time you run. – Kvothe Jun 06 '18 at 15:42
  • @Kvothe Sorry, I meant to write "... should not be Printed in ..." - my point (and @JohnDoty's, if I understood correctly) is the following: Why do you use Print at all? As you said yourself, it's mostly annoying to work with what it produces. If you use your own logging function, you do not need to modify the behavior of a built-in function, which saves you a lot of hassle. Regarding the distribution of $Output: I was suggesting to use DistributeDefinitions, but as I've said, that's probably not the way to go. – Lukas Lang Jun 06 '18 at 15:48
  • @Mathe, ah then I just strongly disagree. Perhaps you can do something equivalent to Printing only this one issue would work with it and in that case it is better but in general printing and then diverting to a text file is perfectly suited for what I want. It is basically what Print is meant for, with the one exception that a text editor is actually better suited to view the printed text than the frontend once the diagnostic texts become to big. Note that the solutions in the linked post has suited my needs perfectly with the one exception that it now does not work for parallel programming. – Kvothe Jun 06 '18 at 15:53
  • @Mathe I suspect that this last is a fixable flaw. And thus that this is just a perfect way to do what I want. I agree that strictly speaking a debugPrint might be nice to separate from a EndUserPrint except that the programs are never supposed to print anything but diagnostics. So this would just leave Print unused and would achieve the same thing. – Kvothe Jun 06 '18 at 15:58
  • @Mathe. Finally, arguments by authority are never the strongest but just notice that all of the greatest MMA stackexchange posters all contributed to or replied to the solution to the previous question and did not question its validity at all. – Kvothe Jun 06 '18 at 15:59

2 Answers2

4

The inadequacy of simple print mechanisms for logging is the reason that Unix and related systems have the syslog mechanism. It's pretty easy to access this from Mathematica.

syslog[s__]:=Put[s,"!/usr/bin/logger"]

On traditional systems like Ubuntu, syslog["Something happened"] puts a line like

Jun 6 17:02:49 Halley jpd: "Something happened"

in /var/log/syslog. In MacOSX they show up in the Console app. The logger shell command takes options that allow you some control over where messages go, see its manpage.

John Doty
  • 13,712
  • 1
  • 22
  • 42
  • Thanks. I don't think however that this is going to be very convenient. I want the logs to be nicely searchable, to this end I want them to carry the name of the nb they came from and the date and time they nb was run. In addition I like to keep them in directories relative to the nb. This means that it is probably much easier to control the output directory from within Mathematica than from a separate shell. (Although I guess all this data can be given to the logger from within Mathematica, it just doesn't seem to be the easiest way). – Kvothe Jun 07 '18 at 08:22
1

We can write a function that writes to a log file.

For instance:

writeLog[logfile_][message__] :=
 Module[{str = OpenAppend[logfile], kernelString},
  kernelString = Which[
        $KernelID == 0, Unevaluated@Sequence[],
     IntegerQ[$KernelID], "Kernel" <> ToString[$KernelID] <> ": ",
     True, Print["Error in writeLog"]; $Failed];
  WriteString[str, 
   StringJoin @@ 
    Riffle[{StringJoin[ToString /@ List[kernelString, message]], 
      "\n"}, "\t"]];
  Close[str];]

We then would like to be able to use Block[{Print=writeLog[logfile]},...], however we need to be careful how to do this when using Parallel versions of Do, Map, Table etc. Taking Do just as an example, the Block has to happen inside the ParallelDo. Or we can define

SetAttributes[ParallelDoWriteLog, HoldAll]
ParallelDoWriteLog[expr_, iter_] := 
 ParallelDo[Block[{Print = writeLog[logfilename]}, expr], iter]

According to Writing data to a common file during parallel processing this can run into trouble when both kernels want to write to the same file at the same time (this didn't cause trouble for my test examples but might be risky). Thus we can also write to separate log files for different subkernels:

SetAttributes[ParallelDoWriteNewLog, HoldAll]
ParallelDoWriteNewLog[expr_, iter_] := 
 ParallelDo[
  Block[{Print = writeLog[addKernelString[logFileStatic]]}, expr], 
  iter]

addKernelString[logfile_] := 
 StringJoin@
  Insert[StringSplit[logfile, "."], 
   "_k_" <> ToString@$KernelID <> ".", -2]

This solution borrowed heavily from How can I suppress Print in parallel evaluation? (celtschk's answer) and Temporarily redirect the output of Print[ ] to a second file (rm -rf's answer).

Kvothe
  • 4,419
  • 9
  • 28