3

I always used mathematica for more "theoretical" computations: maybe I had to handle intricated manipulations, but things that you could almost do on paper.

Now I just joined a new project where we will have to handle HUGE (for my standard) computations: maybe simplifications of 400MB output! This time the difficulty is not "coding intricated manipulations", most of the time the thing we really have to do is a "//Together", but the expressions are so long...

I understand that these kind of things cannot be done simply on a notebook, as I always worked: the next simplification make Mathematica crash and I waste a night worth of computations.

So come my question: what are good "coding habits" to have when working in this kind of projects? Are there Mathematica functions I should be aware of (Monitor, Timing, those kind of things)? Also, good references/tutorials on this kind of topic?

I hope the question is not too vague...

giulio bullsaver
  • 437
  • 2
  • 9
  • 2
    If the cmplexity of symbolic computations becomes so high then a noteworthy question arises: What can be read off from the results? What insights have you gained in the end if the resulting formulae are not human readable? It might or might not be a good strategy to convert symbolic computations to numerical ones in machine precision numbers (these can be processed faster) and to study the resulting quantitive data. Anyways, Lowe's advice still holds in 2019: "Save early, save often." DumpSave might be your friend. – Henrik Schumacher Feb 08 '19 at 09:43
  • well, at the very end we will do some numerical analysis. But first we have to get there by doing things "analitically". Thanks for DumpSave! – giulio bullsaver Feb 08 '19 at 09:51
  • 3
    "But first we have to get there by doing things "analitically"." I am sorry to say that, but that is a a common misconception. My experience is that number crunching on unsimplified expression (using Compile) is not seldomly much faster than doing the same with expressions that first have been simplified with enormous effort. But of course, this depends on the nature of the actual problem. – Henrik Schumacher Feb 08 '19 at 10:14
  • 1
    If on a unix-based system, I find that free -m is useful for predicting when I am likely to bring my machine to a grinding halt. Less clear is what will let one know when a kernel might be about to crash. Might want to use memoryConstrained to see if that helps at least to prevent the crashes. Also might want to try simplifying and replacing subexpressions first, if possible. – Daniel Lichtblau Feb 08 '19 at 17:40
  • If possible, avoid getting to the point of having a 400MB expression. Try to simplify expressions earlier in the overall computation. If this is not possible, using Collect together with Simplify, as described here, sometimes is helpful. – bbgodfrey Feb 09 '19 at 05:13
  • This question is too vague. Please provide a summary of the research problem you are working on. Then people in your field who have also worked on similarly large scale problems and who read this question can better assist you. – QuantumDot Feb 10 '19 at 20:39

0 Answers0