8

Possible Duplicate:
Funny behaviour when plotting a polynomial of high degree and large coefficients

1/x^2 + (3 + x)/(6 (1 - Exp[x] + x))

——This is a expression that seems nothing special, so does its limit when x->0:

Limit[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)), x -> 0]
(* => 1/12 *)

But it becomes strange when trying to calculate a x near zero,

1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)) /. x -> 0.00001
(* => -19403.7 *)
1/0.00001^2 + (3 + 0.00001)/(6 (1 - Exp[0.00001] + 0.00001))
(* => -19403.7 *)
Limit[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)), x -> 0.00001]
(* => 0. *)

Many people (including me) may check the curve of the expression as the first reaction for these results:

Plot[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)), {x, 0, 0.00001}]

And see:

A curve under low precision

Aha, it explains the matter! The curve is actually oscillating near zero!… but wait, try this:

Plot[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)), {x, 0, 0.001}, WorkingPrecision -> 30]

Now we see:

A curve under high presision


So all of these turn out to be a story about precision once again (why I always encounter this! ). I also found some resource for this error:

Series[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)), {x, 0, 10}];
Normal[%] /. x -> 0.00001
(* => 0.0833332 *)

N[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)) /. x -> 10^-5, 6]
(* => 0.0833332 *)

But, can this problem be solved only with ReplaceAll and Limit? Also, we see that Mma doesn't give warnings for these great error (while this sort of thing frequently occurs when I use NDSolve though the error seems not that big…it's another story), any way to make Mma give a warning message or something?

xzczd
  • 65,995
  • 9
  • 163
  • 468
  • Look at this function 2 - 2*E^x + 2 x + x^2 which is part of your expression when you simplify it. Note to which degree the numerical values of the 4 parts in the sum differ. Didn't you have numerical error analysis in university? – halirutan Sep 19 '12 at 11:55
  • 1
    The last sentence in my previous comment sounds harsh. It is not meant like that! It's just a hint where you could start looking in your lecture notes or when searching the net. I don't know how to say this in a different way.. I'm German. Bratwurst and Rucksack, you know.. – halirutan Sep 19 '12 at 11:57
  • Yes, the $1+x-\exp,x$ portion of your function would cause trouble, by virtue of subtractive cancellation... – J. M.'s missing motivation Sep 19 '12 at 12:01
  • 1
    @halirutan Haha, never mind, and in fact I really don't have numerical error analysis in my university…at least it's not a required course for my major and in my memory I never see this in the list of elective course, too… our lessons always talk about those analytical solutions and say little about numeric solutions, usually the books just say "the numeric solution should be done with computer". So, can you explain the details? – xzczd Sep 19 '12 at 12:21
  • I'm not ready to write an exhaustive answer. But try yourself to investigate in the following: x = 10^-7``15;{Precision[x], Precision[2 - 2*E^x + 2 x + x^2]}. This should give you a start. And as @J.M. ensured me, there was already a Q&A about the different forms of numbers and precision tracking. – halirutan Sep 19 '12 at 12:23
  • 3
    And thanks to @J.M. who found this answer. – halirutan Sep 19 '12 at 12:49
  • 2
    @xzczd: The basic principle is this: If you only have a certain number of digits of precision, you'll lose something. Consider for example the expression $10^{100}(1+10^{-100}-1)$. If you have it as analytical formula, it's obvious that the result is $1$. But if you calculate e.g. to 5 digits, then 1.0000+1.0000e-100 = 1.0000 since the 1.0000e-100 is rounded away. Then subtracting 1.0000 again gives 0.0000 and multiplying with 1.0000e100 won't change that. – celtschk Sep 19 '12 at 15:09
  • 1
    Related: http://mathematica.stackexchange.com/q/3152/121 – Mr.Wizard Sep 19 '12 at 15:33
  • 1
    @Mr.Wizard you are joking, right? Related? It basically is the answer. Why haven't I found this earlier then I could have just linked your answer. At least you got now a +1 from me ;-) – halirutan Sep 19 '12 at 15:36
  • It would be nice to see this thread merged with the one referenced by @Mr.Wizard, so that we can maintain a canonical set of answers to questions of numerical imprecision. Surely this will crop up again... – whuber Sep 19 '12 at 18:09
  • @whuber Merge? You mean shape two as one? It is possible? That would be good! I have no objection. – xzczd Sep 20 '12 at 05:39
  • @celtschk So a simpler example can be, 1.00+0.00100 is calculated to 3 digits, then the answer is 1.00, right? If so, OK…I know this, though my lesson say little about numeric error analysis it's still involved in the little, but I don't expect it also exists in mathematica 囧. – xzczd Sep 20 '12 at 05:41
  • 1
    @xzczd: The point is that such a small error can be blown up arbitrarily, as in my example (where instead of 1 you get 0). If you only care about 3 digits, 1.00 instead of 1.001 doesn't seem a big deal, but 0 instead of 1 surely is. The relevant part is the difference between 2 values of approx. the same size. This is what blows up the error (my example was the extreme of losing the complete value). And since by default M. uses builtin floating point, the issue also exists there (it also exists with arbitrary precision numbers, but there M. can at least warn about complete loss of precision). – celtschk Sep 20 '12 at 06:55
  • @celtschk So we can simply regard MachinePrecision as something difficult to handle and always avoid it and choose a exact precision or rationalize the number will be better when having numeric calculation? By the way, it really took me a while to notice M. is for mathematica :D. – xzczd Sep 20 '12 at 07:36
  • 2
    @xzczd: The problem is that exact arithmetcs is often orders of magnitude slower than machine arithmetics (and in many cases, an exact solution will not even be possible where numerics gives good results). I think the right thing to do is to identify key parts where precision may be needed (look out for those sums and differences) and only rewrite those to either use more precise (or, where possible, exact) methods, or to avoid lossy operations by rewriting the functions in question (for example, by using the series close to a critical point, as you did in your last block). – celtschk Sep 20 '12 at 09:19
  • 1
    It would be nice to have some automatic functionality for that, though (say, a function Numericize which takes an expression, and returns another expression which calculates the original expression in a numerically accurate way for numeric values for all variables, but isn't necessarily exact for exact values). BTW, the "M." for Mathematica was due to comment length; I didn't want to add a second comment for just a few characters, so I decided to shorten what I can. I didn't expect the meaning to be that hard to derive. Obviously my expectation was wrong. – celtschk Sep 20 '12 at 09:25
  • @celtschk That's all because that we already have a MachinePrecision in this talk, and, I think, to have two comments won't be a bad thing, since I can give you two upvote then XD. – xzczd Sep 20 '12 at 10:51

1 Answers1

11

Lets summarize my comments into an answer.

When you inspect your expression a bit more you can see that you can drop some parts and still get an expression that's causing trouble:

expr = Numerator[
  Together[Most@ExpandAll[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x))]]]

(*
  2 - 2 E^x + 2 x + x^2
*)

Plot[expr, {x, 0, 10^-6}]

Mathematica graphics

Lets have a look what happens when we work with expr. First, we can just insert a numerical value which has machine precision and where the errors that happen are not traced.

expr /. x -> 4.*^-7
(*
  Out[22]= 1.0488*10^-16
*)

By using the . in the number we say please use the machine arithmetic. We don't know how reliable this result is but we can use Mathematicas arbitrary-precision arithmetics to get a n answer where we know how reliable it is. You have several options to input numbers for arbitrary-precision. If you give more digits than a machine-precision number has then Mathematica switches automatically

x1 = 4.00000000000000000*^-7;
Precision[x1]
expr /. x -> x1
Precision[%]

(*
Out[100]= 17.6021

Out[101]= -2.133*10^-20

Out[102]= 3.727
*)

See how the precision drops due to the evaluation of the expression! Usually, arbitrary-precision numbers are assembled with backticks. For instance 4`20*^-7 gives a number with precision 20. Please have a look at this tutorial and all the other pages about numerics in the documentation of Mathematica.

The most used function for numerical evaluation is maybe N. Here you can input an exact expression and evaluate it to whatever precision you like. Note that I use an exact number 4*^-7 with $\infty$ precision as input.

N[expr /. x -> 4*^-7, 20]
Precision[%]
(*    
Out[107]= -2.1333335466666837333*10^-20

Out[108]= 20.
*)

Therefore, N calculated this expression having a precision of 20 in the end. All this can now be applied to your Limit example

N[Limit[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)), x -> 10*^-6], 15]
Limit[1/x^2 + (3 + x)/(6 (1 - Exp[x] + x)), x -> 10.`35*^-6]
(*
Out[145]= 0.0833332222221759

Out[146]= 0.08333322222217592601410947237
*)

Or of course the Plot. Read the next example carefully. The first one says use machine numbers, the second one says use arbitrary-precision numbers with a precision of $MachinePrecision

Plot[1/x^2 + 3/(6 - 6*E^x + 6*x), {x, 0, 1.*^-6}, 
   WorkingPrecision -> #] & /@ {MachinePrecision, $MachinePrecision}

Mathematica graphics

I'm not sure whether I really answered all of your questions and whether every detail is correct to the point. I kind of left my circle of competence here. I know numerics and how to watch out for errors, but I'm far from being an numerical analyst.

halirutan
  • 112,764
  • 7
  • 263
  • 474
  • 1
    To see why 2 - 2 E^x + 2 x + x^2 is so troublesome to evaluate numerically (due to subtractive cancellation), consider the Maclaurin series for the exponential function... – J. M.'s missing motivation Sep 19 '12 at 15:20
  • I've read the answer and the links given by J.M and @Mr.Wizard. Let me try to retell it to see if I've understand it: The error occurs because when doing numeric calculations Mma will keep the precision step by step i.e. several steps of approximation have been done before the final result comes out. – xzczd Sep 20 '12 at 05:43
  • If so, well, it's a little different from my imagination, since I think the approximation will only be done in the last step and the precision before the last step will be as high as possible (in my mind it almost equals to infinity…), just like what I turn to do when calculating something manually…and now I notice that my imagination is unreasonable. – xzczd Sep 20 '12 at 05:43