1

I've already plowed through the documentation and a bunch of other Questions on this, but I've only found partial or tangentially related answers.

I need numerical results for all of the complex roots of high-order polynomials with real coefficients. But Solve[] give really bad results, for example:

z[n_, c_] := If[n > 1, z[n - 1, c]^2 + c, c];
expr = z[8, c] // Expand; (*Example polynomial*)
numericRoots = Solve[expr == 0, c] // N;
expr /. numericRoots // Abs // Max

The result should be zero, but the numerical solutions numericRoots are so far off that the result is:

(* 1.5218*10^34 *)

Because of the high orders, I need to MMa to return the most precise results possible and express them as precisely as possible. I thought this was the way:

r = SetPrecision[NSolve[1 + 2*c^2 + 3*c^3 + 3*c^4 + 3*c^5 + c^6 == 0, c, WorkingPrecision -> Infinity], Infinity]

But that doesn't actually solve the polynomial for some reason:

(* {{c -> Root[1 + 2 #1^2 + 3 #1^3 + 3 #1^4 + 3 #1^5 + #1^6 &, 1]}, {c -> Root[1 + 2 #1^2 + 3 #1^3 + 3 #1^4 + 3 #1^5 + #1^6 &, 2]}, {c -> Root[1 + 2 #1^2 + 3 #1^3 + 3 #1^4 + 3 #1^5 + #1^6 &, 3]}, {c -> Root[1 + 2 #1^2 + 3 #1^3 + 3 #1^4 + 3 #1^5 + #1^6 &, 4]}, {c -> Root[1 + 2 #1^2 + 3 #1^3 + 3 #1^4 + 3 #1^5 + #1^6 &, 5]}, {c -> Root[1 + 2 #1^2 + 3 #1^3 + 3 #1^4 + 3 #1^5 + #1^6 &, 6]}} *)

I can get numerical results by doing this:

c /. r // N

But I'm not sure that all those high-precision instructions I made in the NSolve[] command above actually made it into these numbers. This isn't a big issue for 6th order polynomials, but it becomes one for 1000th order or so.

So, why didn't my NSolve[] command solve what I wanted it to solve? And how can I get the highest-precision numerical roots I want?

The code in my first example (which is supposed to return 0) is a good test of whether or not a proposed solutions is working right.

Thanks!

Jerry Guern
  • 4,602
  • 18
  • 47
  • 1
    The Root objects are exact. You can't get higher precision than that. You can get lower precision, though, with N[roots, 30] etc., as you probably know. – Michael E2 Oct 31 '15 at 22:21
  • @MichaelE2 Yes, that is an accurate description of my problem. I expected NSolve to numerically solve for roots, not return unsolved Root objects. – Jerry Guern Oct 31 '15 at 22:25
  • 1
    Mathematica thinks it has returned numeric solutions, e.g. NumericQ[Root[1 + 2 #1^2 + 3 #1^3 + 3 #1^4 + 3 #1^5 + #1^6 &, 1]]. -- Or maybe, if I ask this way, I will understand what you want: If a root is, say, Sqrt[2], what answer do you want to get instead of the unsolved square root? -- Or maybe this would clear things up: There is no greatest finite precision, and to get infinite precision is the same as to get an exact answer, i.e. Root[..]. – Michael E2 Oct 31 '15 at 22:32
  • @MichaelE2 I'm not sure what's unclear. I want numerical results, calculated a accurately as possible, expressed as precisely as possible. I put the infinities in my command because I assumed that would just get me highest-possible finite precision. – Jerry Guern Oct 31 '15 at 22:50
  • @MichaelE2 Well, Sqrt[2] is not a valid example, because high-order polynomials don't generally have exact solutions. So high-precision numerical answers and unsolved Roots are the only possible results. – Jerry Guern Oct 31 '15 at 22:56
  • 2
    As I said, there is no highest-possible finite precision, except for system limitations (i.e. memory). For instance, M will calculate ten million digits of the first root in about 7-8 sec. on my machine; to get the next twenty million takes nearly 20 sec. and only 8MB. I think you have to tell M how many you want. (I used N[Root[1 + 2 #1^2 + 3 #1^3 + 3 #1^4 + 3 #1^5 + #1^6 &, 1], 2 10^7]; //AbsoluteTiming) – Michael E2 Oct 31 '15 at 22:59
  • @MichaelE2 I'm evidently not asking my question clearly. I need numerical results not Root objects. I added a a coded example to my question to clarify the issue. – Jerry Guern Oct 31 '15 at 23:08
  • 1
    @MichaelE2 For one thing I need to plot them. To plot the Root[] object on the complex plain, it will have to be solved at some point, and as my first example illustrates Solve[] returns pretty bad numeric results. – Jerry Guern Oct 31 '15 at 23:18
  • Related/duplicate: http://mathematica.stackexchange.com/questions/51098/nsolve-for-high-degree-univariate-polynomials/51257#51257 – Michael E2 Nov 01 '15 at 00:23
  • 3
    (1) As @JohnDoty noted in response, the roots are numerically fine. Only the residuals are seemingly amiss. In general the roots can be fine to machine precision (as these are) and you might still get bad residuals from cancellation error, even when using a nicer form of the function. – Daniel Lichtblau Nov 01 '15 at 02:31
  • 2
    (2) The Root objects are returned by Solve, not by NSolve. – Daniel Lichtblau Nov 01 '15 at 02:32
  • 3
    (3) the Root objects in the indicated result are in no sense "unresolved". They encode numeric approximations, isolating intervals, and can be refined (via N[...,...]) to arbitrarily high precision, subject only to computer and bignum numerics limitations. – Daniel Lichtblau Nov 01 '15 at 02:34
  • 2
    Back to my item (2), I see the Root objects were also given by infinite precision NSolve. Suffice it to say that infinite precision NSolve is, to good approximation, another exact version of Solve`. – Daniel Lichtblau Nov 01 '15 at 02:36
  • 3
    (4) The remark "The code in my first example (which is supposed to return 0) is a good test of whether or not a proposed solutions is working right." is some combination of correct, incorrect, and seriously incorrect. One must account for scaling, along with cancellation and truncation error, when testing residuals for being zero. The general rule is that you should not compare approximate values to zero unless you know good and well what the result indicates. In this instance it indicates only that truncation and maybe cancellation error are rearing their heads.... – Daniel Lichtblau Nov 01 '15 at 02:44
  • 2
    ... To understand what truncation error can do in a case like this, noting that the largest norm of a root is around 2, look at this. `In[123]:= Expand[(2 + eps)^100] /. eps -> 10.^(-16)

    Out[123]= 1.26765*10^30`.

    – Daniel Lichtblau Nov 01 '15 at 02:46
  • @DanielLichtblau Thank you for your insights. But I think that the residual I'm getting here is about 20 orders of magnitude too large to be explained by truncation. Could you weight in on the discussion below JohnDoty's Answer? – Jerry Guern Nov 01 '15 at 05:51
  • "I think" and "what actually happens" can be quite different, especially if you're not familiar with the many pitfalls of inexact arithmetic. – J. M.'s missing motivation Nov 01 '15 at 05:53
  • Sorry, I gave an incomplete example above. Here is better one. This time I use the expanded polynomial, subtracting the valuation thereof at 2 from the same evaluated at 2+epsilon. I then evaluate numerically when epsilon is an ULP (thereby showing a reasonable approximation of what truncation error can do here). `In[133]:= Expand[(expr /. c -> 2 + eps) - (expr /. c -> 2)] /. eps -> 2.^(-53)

    Out[133]= 2.08111*10^36`

    – Daniel Lichtblau Nov 01 '15 at 17:30

1 Answers1

11

Your numericRoots are OK. The core of your problem is that you use Expand to put your polynomial into power series form. High order power series are numerically unstable. The form your z function yields by itself is much better:

z[8, c] /. numericRoots // Abs // Max

(* 9.87841*10^-11 *)

Expanding on my answer here:

"I can see that what you're saying is correct, but it leaves me even more confused than before. z[8,c] and z[8,c]//Expand are exactly the same because the coefficients are integers."

But once you apply one of your numericRoots rules, the expressions are not exactly the same. Your rules substitute "machine numbers" for c. When you do mixed arithmetic with integers and machine numbers, you wind up with machine numbers. Machine arithmetic is sensitive to the order of operations. Mathematically equivalent expressions that are different in form will generally yield different results in machine arithmetic.

terms = expr /. Plus -> List

(* large expression that looks like an alluvial fan *)

You can see that while the coefficients are indeed integers, some are very large, too large for accurate representation as machine numbers. Then, even though the roots are modest, you're raising them to high powers, so the numbers you're adding are very large indeed:

terms /. numericRoots[[9]] // Abs // Max

(* 3.02339*10^49 *)

When you're adding a bunch of numbers of this magnitude in machine precision (~16 decimal digits), a residual of 10^34 is about as close to zero as you can reasonably expect.

"if simply changing the polynomial to a different but equivalent form creates a residual that humongous, how can I have any confidence in the root Mathematica gave me?"

Computation using machine numbers is fast, but (like other programming systems) Mathematica does not track the resulting precision. Sometimes, as in your formulation, the result contains no significant digits. There are, however, ways you can check.

One way is to use exact roots:

roots = Solve[expr == 0, c];

Don't look at them: they are mostly "algebraic numbers" expressed as complicated Root objects. But they are true, precise numbers. They are expressed in notation that is unfamiliar to most, but there is no general way to express algebraic numbers in familiar notation. Be happy that Mathematica can do this: in some situations it's very powerful. Mathematica is a bit lazier about computing with general algebraic numbers than with other kinds of numbers, but it can here if you coax it, and wait a few minutes:

Simplify[expr /. roots] // Abs // Max

(* 0 *)

So Mathematica asserts that its results are perfect ;-)

Another approach is to use arbitrary precision instead of machine precision:

numericRoots = N[Solve[expr == 0, c], 50];
expr /. numericRoots // Abs // Max // FullForm

(* 0``-2.568868432106492 *)

In arbitrary precision arithmetic, Mathematica tracks the precision of the result. Here, the result is 0, but with an accuracy of -2.56... decimal digits, so it might be 300 or so. That's a lot better than 10^34, and Mathematica informs you of the accuracy. You can do much better than a precision of 50 for your roots, of course:

numericRoots = N[Solve[expr == 0, c], 1000];
expr /. numericRoots // Abs // Max // FullForm

0``947.4311315678933

The residuals are zero to better than 947 decimal places.

John Doty
  • 13,712
  • 1
  • 22
  • 42
  • I can see that what you're saying is correct, but it leaves me even more confused than before. $z[8,c]$ and $z[8,c]//Expand$ are exactly the same because the coefficients are integers. So how on earth can one have a residual of order 10^-11 and the other 10^+34? And if simply changing the polynomial to a different but equivalent form creates a residual that humongous, how can I have any confidence in the root Mathematica gave me? And what if I only had the polynomial to work with, not the un-Expanded z[n_,c_] form? Sorry my response isn't more coherent, but your Answer kinda blew my mind. – Jerry Guern Nov 01 '15 at 05:44
  • Also, it seems like something much deeper has to be wrong for me to get a residual of order 10^+34. That's 20 orders of magnitude larger than the coefficients of z[8,c], much to large to be caused by a small error in the root. – Jerry Guern Nov 01 '15 at 05:47
  • @Jerry: "if simply changing the polynomial to a different but equivalent form creates a residual that humongous" - using a different form makes a lot of difference in numerical computing. Try plotting $(x-1)^{20}$ in its expanded and factored forms, and you are likely to see two different pictures. – J. M.'s missing motivation Nov 01 '15 at 05:52
  • @J.M. Ah, okay, that was helpful. I still think, though that 10^34 is too large a residual to attribute to truncation, because the sum of the coefficients in z[8,c]//Expand is only of order 10^22. Am I wrong about that too? – Jerry Guern Nov 01 '15 at 06:02
  • 1
    @Jerry, people have gotten into unluckier situations than yours due to catastrophic cancellation in the expanded form. Your polynomial is... specially structured; if you don't mind a bit of a wait, I'll write a detailed answer later. For now: instead of Expand[], try using HornerForm[]. – J. M.'s missing motivation Nov 01 '15 at 06:06
  • I'm getting a definite sense here that my problems go deeper than how to coax more decimal places out of Mathematica. – Jerry Guern Nov 01 '15 at 06:06
  • @Jerry, "go deeper" - in a sense, yes. – J. M.'s missing motivation Nov 01 '15 at 06:07
  • @J.M. I tried that, but (z[8, c] // HornerForm) /. numericRoots // Abs // Max returns almost as bad a residual as //Expand. For some reason, z[8, c] /. numericRoots // Abs // Max returns the best results, even though I used the Expanded form in the Solve[]. Very confusing. – Jerry Guern Nov 01 '15 at 06:22
  • 1
    @Jerry, "z[8, c] /. numericRoots // Abs // Max returns the best results": it's not the expanded form with the risk of cancellation looming over it, so I'd expect that. But in that case, you definitely will want to wait for my post later, if Horner doesn't do the job... – J. M.'s missing motivation Nov 01 '15 at 06:42
  • To better understand the truncation error effect here, consider the following. The degree 88 term has coefficient 14642200658326619088. Take a root, call it r. Truncation error means that N[r] can be off by a unit in the last place (ULP), (for machine precision that amounts to about 2^(-53)*r). To gauge the effect on that term alone, consider the difference, expanded, of 14642200658326619088*(r+epsilon)^88 and 14642200658326619088*r^88. Use the binomial theorem to expand that first and you start to see where big error values can appear. – Daniel Lichtblau Nov 01 '15 at 17:39
  • 1
    Just for fun, I took the decimal machine number representation of Mathematica's ninth root (the toughest, c->-1.9997740486937272) and fed it to Python versions of z[8,c] and expr. Python does a little better on the residuals here, but not a lot. Using z[8,c], the residual is -1.2048140263232199e-12, but for expr it's 7.03031399207685e+33. – John Doty Nov 01 '15 at 23:17
  • @JohnDoty The slight difference might be due to the binary-to-decimal-to-binary you introduced, assuming you fed in that decimal representations shown above. – Daniel Lichtblau Nov 01 '15 at 23:23
  • @Daniel yes, very possible. Or some other fiddly implementation detail that makes Python look slightly better in this case. But qualitatively, it's the same as MMA, as expected. – John Doty Nov 01 '15 at 23:41
  • Thank you, everyone, for this great discussion. The help I got here when far beyond MMa coding, and I learned some really important lessons about numerical problems from the discussion. – Jerry Guern Nov 13 '15 at 21:46
  • Heya, @JohnDoty, I posted another Question related to this, where this solution didn't work (or I did it wrong). Care to have a look? Thanks – Jerry Guern Jan 25 '16 at 20:02