Is there a smart way to get a consistent accuracy with fp package?

Question

Is there a smart way to get a consistent accuracy with fp package? I don't want to use round for sure!

enter image description here

\documentclass[preview,border=12pt]{standalone}

\usepackage[nomessages]{fp}
\FPeval\Sixty{trunc(cos(pi/3):12)}
\FPeval\ThreeHundred{trunc(cos(5*pi/3):12)}
\usepackage{amsmath}

\begin{document}
$\!
\begin{aligned}
    \cos 60^\circ &= \Sixty\\
    \cos 300^\circ &= \ThreeHundred
\end{aligned}
$
\end{document}

LaTeX3's \fp_eval:n returns 0.5000000000000001 for both. Does really a difference in the 12th decimal digit bother you? — egreg, Apr 26 '13 at 18:15
Alternatively, if you're only using trig functions, and if the first quadrant results are always the most accurate, you may be able to write your own wrapper functions to convert angles into their first quadrant equivalents, evaluate the core trig function, and then change the sign on the result as necessary before returning. — Mike Renfro, Jun 10 '17 at 12:53

score 7 · Accepted Answer · edited Jun 12 '17 at 12:10

In your case the calculation of cos(5*pi/3) is done with higher precision, but the choice of trunc is worse than round:

\documentclass[preview,border=12pt]{standalone}

\usepackage[nomessages]{fp}
\FPeval\Sixty{trunc(cos(pi/3):12)}
\FPeval\ThreeHundredTrunc{trunc(cos(5*pi/3):12)}
\FPeval\ThreeHundredRound{round(cos(5*pi/3):12)}
\FPeval\Tmp{cos(5*pi/3)}
\FPeval\TruncDelta{\Tmp-\ThreeHundredTrunc}
\FPeval\RoundDelta{\Tmp-\ThreeHundredRound}
\usepackage{amsmath}

\begin{document}
$\!
\begin{aligned}
    \cos 60^\circ &= \Sixty\\
    \cos 300^\circ &= \ThreeHundredTrunc~\text{(trunc)}
    & \Delta &= +\TruncDelta\\
    \cos 300^\circ &= \ThreeHundredRound~\text{(round)}
    & \Delta &= \RoundDelta
\end{aligned}
$
\end{document}

In this case the final error of the calculation using trunc is 999999 times greater than the variant using round.

Float arithmetic with decimal (or binary) numbers has its limitations, because already rational numbers cannot be expressed with a finite number of digits in the decimal (or binary) system. The typical workaround is using a higher precision in the internal representation and rounding the result. Using truncating makes it worse.

A more domain-agnostic way of phrasing the workaround is: accuracy is different from precision. Computing an answer to precisely 12 decimals will not necessarily give it accurate to 12 decimals if the error in the computation was larger than 1e-12. And since the error in the computation is at least as large as the error in the input, your data should be more accurate than you expect your result to be. Hence, compute with more than 12 decimal places. — Ryan Reich, Apr 26 '13 at 18:52

score 5 · Answer 2 · answered Apr 26 '13 at 18:19

Maybe LaTeX3 is an option. Related to the documentation of l3fp:

\documentclass[11pt]{article}
\usepackage{expl3,l3fp}
\usepackage{xparse, siunitx}
\ExplSyntaxOn
\NewDocumentCommand { \calcnum } { m }
  { \num [  round-mode=places , round-precision=2 ] { \fp_to_decimal:n {#1} } }
\ExplSyntaxOff


\begin{document}
\[\cos(60)=\calcnum { cos( pi/3 ) }\]

\[\cos(300)=\calcnum { cos( 5*pi/3 ) }\]
\end{document}

Is there a smart way to get a consistent accuracy with fp package?

2 Answers2