4

I must admit that I am frequently stumbling over the limited numerical precision issues with pgfplots(table). I am aware that \pgfset{fpu} activates the IEEE double precision unit. Moreover, the latest manual (v1.15) states:

This FPU is also part of PgfplotsTable, and it is activated by default for create col/expr ...

With that being said, please help me understand my questions about the following MWE. Take note of the three different expressions for create col/expr={abs(...)}.

\documentclass{article}
\usepackage{pgfplotstable}

\begin{filecontents}{table.csv}
values
392.50271975
392.5024343
392.5016196
392.5005834
392.4992593
\end{filecontents}

\pgfplotstableread[col sep=comma]{table.csv}\myerrortable

\pgfplotstableset{
    create on use/error/.style={
        create col/expr={
            abs(785/2 )%-\thisrow{values})
            %abs(392.5-\thisrow{values})
        }
    },
}


\begin{document}
\pgfplotstabletypeset[
    columns={error},
    columns/error/.style={
        column name={error},
        fixed,
        precision=8,
    },
    ]\myerrortable
\end{document}
  1. abs(785/2 ) = 392.499: Why are only three digits printed? This does not look like the fpu is used.
  2. abs(785/2 -\thisrow{values}) = 0.00372, ...: As you see the values table contains more precision digits and they are partially used. The error is always in the third digit resulting from the limited precision result of 785/2. So, why is 785/2 limited to 3 digits, and the values to 5?
  3. abs(392. - \thisrow{values}) = 0.00272, ...: If the correct result for 785/2 is used directly, the output is now correct for the given 5 precision digits. But why only 5 digits? The values column contains more and I requested 8. I thought fpu gives full double precision?
Frederik Heber
  • 131
  • 1
  • 6
  • I have not accepted Marijn's answer, yet, as I would really like to see an answer to my questions above. Is this a bug in pgfplots(table) or am I doing/understanding something fundamentally wrong? Why are the calculations taking place at two different precision levels? – Frederik Heber Sep 25 '18 at 10:59

1 Answers1

3

As a workaround you can use the expl3 FPU (see, e.g., How to use the TikZ fpu library?).

MWE:

\documentclass{article}
\usepackage{pgfplotstable}
\pgfplotsset{compat=1.15}
\usepackage{expl3}
\ExplSyntaxOn
\cs_new_eq:NN \fpeval \fp_eval:n
\ExplSyntaxOff
\begin{filecontents}{table.csv}
values
392.50271975
392.5024343
392.5016196
392.5005834
392.4992593
\end{filecontents}

\pgfplotstableread[col sep=comma]{table.csv}\myerrortable
\pgfplotstableset{
    create on use/error/.style={
        create col/expr={
            \fpeval{abs(785/2-\thisrow{values})}
        }
    },
}

\begin{document}
\pgfplotstabletypeset[
    columns={values,error},
    columns/values/.style={column name={value},fixed,precision=8,},
    columns/error/.style={column name={error},fixed,precision=8,},
    ]\myerrortable
\end{document}

Result:

enter image description here

Marijn
  • 37,699
  • Thank you, Marijn, for that workaround! This is indeed very useful to get out of my current misery and keep my workflow using pgfplots. – Frederik Heber Sep 23 '18 at 20:29