Integer and floating point accuracy in LaTeX3

Question

The L3fp package proposes the range of randint parameters to be +- 10^16 - 1; yet, I seem to be restricted to +-2^31 - 1, any value above it produces the Number too big compilation error. On the other hand I am getting the expected full 16-decimal digit accuracy for fp values. Why?

\documentclass{article}
% RN. 15 April 2017
% BRIEF DESCRIPTION:
%=======================
\usepackage[check-declarations]{expl3}
\usepackage{xparse}
%-----------------------
\ExplSyntaxOn
\int_new:N \l_rn_someInteger_int
\fp_new:N \l_rn_someFp_fp
\NewDocumentCommand\mySetInteger{m}
  {
    \int_set:Nn \l_rn_someInteger_int {#1}
    some~integer:~\int_use:N \l_rn_someInteger_int\\
    \int_set:Nn \l_rn_someInteger_int {\fp_eval:n {randint(#1)}}
    some~random~integer:~\int_use:N \l_rn_someInteger_int\\
    \fp_set:Nn \l_rn_someFp_fp {\fp_eval:n {rand()}}
    some~random~real:~\fp_use:N \l_rn_someFp_fp\\
    -------------------------------------------\\
  }
\ExplSyntaxOff
%-----------------------
\begin{document}
    \mySetInteger{1234}
    \mySetInteger{2147483647}

%   \mySetInteger{2147483648}
%   \mySetInteger{9999999999999999}
\end{document}

@Henri Menke et.al. The question as such was not what maximum integer can be stored in a LaTeX counter, but how to achieve the range +- 10^16 - 1 as parameters for the L3fp randint() function, and that questions had been answered by @egreg: use the xfp package and operate with fp variables rather than with ints. — Reinhard Neuwirth, Apr 15 '17 at 09:32

score 7 · Accepted Answer · answered Apr 14 '17 at 23:05

We can look at how l3fp stores a floating point expression:

\documentclass{article}
\usepackage{xfp}

\begin{document}

\ttfamily

\ExplSyntaxOn % we want to do tests

\fp_set:Nn \l_tmpa_fp { randint(10^15,10^15+10^12) }

\fp_eval:n { \l_tmpa_fp }

\par

\cs_meaning:N \l_tmpa_fp

\ExplSyntaxOff

\end{document}

In one experiment, I get

that shows the random integer is not stored as an integer in TeX's original meaning, because the range is limited to the range –2³¹ to 2³¹–1.

One simply can't assign an integer variable a value outside the above mentioned range.

Operating with “floating point integers” is subject to the standard limitations of floating point arithmetic, when operations are performed.

Similarly, after changing randint to rand, I got

The number is stored with the exponent and four groups of four digits for the significand. The two internal functions \s__fp and \__fp_chk:w are used for manipulating (expandably) the number. A terminator ; ends the internal representation.

So, integers in the range -2^15 ... +_2^15 - 1, but package xfp allows me to work with fp variables that look like integers. I plugged your solution into my code and it does everything I need. — Reinhard Neuwirth, Apr 15 '17 at 01:55
@ReinhardNeuwirth As long as you don't perform operations that overflow –10^16..10^16, you can trust those integers. — egreg, Apr 15 '17 at 07:40

score 4 · Answer 2 · answered Apr 14 '17 at 22:42

The number 2147483648 is 2^31 exactly and \int_... variables are TeX count registers actually, which have a 'limited' number range, as have usual LaTeX counters, which is - 2^{31} to 2^{31} - 1, exactly 2^32 numbers.

If you look into the .log file of a file with expl3 loaded, you will see that the \int... macros are \countXYZ definitions actually.

Trying to store 2147483648 will generate an overflow as \setcounter{foo}{2147483648} would do as well.

Floating point numbers are stored differently as dimension registers and allow larger numbers, but the accuracy isn't better.

Please have a look on What is the maximum integer that can be saved in a LaTeX counter? as well.

would you agree though that package xfp does provide extended accuracy for fp numbers? — Reinhard Neuwirth, Apr 15 '17 at 02:10
@ReinhardNeuwirth: No, not really. xfp is intented to be a wrapper for the \fp_eval:n features from expl3 (or better: l3fp) — , Apr 15 '17 at 07:29

Integer and floating point accuracy in LaTeX3

2 Answers2

Linked