2

A number like 0.644696875 is represented internally as 0.6446968749999...:

N[FromDigits[RealDigits[0.644696875, 2], 2], $MachinePrecision]
(* 0.6446968749999999 *)

So if I ask NumberForm to print this number with 8 decimals, I would expect it to be 0.64469687 and not 0.64469688 because the digit after the 87 is a 4 which is less than 5. But it is not what we get with NumberForm:

NumberForm[0.644696875, {8, 8}]
(* 0.64469688 *)

So it looks like we have here two round operations when only one was requested:

  • First Rounding: From 0.6446968749999999 to 0.644696875
  • Second Rounding: From 0.644696875 to 0.64469688

I found this while comparing to Python, which doesn't double round. This leads to a result which I believe is correct: 

ExternalEvaluate["Python", "'{:.8f}'.format(0.644696875)"]
(* 0.64469687 *)

Also notice that this floating point number is stored in the same way in both systems: 

Divide @@ ExternalEvaluate["Python", "0.644696875.as_integer_ratio()"] == FromDigits[RealDigits[0.644696875, 2], 2]
(* True *)

Is Mathematica double rounding? Can this be avoided while still using machine numbers?

Motivation: I'm working on making a hash on an array of floating point numbers. This calculation should be the same from Mathematica and from Python. For this I need to be able to produce the same string representation of numbers in both system. This has proven more challenging than expected.

Update: I think Mathematica is double rounding just like Java does. Please see:

Update2: I asked support about this [CASE:4304365] and they said "It does appear that NumberForm is not behaving properly".

J. M.'s missing motivation
  • 124,525
  • 11
  • 401
  • 574
Gustavo Delfino
  • 8,348
  • 1
  • 28
  • 58

1 Answers1

3

I think NumberForm, being a formatting wrapper, formats the machine number using the available digits that are given to it. For your example, the available digits are:

.644696875 //FullForm

0.644696875`

So, the double rounding you're observing is due to the following:

  1. Mathematica uses a rounded representation of the machine number for display
  2. NumberForm rounds the displayed number

Misconception

I think you have a misconception about $MachinePrecision. When you do

N[FromDigits[RealDigits[0.644696875, 2], 2], $MachinePrecision]

you are actually creating an extended precision object with $MachinePrecision digits, and not a machine number. Compare:

exact = FromDigits[RealDigits[0.644696875, 2], 2];
N[exact, MachinePrecision] //FullForm
N[exact, $MachinePrecision] //FullForm

0.644696875`

0.6446968749999999470645661858725361526`15.954589770191003

Notice how the latter number has digits following the `. This is the signature of an extended precision number. If you want to create a MachinePrecision object with N, you need to use the single argument version, or you need to specify MachinePrecision and not $MachinePrecision.

Workaround

I think you can take your observations and create a workaround by first converting to an extended precision number and then using NumberForm:

myNumberForm[n_?MachineNumberQ, spec__] := NumberForm[
    SetPrecision[n, $MachinePrecision],
    spec
]

Then:

myNumberForm[0.644696875`, {8,8}]

0.64469687

Carl Woll
  • 130,679
  • 6
  • 243
  • 355