How is Precision of Imported Data managed?

Question

I am working on a financial/banking type application and part of the work is to import numeric data of 15 to 20 digits of precision. I am having trouble determining the best way this should be done. Here is an example highlighting my problem.

Imported data is a CSV file, the example below shows three comma-separated numbers:

1.00000000000001, 2.1234567890123456789,3.01

Using the statement:

x = Import["precisionTest.csv"];
x
{{1.,2.1234567890123456789,3.01}}

Testing the precision of each list item:

Precision[x[[1]][[1]]]
MachinePrecision

Precision[x[[1]][[2]]]
19.327

Precision[x[[1]][[3]]]
MachinePrecision

So, in my list, my precise number 1.00000000000001 is truncated to 1.0, where as the second number's high precision is honored by the input, and the third number is correctly interpreted fitting within MachinePrecision. This stumped me. Why did the first number, 1.00000000000001, get truncated whereas the second one did not. I would expect the same behavior for both.

I redid my import but first I set the \$MinPrecision constant to 20 yet I got exactly the same results. Maybe this failed for reasons of using $MinPrecision incorrectly.

[More] I noticed that using the AccountingForm function on my list x reclaimed the correct value, that is:

AccountingForm[x,20]
{{1.00000000000001,2.1234567890123456789,3.01}}

So, I get that for display only I guess -- But, why is the first number truncated (as I asked earlier) whereas the second one is left with full precision to represent the imported number?

I posted an answer attempting to example a little of what is going on here but I infer an underlying problem you wish to solve. Would you explain how you would actually like this data to be handled in Mathematica? — Mr.Wizard, Jul 20 '15 at 16:58
part of the issue is one of display, if you do RealDigits[x[[1]][[1]]] you'll see your 10^-14 digit is there. — george2079, Jul 20 '15 at 17:01
interesting, for me (v9) AccountingForm as well as NumberForm round the display of the second entry so it ends in 679 .. The most reliable way to see what you really have seems to be InputForm or FullForm — george2079, Jul 20 '15 at 17:12
Mr.Wizard Your answer, plus the comments, and related links posted are helpful and probably will aid me in finding the right solution. What is that right solution? I need to import data, perform arithmetic, call functions, and so on, without fear of losing precision somewhere along the line with numbers that easily have 12 to 15, maybe up to 20 digits. In many of the calculations, actual result data is rarely more than 12 digits in length but intermediate values could easily be 15 to 20 digits (I think). And, some of the applications are external processes, thus my use of Import. — K7PEH, Jul 20 '15 at 17:45
Continuing. Ideally, I would have a data type to represent money type data and supporting arithmetic. My guess is that I can fashion together that behavior using various functions available with Mathematica. I do admit to be somewhat surprised with the results I posted and how the numbers were treated differently such as why was the second number not truncated down to the display precision. My guess is that if the number is within $MachinePrecision, it follows the default display precision format. If it is not then it uses the precision necessary for display. — K7PEH, Jul 20 '15 at 17:52
One of my first contract programming jobs about 47 years ago was with an insurance company. The programming language was Cobol. As objectionable as many see Cobol, it has a few redeeming features such as the way it handled money and maintaining fractional accuracy and handling that correctly in all computations. Actually, RPG supported a similar money computation feature. By the way, that was the only time and last time I programmed using Cobol. — K7PEH, Jul 20 '15 at 17:59
Looking at what you are trying to do, your best approach may be to import the data as strings and parse them into some exact representation (rational, lists of digits, integers, etc). You will have to work out how to do the arithmetic, but then you will have full control over how precision is handled. — george2079, Jul 20 '15 at 18:11
@george2079 If that is what would be required (your suggestion) then I would probably abandon Mathematica as my calculation engine for this particular application. From what I have learned so far (answers here), I think the methods available will help me do what I need to do in Mathematica. — K7PEH, Jul 20 '15 at 18:13
@K7PEH Is it necessary for every number to have its precision set to exactly the number of digits that appear, or can you artificially increase the precision of every number to a certain value, e.g. 20? i.e. does "3.01" need a precision of 3 or can it be represented as "3.0100000000000000000"? — Mr.Wizard, Jul 21 '15 at 02:28

score 4 · Answer 1 · edited Apr 13 '17 at 12:55

The second number has too many digits to fit into machine precision therefore it is automatically cast to arbitrary precision. I do not believe this is specifically related to importing, e.g.:

$MachinePrecision

a = 1.00000000000001;
b = 2.1234567890123456789;
c = 3.01;

Precision /@ {a, b, c}

15.9546
{MachinePrecision, 19.327, MachinePrecision}

Now consider the description of $MinPrecision:

$MinPrecision gives the minimum number of digits of precision to be allowed in arbitrary-precision numbers.

Definitionally this affect not machine-precision but arbitrary-precision numbers:

low = 0.0712`3;
low // Precision

3.

$MinPrecision = 10;
low // Precision

Precision::mnprec: Value 3.` would be inconsistent with \$MinPrecision; bounding by \$MinPrecision instead. >>
10.

Regarding the apparent truncation of the first number and its reclamation please see:

Annoying display truncation of numerical results

An example:

a

Style[a, PrintPrecision -> 20]

1.
1.00000000000001

How is Precision of Imported Data managed?

1 Answers1