PlotRange precision limit?

Question

I'm trying to plot a set of points with 9 digits precision with a properly range labelling Y-axis, from a previous question I got two possible ways of doing this:

1st: suggested by @Carl Woll Using BaseStyle -> {PrintPrecision -> 9} as a option in listplot, but Mathematica just seems to ignore the option:

ListPlot[RandomVariate[NormalDistribution[0.8037709, 10^-10], 20], 
 PlotRange -> {0.80377085, 0.80377095}, 
 BaseStyle -> {PrintPrecision -> 9}]

2nd: suggested by @george2079 using this Ticks options:

ListPlot[RandomVariate[NormalDistribution[0.8037709, 10^-10], 20], 
 PlotRange -> {0.80377085, 0.80377095}, 
 Ticks -> {Automatic, {#, NumberForm[N@#, {12, 9}]} & /@ 
    FindDivisions[{0.80377085, 0.80377095}, 10]}]

Both options work fine if PlotRange has 7 digit precision or less, but don't show good results for 8+ digits range, any ideas how to deal with this?

Answer 1

It's bad habit to plot stuff with a large bias and a small variation. The plotting functions get confused. I'd suggest subtracting the mean and plotting things around 0, and then adding the mean again to the ticks.

mean = 0.8037709;
std = 10^-7;
data = RandomVariate[NormalDistribution[mean, std], 20];
ListPlot[data - mean,
 PlotRange -> {All, {-3 std, 3 std}},
 Ticks -> {Automatic, 
   Table[{y, NumberForm[y + mean, 10]}, {y, FindDivisions[{-3 std, 3 std}, 5]}]
 },
 AxesOrigin -> {0, -3 std}
 ]

Also, the problem you're specifically referring to is probably due to precision issues, i.e. the fact that the precision of the mean is if of the order of the variation, so Mathematica might treat them as the same number. If you use an exact number for the mean you can easily use more digits:

mean = 803770945/1000000000;
std = 10^-15;
data = RandomVariate[NormalDistribution[mean, std], 20];
ListPlot[data - mean,
 PlotRange -> {All, {-3 std, 3 std}},
 Ticks -> {Automatic, 
   Table[{y, NumberForm[y + N[mean, 100], 17]}, {y, FindDivisions[{-3 std, 3 std}, 5]}]
 },
 AxesOrigin -> {0, -3 std}
]

Answer 2

The problem with my BaseStyle->{PrintPrecision->10} suggestion in M11 is that ListPlot ignores the PlotRange option. So, one can instead use Show afterwards to get the right PlotRange. In M11:

Show[
    ListPlot[
        RandomVariate[NormalDistribution[0.8037709,10^-10],20],
        BaseStyle->{PrintPrecision->10}
    ],
    PlotRange->{{0,20},{0.8037708995,0.8037709005}}
]

Answer 3

As of V11, it appears that ticks will print with appropriate precision when the plot range is small. The problem of poorly scaled data may be addressed by rescaling with ScalingFunctions and Rescale.

data = RandomVariate[NormalDistribution[0.8037709, 10^-10], 20];
ListPlot[data,
 ScalingFunctions -> {
   Rescale[#, MinMax[data], {0, 1}] &,
   Rescale[#, {0, 1}, MinMax[data]] &},
 PlotRange ->         (* N.B. PlotRangePadding messes things up! So use manual padding *)
    {{1.05, -0.05},
     {-0.05, 1.05}}.MinMax[data]
 ]

Odd restriction: The above plot was generated by executing the right after executing

SeedRandom[1];
RandomReal[];

If I omit RandomReal[], then the ticks of the plot are rounded to three decimal places, no matter what seed I tried. But if I followed SeedRandom with a Random* call, the ticks would be rendered as above.

---

Alternative: Scale the data and rescale the ticks. This approach works with ListPointPlot3D as well; e.g. it can be used to solve ListPointPlot3D small and large values.

data = RandomVariate[NormalDistribution[0.8037709, 10^-10], 20];
With[{minmax = MinMax[data]},
 ListPlot[
  Rescale[data, minmax, {0.`, 1.`}],
  Ticks -> {Automatic, 
    Charting`ScaledTicks[{Rescale[#1, minmax, {0.`, 1.`}] &, 
      Rescale[#1, {0.`, 1.`}, minmax] &}]}
  ]
 ]

If the data consists of coordinates instead of being a list of values, then one has to process just the last coordinate. One substitute, for both 2D and 3D data,

minmax = MinMax[data[[All, -1]]]

MapAt[Rescale[#1, minmax, {0.`, 1.`}] &, data, {All, -1}]

Ticks -> {Automatic, Automatic, 
  Charting`ScaledTicks[{Rescale[#1, minmax, {0.`, 1.`}] &, 
    Rescale[#1, {0.`, 1.`}, minmax] &}]}