5

I have the results of a process involving FindDistribution as a Dataset of probability distributions. Attempting to plot the PDFs of these from Dataset I kept getting failures. I eventually narrowed this down to the MixtureDistributions. A minimal example follows.

ds = Dataset@<|
   1 -> MixtureDistribution[{.2, .8}, {UniformDistribution[{-.08, 3. 10^6}], 
         GammaDistribution[.5, 88 10^4]}],
   2 -> ExponentialDistribution[1 10^-6]
   |>

Attempt to create a Dataset of PDF plots from Dataset Query fails.

ds[All,
 With[{pdf = PDF[#, x]},
   Plot[pdf, {x, 0, Quantile[#, .95]}]
   ] &
 ]

Mathematica graphics

Attempt to create list of PDF plots outside of Dataset Query succeeds.

With[{pdf = PDF[ds[#], x]},
   Plot[pdf, {x, 0, Quantile[ds[#], .95]}]
   ] & /@ Range[2]

Mathematica graphics

Quantile succeeds in the query,

ds[All, Quantile[#, .95] &]

Mathematica graphics

but PDF fails,

ds[All, PDF[#, x] &]

Mathematica graphics

This was narrowed to the MixtureDistribution (first row in Dataset and first in list).

ds[#, PDF[#, x] &] & /@ Range[2]

Mathematica graphics

Remove the MixtureDistribution and the Query succeeds.

ds2 = Dataset@<|
   1 -> NormalDistribution[],
   2 -> ExponentialDistribution[1 10^-6]
   |>;

ds2[All,
 With[{pdf = PDF[#, x]},
   Plot[pdf, {x, 0, Quantile[#, .95]}]
   ] &
 ]

Mathematica graphics

May someone confirm.
Mma 11.1 , Win 7 Ent

CASE: 3866316

Edmund
  • 42,267
  • 3
  • 51
  • 143

1 Answers1

4

There are three possible solutions imo to the problem:

General Advice: Use Datatset to display results only

This was already mentioned by @b.gatessucks in his comment and in (87360) gives some deeper insights into why working on Dataset can sometimes fail unexpectedly. So making ds above a simple Association will avoid the problem.

As a general recommendation I find it rather robust to avoid Dataset for processing data and only use it at the very end to (nicely) present the results. So instead of using ds[ operator1, operator2, ...] I will use ds // Query[ operator1, operator2, ...] which will work on pretty much anything.

Edit: Unfortunately in this case the problem is numerical so that only solutions B and C offer a cure. So the former "Solution A" had to be downgraded to general advice.

Solution B: Examine the error message and work on it

The error message shown is related to Reduce and the message mentioned is Reduce::ratnz. The documentation will tell you that this has to do with inexact numbers. So making MixtureDistribution use exact numbers (Rationalize) will work: 

dsExact = Dataset @ Association[ 
    1 -> Rationalize @ MixtureDistribution[ 
             {.2, .8}, 
             { UniformDistribution[{-.08, 3. 10^6}], GammaDistribution[.5, 88 10^4]}
         ],
    2 -> ExponentialDistribution[1 10^-6]
]

dsExact[ All, With[
       {pdf = PDF[#, x]}, Plot[ pdf, {x, 0, Quantile[#, .95]} ]
   ] &
]

AOK

Solution C: Datasets are very easily 'upset' - so be quiet

ds[ All, Quiet @ With[
       {pdf = PDF[#, x]}, Plot[ pdf, {x, 0, Quantile[#, .95]} ]
    ] &
]

AOK

This will work with the OP's ds without any problem. So Dataset gets upset easily by error measages within.

Edit

Since Edmund has given a reasonable objection to "quieting" everything (one never knows what) I should mention, that we may as well only quiet the specific message that we do not care about, so

Off[ Reduce::ratnz ]

ds[ All, ... ]

will work as well.

Edit2:

Another way to deal with failure messages is to use the option FailureAction. So instead of using Quietwe may have done:

ds[All,
   With[ {pdf = PDF[#, x]},
      Plot[pdf, {x, 0, Quantile[#, .95]}]
   ] &,
   FailureAction -> None 
]
Community
  • 1
gwr
  • 13,452
  • 2
  • 47
  • 78
  • Thanks for the investigations (+1). Solution A is an interesting approach. Solution B does not work in the real case as the parameter values are not so well behaved as in the minimal working example I provided, the actual distributions are a result of FindDistribution. I'm not too keen on solution C as you never know what you are shushing and it can lead to hard to trace errors. I think I might use Solution A in this particular case but I like the syntactical sugar of auto-calling Query with Dataset so I don't see myself dropping that any time soon. – Edmund Mar 20 '17 at 17:38
  • @Edmund I have edited my answer with regard to the very reasonable objection against Solution C. Using Off we very well know what we are "shushing". – gwr Mar 20 '17 at 17:47
  • 1
    A heads up that solution A gives the same failure result as the Dataset Query: Normal@ds // Query[All, With[{pdf = PDF[#, x]}, Plot[pdf, {x, 0, Quantile[#, .95]}]] &] . Currently I'm just mapping and converting like @b.gatessucks comment in the OP. – Edmund Mar 20 '17 at 19:31
  • @Edmund Ok, I must admit, that I had not checked this more thorouhgly - but it should serve as a general remark in this case. So it is just Solution C : Off[ Reduce::ratnz ] which will make this work out. – gwr Mar 20 '17 at 19:46