Valid predicates for Probability[]?

Question

I don't understand what counts as a valid predicate to Probability[].

This does not work:

Probability[EvenQ@x, x \[Distributed] Range@10]

(*  0  *)

According to the docs, this should be equivalent to

Sum[Boole@EvenQ@x, {x, Range@6}]/Length@Range@6

(*  1/2  *)

which does give the expected answer. This also works,

Probability[Mod[x, 2] == 0, x \[Distributed] Range@6]

(*  1/2  *)

As does this, when the output is subsequently evaluated:

Probability[Defer@EvenQ@x, x \[Distributed] Range@6]

(*  1/6 (Boole[EvenQ[1]] + Boole[EvenQ[2]] + Boole[EvenQ[3]] + 
    Boole[EvenQ[4]] + Boole[EvenQ[5]] + Boole[EvenQ[6]])  *)

I hesitate to blame the tool, but when the explicitly stated equivalent formulation gives a different answer, it sure looks like it's not confusion on my part. Is it me? What am I missing?

Thanks.

My guess is that it is the way EvenQ works. From the documentation on EvenQ: EvenQ[expr] returns False unless expr is manifestly an even integer (i.e. has head Integer, and is even). Without the Defer it immediately returns False inside the Probability function. — JimB, Mar 06 '17 at 19:41
The use of the term "manifestly" in the documentation of a function (EvenQ, IntegerQ, etc.) is what you need to watch out for. In those cases you'd need to use Defer. — JimB, Mar 06 '17 at 20:01
Something like that was the only thing I could think of, but it was puzzling why EvenQ and Equal (in Mod[x,2]==0) would work differently. But that doesn't explain what the documentation for Probability says: "For a dataset data, the probability of pred is given by Sum[Boole[pred],{x,data}]/Length[data]"; that code gives the right answer so it must only be 'notionally given by' it. — Scott Crittenden, Mar 06 '17 at 20:10
Trace is enlightening. For the Probability version, the first thing that happens is that EvenQ[x] is evaluated and, per Jim's observation gives False, whereas, for the Sum version, EvenQ is only evaluated after x is replaced with an integer. — Scott Crittenden, Mar 06 '17 at 20:58
@JimBaldwin Defer is meant only for formatting. It is absolutely not meant for evaluation control. Never try to use it for programming, and controlling program flow. It does not even do anything useful without a notebook interface. — Szabolcs, Mar 07 '17 at 13:02

score 4 · Answer 1 · edited Apr 13 '17 at 12:55

The reasons the supposedly false behavior of EvenQ have been explained nicely in (139281). Basically there are three solutions:

1) Avoid predicates which will work on symbolic inputs

Next to using Mod[ x, 2] == 0 there is a more compact form available:

Probability[ x ~ Divisible ~ 2, x \[Distributed] Range @ 10 ]

1/2

2) Modify the predicates so they will not work on symbolic inputs

This is the solution given in the thread that I have linked. For completeness:

nEvenQ[ x_?NumericQ ] := EvenQ @ x

Probability[ nEvenQ @ x, x \[Distributed] Range @ 10 ]

1/2

3) Prevent the predicates from evaluating prematurely

While in the OP Defer has been used already, a nicer solution (imo) exists using Inactivate and Activate:

Probability[ Inactivate @ EvenQ[x], x \[Distributed] Range @ 10 ]

(* 1/10 (Boole[Inactive[EvenQ][1]] + Boole[Inactive[EvenQ][2]] + 
Boole[Inactive[EvenQ][3]] + Boole[Inactive[EvenQ][4]] + 
Boole[Inactive[EvenQ][5]] + Boole[Inactive[EvenQ][6]] + 
Boole[Inactive[EvenQ][7]] + Boole[Inactive[EvenQ][8]] + 
Boole[Inactive[EvenQ][9]] + Boole[Inactive[EvenQ][10]]) *)

Activate @ %

1/2

Szabolcs · Accepted Answer · 2017-03-07T13:06:15.233

Here's a different perspective to complement gwr's answer:

Mathematica has functions for traditional programming, and functions which represent Mathematical concepts and are suitable for symbolic computation. The line between the two is somewhat blurred. This is good for keeping things simple, but it can also be confusing. A stricter CAS would make this distinction explicit at the cost of some convenience, and would provide separate tools for these two types of uses.

"Programming functions" will evaluate immediately, just as they do in other languages. EvenQ is such a function. Generally, functions ending in Q will evaluate immediately to either True or False. Example:

EvenQ /@ {0, 1, x}
(* {True, False, False} *)

Thus it is better to think of EvenQ[x] as "x is of Integer datatype and is even". It will return False for anything that is not of the Integer datatype, which is not the same thing as an integer in the mathematical sense. E.g., EvenQ[2.0] is False because 2.0 is Real. $k\in\mathbb{Z}$ is a mathematical concept, datatypes are a programming concept.

"Mathematical functions" do not evaluate with symbolic values and can be used to represent mathematical statements. Positive is such a function. Note that even though it returns True or False, it does not have Q in the name. Example:

Positive /@ {0, 1, x}
(* {False, True, Positive[x]} *)

It can be used to represent a Mathematica statement, on which we can perform operations:

FullSimplify[Positive[x^2], Element[x, Reals] && x != 0]
(* True *)

An example of a function that blurs the line between math and programming is If. If its first argument is neither True nor False, then it does not evaluate, and can be used as a sort of substitute for Piecewise. Note though that it has a fourth argument: when specified, If will evaluate immediately.

In general, you can think of programming function as "code" that is meant to run. You can think of mathematical functions as data that can be operated on. Probability is meant to be used with mathematical functions—anything else will evaluate before Probability even has a chance to see it. That excludes anything that has a Q in the name.

A note about Defer: this is a function that is meant purely for displaying/formatting things in notebooks. It is not meant for evaluation control.

Valid predicates for Probability[]?

2 Answers2