13

Take the following example:

big = RandomReal[1, 10000000];
Head[big >= 0.66] // AbsoluteTiming

(* {0.53489, GreaterEqual} *)

Why does this take a long time to evaluate, given that the expression doesn't actually change?

Update: Simon Woods points out that the slowdown is due to unpacking. There is also some strange interaction with ReplaceAll that I do not understand. Both are demonstrated by the following:

r=RandomReal[1,10000000];

On["Packing"]

Hold[Greater[r,0.5]]/.{x_?ArrayQ:>x}//AbsoluteTiming
Developer`FromPackedArray::unpack: Unpacking array in call to Greater.
{0.497073,Hold[r>0.5]}

Hold[greater[r,0.5]]/.{x_?ArrayQ:>x}//AbsoluteTiming
{0.000017,Hold[greater[r,0.5]]}

ArrayQ[r]//AbsoluteTiming
{3.*10^-6,True}

Note:

  • big is a large packed array
  • big >= 0.66 is the same as GreaterEqual[big, 0.66]
  • the result is identical to GreaterEqual[big, 0.66]

This is a problem for my BoolEval package, which has a BoolEval function that transforms the expression big >= x to UnitStep[big - x] to speed up element-wise comparisons. big >= x takes longer to evaluate than UnitStep[big - x].

Here's a much simplified version of the function:

greatereq[a_, b_] := UnitStep[a-b]
BoolEval[expr_] := expr /. GreaterEqual -> greatereq

A potential solution is 

SetAttributes[BoolEval, HoldAll]
BoolEval[expr_] := Unevaluated[expr] /. GreaterEqual -> greatereq

The problem with this is that it will not work with

f[] := big >= 66
BoolEval[f[]]

Such a function would simply be too confusing to use.

Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263
  • @Kuba The actual function can handle arbitrary Boolean expressions such as 0.1 < arr < 0.5 && arr != 0.4. This solution would hide the performance problem for a few common cases, but otherwise it will still be slow. So I'm not fully happy with it. – Szabolcs Jul 12 '16 at 13:30
  • Yep, seems like a dead end for a general function. – Kuba Jul 12 '16 at 13:45
  • 2
    On["Packing"] reveals that the array is unpacked in the call to GreaterEqual – Simon Woods Jul 12 '16 at 20:21
  • 1
    How about the HoldAll version but instead of Unevaluated use Block[{GreaterEqual}, ...] – Simon Woods Jul 12 '16 at 20:25
  • 2
    @SimonWoods I thought of it but the difficulty with that is the following: What if someone does BoolEval[myFun[] > 1] and myFun[] uses Greater? I am probably going to do HoldAll and require the boolean expressions and comparisons to appears explicitly in the argument. I.e. BoolEval[x > 1] is okay, but fun[]:=x>1; BoolEval[fun[]] is not. However, I am not willing to break BoolEval[myFun[] > 1] as well. – Szabolcs Jul 13 '16 at 08:29
  • What "strange interaction with ReplaceAll" are you observing? The unpacking? That comes from ArrayQ[Greater[r, 0.5]]. Can we directly overload all boolean operators and apply those within your call, Block[{Greater = myGreater, Less = myLess}, body]? Make them general and myFun[] > 1 should still work, right? – Mr.Wizard Feb 26 '17 at 17:37
  • 1
    @Mr.Wizard I more or less gave up on this question, partly because WRI explained why there's unpacking (and it's unlikely to go away), and partly because I don't think I was taking the correct approach anyway. You are right about the ReplaceAll—embarrassing oversight. I don't think Block will be a good approach, but too little space to explain here ... I wanted to entirely rewrite BoolEval and blog about it. We can discuss next week in chat if you're interested, otherwise I'll ping you when the blog is out. – Szabolcs Feb 26 '17 at 19:38
  • I look forward to either eventuality. :-) – Mr.Wizard Feb 26 '17 at 19:39
  • @Szabolcs the classic definition pattern I've seen is to use HoldAll, a basic definition respecting this HoldAll, and one that evaluates to check anything not matching this form: i.e. define BE[ a_ > b_ ] := greater[a, b] and then BE[ e_ ] /; !TrueQ[$inBE] := Block[{$inBE=True, res}, res = BE[Evaluate@e] /; Head[res] =!= BE ]. Unfortunately this would probably confuse users who notice performance differences between BE[ a > b] and BE[ f[] ] for f[] := a > b – b3m2a1 Feb 01 '18 at 19:15

0 Answers0