1

This piggy-backs on my previous question: Parameter space search routine is too fast?

I am looking for a fast way to evaluate a symbolic list over many points. So lets say I have a list of symbolic expressions like

ListA={a*b*c>0, a*b*(c+1)>0, a*b*(c-1)>0, etc.}

and a list of tuples of the form

ListB={{1,1,1}, {1,1,2}, {1,2,1}, {1,2,2}< etc.}

and I want to evaluate ListA over each tuple of ListB like

ListA/.Thread[{a,b,c} -> ListB[[1]]]
ListA /.Thread[{a,b,c} -> ListB[[2]]]

Now, my listA can have upwards of tens of thousands points and each expression can be over a hundred lines. My ListB can also be ginormous, like upwards of tens of millions of points, but each tuple only has ~5 elements and I've chunked it into sizes of roughly 100-1000 tuples. My question is then what would be the best way to quickly perform this type of replacements/association?

My first attempt used ParallelMap but this took ages still. Then I looked into Associations and this cut the time down but each replacement of an element of ListB still takes like 1.5 - 2 seconds, which I need to cut down considerably. Here is a MWE for reference:

func = (-2^(1 - px) (-1 + px) px Coth[
       rx sx]^2 (-2 sx y Sech[sx (-rx + x^2 + y^2)]^2 + 
        2 sx y Sech[sx (rx + x^2 + y^2)]^2)^2 (Coth[
         rx sx] (-Tanh[sx (-rx + x^2 + y^2)] + 
          Tanh[sx (rx + x^2 + y^2)]))^(-2 + px) - 
    2^(1 - px) px Coth[
      rx sx] (Coth[
         rx sx] (-Tanh[sx (-rx + x^2 + y^2)] + 
          Tanh[sx (rx + x^2 + y^2)]))^(-1 + 
        px) (-2 sx Sech[sx (-rx + x^2 + y^2)]^2 + 
       2 sx Sech[sx (rx + x^2 + y^2)]^2 + 
       8 sx^2 y^2 Sech[sx (-rx + x^2 + y^2)]^2 Tanh[
         sx (-rx + x^2 + y^2)] - 
       8 sx^2 y^2 Sech[sx (rx + x^2 + y^2)]^2 Tanh[
         sx (rx + x^2 + y^2)]) + 
    2^-px (-1 + px) px Coth[
       rx sx]^2 (-2 sx y Sech[sx (-R - rx + x^2 + y^2)]^2 + 
        2 sx y Sech[sx (-R + rx + x^2 + y^2)]^2)^2 (Coth[
         rx sx] (-Tanh[sx (-R - rx + x^2 + y^2)] + 
          Tanh[sx (-R + rx + x^2 + y^2)]))^(-2 + px) + 
    2^-px px Coth[
      rx sx] (Coth[
         rx sx] (-Tanh[sx (-R - rx + x^2 + y^2)] + 
          Tanh[sx (-R + rx + x^2 + y^2)]))^(-1 + 
        px) (-2 sx Sech[sx (-R - rx + x^2 + y^2)]^2 + 
       2 sx Sech[sx (-R + rx + x^2 + y^2)]^2 + 
       8 sx^2 y^2 Sech[sx (-R - rx + x^2 + y^2)]^2 Tanh[
         sx (-R - rx + x^2 + y^2)] - 
       8 sx^2 y^2 Sech[sx (-R + rx + x^2 + y^2)]^2 Tanh[
         sx (-R + rx + x^2 + y^2)]) + 
    2^-px (-1 + px) px Coth[
       rx sx]^2 (-2 sx y Sech[sx (R - rx + x^2 + y^2)]^2 + 
        2 sx y Sech[sx (R + rx + x^2 + y^2)]^2)^2 (Coth[
         rx sx] (-Tanh[sx (R - rx + x^2 + y^2)] + 
          Tanh[sx (R + rx + x^2 + y^2)]))^(-2 + px) + 
    2^-px px Coth[
      rx sx] (Coth[
         rx sx] (-Tanh[sx (R - rx + x^2 + y^2)] + 
          Tanh[sx (R + rx + x^2 + y^2)]))^(-1 + 
        px) (-2 sx Sech[sx (R - rx + x^2 + y^2)]^2 + 
       2 sx Sech[sx (R + rx + x^2 + y^2)]^2 + 
       8 sx^2 y^2 Sech[sx (R - rx + x^2 + y^2)]^2 Tanh[
         sx (R - rx + x^2 + y^2)] - 
       8 sx^2 y^2 Sech[sx (R + rx + x^2 + y^2)]^2 Tanh[
         sx (R + rx + x^2 + y^2)]));

parameters = {px, pz, R, rx, rz, sx, sz}
variables = {x, y, z}


Quantifier[coords_, params_] := 
 Function[Evaluate@Join[variables, parameters], Evaluate@(func > 0)][
  Sequence @@ Join[coords, params]]


SpaceA = Tuples[Range[-2, 2, 0.2], 3];


ListA = Quantifier[#1, parameters] & /@ SpaceA;
ListB = Tuples[Range[1, 4, 0.4], 7];
(ListB contains~2 million elements)

Now, evaluating ListA over ListB would proceed like

(AllTrue[ListA /. Thread[parameters -> #], TrueQ]) & /@ ListB
(*Careful running this, it will probably take a few months :( *)

My problem is that even a single association like

ListA/.Thread[parameters->{1,1,1,1,1,1,1}]

takes about 2 seconds. So repeating this over a list of ~2 million points would take a century.

Would a compiled function be useful? I dont have much experience using the compile functionality so I'm wondering if it would be advantageous to explore that. I appreciate any insight!

Update

Thanks to @flinty suggestion, using With seems to speed up the assignment considerably. Heres a short timing experiment:

Here, QuantifieroverSpace corresponds to ListA in the MWE above.

ClearAll[\[Epsilon], px, pz, R, rx, rz, sx, sz]
ByteCount[QuantifieroverSpace]

With[{[Epsilon] = 2, px = 1, pz = 5, R = 1, rx = 2, rz = 2, sx = 2, 
   sz = 2},
  Evaluate@AllTrue[QuantifieroverSpace, TrueQ]] // Timing


AllTrue[QuantifieroverSpace /. 
   Thread[{[Epsilon], px, pz, R, rx, rz, sx, sz} -> {2, 1, 5, 1, 2, 
      2, 2, 2}], TrueQ] // Timing


(126992696)
({0.000026, False})


({2.08846, False})

So using With instead of ReplaceAll is many orders of magnitude faster, which is interesting. I will implement this in my search routine and see how much it improves it.

Update 2

So my next issue is that I need the first argument of With to be modular to the number of arguments, i.e. it needs to be able to take in either a 3 variable set like {a=1, b=1,c=1} or a different number like {a=1}. My first though would be to do something like

With[
     {Thread[SymbolList = ArrayofValues]}, 
     ...
     ]

but mathematica is assigning the values in ArrayofValues to the symbols in SymbolList so that the variable a for example has value 1. I then tried

init = MapThread[HoldForm[#1=#2]&, {SymbolList, ArrayofValues}];
With[
     Evaluate@ReleaseHold[init],
     ...
     ]

but this does the same thing, assigning the values to the symbols. Interestingly, mathematica still executes the with expression using the values in the first argument, but its assigning the value to the symbol still, which would slow down the execution of my search routine if I wanted to undo the assignment. I need to somehow halt the Seting assignment but still maintain the form a=1 in a way dynamic to the number of variables.

Update 3

Well upon further inspection, I figured out why With appears to be so much faster. Its because its not actually subsituting the values of the first argument into the expression. For example,

a = {l, s};
With[{l = 3, s = 12},
  Print[Evaluate[a]]
  ];

({l,s})

So I guess I'm back at square one trying to find a faster way to assign values to parameters inside a big symbolic array.

shanedrum
  • 577
  • 2
  • 8
  • 1
    Have you tried With[{a = 3, b = 4, c = 1}, Evaluate[thelist]] ? I'm not sure it will be any faster though. – flinty Dec 11 '20 at 22:11
  • Wow, actually for my inital test, With seems to be considerably faster. I didnt expect that. I'll update my post with a short test. Thanks for the insight! – shanedrum Dec 11 '20 at 22:35
  • What would be a good way to thread the parameter values in the first argument of With? I tried With[Evaluate@Thread[{a,b,c}={1,2,3}], ...] but that returns With::lvws: Variable 2 in local variable specification {2,1,1,2,2,2,2,2} requires a value.. The variables dont have any assignment before evaluating the With. I need to basically map the With over the ListB – shanedrum Dec 11 '20 at 23:07
  • I've never used it this way but it's possible apparently: https://mathematica.stackexchange.com/a/204117/72682 . The variable spec in the with needs to be held. – flinty Dec 11 '20 at 23:27
  • Thanks for the reference. I took a look and im still having issues with it. I need to be able to basically input a varying number of parameters since my program is supposed to be modular to the number of parameters, which is why I'm trying to use Thread so that I could do something like With[ Thread[parameters = Listb[[i]]], ...]. I tried using the OwnValue approach, but that fails with the same above error message. The issue is Set is assigning the values of ListB[[i]] to the symbol used in the parameter. I need to be able to Hold the set somehow – shanedrum Dec 12 '20 at 01:35
  • The usual approach to evaluating inequalities is to use UnitStep[a - b], which codes a>=b/a<b as 1/0, and to evaluate over packed arrays. A similar question and some answers related to this approach: https://mathematica.stackexchange.com/a/29351 https://mathematica.stackexchange.com/a/201794 https://mathematica.stackexchange.com/a/3500 – Michael E2 Dec 12 '20 at 16:55
  • Have you tried using an actual function? Define ListA[a_, b_, c_] = {a*b*c > 0, a*b*(c + 1) > 0, a*b*(c - 1) > 0} and ListB = {{1, 1, 1}, {1, 1, 2}, {1, 2, 1}, {1, 2, 2}}, and then compute ListA @@@ ListB. – Roman Dec 12 '20 at 18:38

1 Answers1

1

This is only a partial answer, but...

Your function is pretty complicated, and speeding that up is probably more important than how you feed values to it. Compile is your friend here.

cfunc = Compile @@ {Join[variables, parameters], func, 
   CompilationTarget -> "C", "RuntimeOptions" -> "Speed", 
   RuntimeAttributes -> {Listable}}

RepeatedTiming[AllTrue[cfunc @@ Join[SpaceA // Transpose, ListB[[1]]], Positive]]
{0.0051, False}

At 5ms for a single row of ListB, it's still going to take a long time, though it's getting more realistic.

Simon Woods
  • 84,945
  • 8
  • 175
  • 324
  • Thanks for the suggestion! I've tried to implement a compiled function with varying success, but I'll try your code and see how it goes – shanedrum Dec 12 '20 at 19:53
  • Wow, that works beautifully. With a diffeerent function (only an analytic function with no hyperbolic trigs), I get roughly 0.0008 seconds for execution. So Listable allows the function to take in a list of parameters, and then Transpose changes the list of 3-tuples to a 3 element nested list of the ranges of each element of the 3-tuples? That makes sense with using the listable attribute. Thank for the great suggestion! That will speed up my search routine considerably – shanedrum Dec 12 '20 at 20:04
  • This actually solves my specific problem, so i'll mark it as such. Perhaps theres still a better way to solve the problem of my original post, but i'm satisfied! Thanks again. – shanedrum Dec 13 '20 at 01:14