Manipulating parameters in a complex algorithm

Question

I have a complex algorithm that depends on:

A wavelet basis
A number of sampling points out of a distribution
A thresholding value

I want to be able to play with different scenarios using Manipulate, but I am hitting a wall trying to ensure that when one updates the threshold, it does not resample the data. I have noticed that Manipulate is kind of smart enough to pick up dependencies and recalculate only what's needed, but still puzzles me quite a bit.

So, I have prepared a simple example that illustrates my ignorance...

The following code generates only one sample, but it triggers the call to take an snapshot of the sample every time one changes the threshold (th in code) although the threshold is not involved in the computation of data.

Module[{dist, dataAll},
 dist = MixtureDistribution[{8, 2},
   {NormalDistribution[0.5, 0.05], NormalDistribution[0.7, 0.01]}];
 dataAll = (Print["New data"]; RandomVariate[dist, 2000]);
 Manipulate[
  Module[{data},
   data = (Print["New sample"]; Take[dataAll, nn]);
   ColumnForm[{
    {{Min[dataAll], Max[dataAll]}, {Min[data], Max[data]}} // 
     MatrixForm,
    th}]
    ]
   ],
  {wave, {HaarWavelet[], DaubechiesWavelet[2], DaubechiesWavelet[3], 
    DaubechiesWavelet[4], SymletWavelet[4]}},
  {nn, {80, 100, 200, 300, 500, 800, 1300, 2000}},
  {th, 0., 1., 0.05}
  ]
 ]

Changing the wave variable does not trigger any calculation; apparently because it is not being used at all.

If I change the expression inside Manipulate to something without th, then it does not print the message, i.e. it does not trigger... but this is not useful.

My algorithm is more complex than above. I need guidance on how to setup a Manipulate where certain steps of the process depend on certain manipulate parameters, but I don't want the whole expression (whole algorithm) to be calculated from top to bottom. I started trying also Manipulate inside Manipulate, but it gets very ugly.

So, in abstract, if one has an algorithm like

step 1
step 2, depends on param A
step 3
step 4, depends on param B
step 5
step 6, depends on param C
step 7 -> final expression/plot/etc

where one wants to manipulate params A, B and C. How does one replay the algorithm from the steps that are actually required? For example, if one changes B then only steps from 4 to 7 (final expression) need to be calculated. The issue being that those steps setup a number of auxiliary variables, etc, so splitting that into functions will get messy fairly quickly, I believe. And still I am unsure that doing that will solve the problem. I'd prefer an approach whereby those things are packed in modules inside modules, but would that help?

As a concrete question: how would you change the code above so the resulting expression ColumnForm[...] still refers to th, but the data is not recalculated when changing th?

Any pointers would be appreciated.

@belisarius, wow! it seems to work on the above toy example; I would give it a go with my complex algorithm (and update question) if it doesn't help. Tnx!! — carlosayam, Sep 24 '14 at 02:40
If you look at this link it summarizes all the different cases for Manipulate. For you example, I would use the second argument of dynamics, it is the most flexible way to handle very complicated events and interactions since it is based on then event/callback way of looking at things. — Nasser, Sep 24 '14 at 03:14
@Nasser, so, should I use the option to create the control manually {u, f} where the f function that creates the UI element has that callback in the generated Dynamic? — carlosayam, Sep 24 '14 at 04:11
To use second argument of dynamics, yes, you create each control using direct Dynamic syntax. For example, for the slider, you use Manipulator etc.. Then add the specific logic right there in the second argument as pure function. This way, the second argument acts like the callback. Much easier to manage. here is an example http://mathematica.stackexchange.com/questions/42796/how-can-i-selectively-trigger-computations-in-manipulate/42807#42807 I do all my demos using this method now. I find this method easier to work with. — Nasser, Sep 24 '14 at 04:38
@Nasser, I just did that and works. I will publish my snipped here for others to see and fix. — carlosayam, Sep 24 '14 at 12:54
@Nasser, based on your comments I updated the question with my approach - would be happy to hear if a more idiomatic Manipulate is possible... — carlosayam, Sep 25 '14 at 02:12
it looks good. Tried it, was little slow when N was 500 and higher. May be this is algorithm tune-up. But if it works for you and no more issues, then you can answer your own question with your answer above. Unless you have any specific questions. — Nasser, Sep 25 '14 at 03:01
@Nasser, now I am curious; I didn't provide CalcG1, what did you do? :) — carlosayam, Sep 25 '14 at 13:44
I just copied your code and it ran as is. Here is screen shot. maybe this function was not used when I run it. do not know. — Nasser, Sep 25 '14 at 13:58
@Nasser I see :) there is only one function in your plot: the PDF of the known distribution, not the estimate. I wish there were a more idiomatic Manipulate solution to this; other than building the interface by hand... But it works, Tnx for your guidance by the way !! — carlosayam, Sep 25 '14 at 14:34

score 1 · Answer 1 · answered Sep 25 '14 at 13:49

Based on comments, I came to the following approach:

DynamicModule[
 {nn, tn, dist, dataAll, data, dx, func, wave, f},
 ColumnForm[{
   Button["New data", 
    dataAll =  RandomVariate[dist, 2000]; 
    data = Take[dataAll, nn];
    func = CalcG1[wave, data, 0, Ceiling[Log2[nn] - Log2[Log2[nn]]]];
    f = func[tn];
    ],
   PopupMenu[Dynamic[wave,
     (wave = #;
       func = 
        CalcG1[wave, data, 0, Ceiling[Log2[nn] - Log2[Log2[nn]]]];
       f = func[tn];
       ) &
     ], {HaarWavelet[],DaubechiesWavelet[3],SymletWavelet[1],SymletWavelet[3]}
   ],
   PopupMenu[Dynamic[nn,
     (nn = #;
       data = Take[dataAll, nn];
       func = 
        CalcG1[wave, data, 0, Ceiling[Log2[nn] - Log2[Log2[nn]]]];
       f = func[tn]
       ) &
     ], {100, 200, 300, 500, 800}],
   Slider[Dynamic[tn,
     (tn = #;
       f = func[tn]) &
     ], {0., 1.}],
   Dynamic[
    dataAll; (* ensures dataAll is a tracked symbol *)
    Plot[{PDF[dist, x], f[x]}, {x, -1, 2},
     PlotRange -> {{0, 1}, {-0.1, 1.5 PDF[dist, 0.5]}},
     PlotLabel -> Sum[(f[x] - PDF[dist, x])^2 dx, {x, 0., 1., dx}],
     ImageSize -> 500,
     Epilog -> {Black, 
       Opacity[0.4], (Line[{{#, -0.1}, {#, 0.}}] & /@ data)}
     ], SynchronousUpdating -> False
    ]
   }],
 Initialization :> (
   nn = 100;
   tn = 0.;
   dist =  MixtureDistribution[{8, 2},
     {NormalDistribution[0.5, 0.05], NormalDistribution[0.7, 0.01]}];
   dataAll = RandomVariate[dist, 2000];
   data = Take[dataAll, nn];
   dx = 1/nn;
   wave = HaarWavelet[];
   func = CalcG1[wave, data, 0, Ceiling[Log2[nn] - Log2[Log2[nn]]]];
   f = func[tn];
   )
 ]

The code for CalcG1 is based on here, but I can't share it at the moment. This returns a memoized wavelet-like function to which one can apply a threshold, which returns the final function f. As you can see, it is vital -for example- to be able to redo just the last step to see the impact of the threshold on the estimate.

So, the Initialization in the outer DynamicModule has the whole algorithm; then each control just recalculates the variables needed to have the plot.

If anyone can provide an idiomatic solution using Manipulate, I would be delighted to acknowledge it as answer. Here the current interface:

Mathematica graphics

Manipulating parameters in a complex algorithm

1 Answers1