How can I implement a progress indicator construct which would work on a Manipulate panel?

Question

I usually process data such as follows:

Module[
 {vars},
 ...
 Map[
  Composition[
   h,
   g,
   f]
  ]@lst
 ...
]

For some lists which can take quite a long time to process I want to show a progress indicator.

Dynamic[ProgressIndicator[$progress/$total]]

where

$progress = listitem currently being processed;
$total = length of list

both

$progress, $total 

are global variables.

The issue is about how to implement progress indicators with as minimal change to the current programs as possible.

I was thinking of implementing the following 'standard'function for progress update:

p[]:=$progress++

this function can then be composed as last function of the current function compositions.

In each module initialize

$progress = 1
$total = Length[lst]

Finally, add p[] to the composition.

Rapping up:

Dynamic[ProgressIndicator[$progress/$total]]
Module[
 {$progress=1,$total=Length[lst]},
 ...
 Map[
  Composition[
   p,
   h,
   g,
   f]
  ]@lst
 ...
]

This method works but is not ideal. I haven't succeeded in getting this to work in a Manipulate Panel. I usually select lists and other parameters in a Manipulate program and start a processing module with an action Button, ideally I would like to see the progress in the Manipulate panel.

Questions: how can I implement a progress indicator construct which would work on a Manipulate panel?

Note that both the module and the Manipulate functions are members of a package, i.e.:

processRows[]

shows a Manipulate in the Notebook of the package user with several controls to select data and buttons to start the processing. I don't want to change this. The only additional requirement is the additional display of a progress indicator of some kind.

Answer 1

Although the Manipulate program is still in development, it does contain a working progress indicator. This is the code ( as is, per now ):

guidsBkmkCompEvents[] := Manipulate[
  a = (Select[
       MemberQ[bkmk, #bookmaker] && MemberQ[comp, #compName] &]@
      dsBkmkComp) // no;
  If[Length[a] > 0, $total = Length[a]];

  Column[{
    If[Length[a] > 
      0, (Select[
         MemberQ[bkmk, #bookmaker] && MemberQ[comp, #compName] &]@
        dsBkmkComp)[All, {"url"}]],
    If[Length[a] > 0, a],
    If[Length[a] > 0, Dynamic[ProgressIndicator[$progress/$total]]],
    If[bShow,
     While[$progress < $total, 
      res = dsBCE[a][All, {"id", "bookmaker", "compName", "url"}]]
     ],
    If[bShow, res]
    }],

  {bkmk, ListPicker[#1, 
     DeleteDuplicates[dsBkmkComp[All, "bookmaker"] // no]] &},
  {comp, ListPicker[#1, 
     DeleteDuplicates[dsBkmkComp[All, "compName"] // no]] &},

  Button["Release swarm",
   $progress = 0;
   bShow = True],

  Button["Clear",
   $progress = 0;
   $total = 1;
   bkmk = {};
   comp = {};
   bShow = False],

  Initialization :> {
    bShow = False,
    $progress = 0,
    $total = 1,
    res = {<||> // ta} // ds
    }
  ]

Contrary to what I thought earlier,

Dynamic[ProgressIndicator[$progress/$total]]

does work on a Manipulate panel. Key to getting it grow to 100%, while the program dsBCE is working, is to enclose it in Dynamic and maintaining the

$progress, $total 

variables in the program dsBCE.