Using Apply inside Compile

Question

In this great answer a compiled version of the Nelder-Mead algorithm is presented.

Since it works on arbitrary dimensions (i.e. arbitrary number of arguments), it has to use apply on the objective function. The problem is that Apply is not directly supported inside Compile. To overcome this limitation the following code is used:

(* Produces compiled code for the Nelder-Mead algorithm with the objective function inlined *)
ClearAll[apply];
SetAttributes[apply, HoldAll];
apply[f : (_Function | _CompiledFunction), vars : {__Symbol}] :=
    With[{applied := f @@ vars},
      Function[arglist, Block[vars, vars = arglist; applied]]
     ];

This seems to work (inside the package), but I have no idea, how it works.

Could somebody explain the techniques behind this code snippet? Specifically:

• What does SetDelayed do inside With?

• Why does vars appear twice in the arguments to Block?

• Why doesn't this minimal example work? (I believe it mimics what is done in the mentioned answer...)

  Clear[a, b, c, x, y, z, objectiveFunction, cfunc];
  objectiveFunction = Compile[{a, b, c, x, y, z} ,
     (a - x)^2 + 50 (b - y)^2 + (c - z)^2];
  
  cfunc = With[{f = apply[objectiveFunction, {a, b, c, x, y, z}]},
    Compile[ {{pts, _Real, 1}}, f@pts]
    ]
  << CompiledFunctionTools`
  CompilePrint[cfunc]
  

EDIT:

Here is the working snippet. Note that "InlineCompiledFunctions" must be set to True to avoid the call to MainEvaluate.

ClearAll[a, b, c, x, y, z, apply, objectiveFunction, cfunc];
(* Produces inlinable code for use inside Compile (where Apply is not \
supported directly) *)
SetAttributes[apply, HoldRest];
apply[f : (_Function | _CompiledFunction), vars : {__Symbol}] :=
       Function[arglist, Block[vars, vars = arglist; f @@ vars]];

objectiveFunction = Compile[{a, b, c, x, y, z} ,
   (a - x)^2 + 50 (b - y)^2 + (c - z)^2];
apply[objectiveFunction, {a, b, c, x, y, z}]
cfunc = With[{f = apply[objectiveFunction, {a, b, c, x, y, z}]},
  Compile[ {{pts, _Real, 1}}, f@pts,
    CompilationOptions -> {"InlineCompiledFunctions" -> True}]
  ]
<< CompiledFunctionTools`
CompilePrint[cfunc]

Answer 1

First question : With accepts a syntax like

  With[{var:=value}, expression]

in which case, value is injected into expression unevaluated. As far as I know, this syntax is not documented. You can achieve a similar effect with the replacement rules, by using

Unevaluated[expression]/.HoldPattern[var]:>value

There are some subtle differences between the semantics of With and repalcement rules though, mostly related to the treatment of nested scoping constructs and variable name conflicts in them.

Second question: vars appear twice because they must first be Block-ed, and then there is a massive assignment to them performed in the body of the Block. This is probably the most economical way of blocking a number of variables and assigning values to them simultaneously - otherwise a more complex code-generation will be needed. You can see another example of that in this answer (and if you look at the revision history for that answer, you can find an alternative, harder way to do this, in one of the previous revisions).

Third question: this does not work because the apply function was made HoldAll (which isn't quite necessary), and the pattern-matching does not work. There were some past discussions on this topic on SO, but can't find them right now. But I discussed this topic at length also in my book. The idea is that at the pattern-matching time, all seen by apply is a variable objectiveFunction, and because it does not evaluate it, the pattern _CompiledFunction is not matched. The solution is to make apply HoldRest, and then it works:

ClearAll[apply];
SetAttributes[apply, HoldRest];
apply[...]:=...