Replace nested external definitions in Compile

Question

f = #^2 &;
Compile[{{x, _Real, 1}}, f /@ x, 
  CompilationOptions -> {"InlineExternalDefinitions" -> 
     True}] // CompilePrint

works fine.

f = g[#]^2 &;
g = # - 1 &;
Compile[{{x, _Real, 1}}, f /@ x, 
  CompilationOptions -> {"InlineExternalDefinitions" -> 
     True}] // CompilePrint

will have MainEvaluate[ Hold[g][ R1]]. I guess the automatic replacement will only do such replacement once.

I have found one workaround so far, but it requires listing all the related functions (not necessary in the correct replacement order).

ReleaseHold[
  Hold[Compile[{{x, _Real, 1}}, f /@ x]] //. 
   Flatten[OwnValues /@ Unevaluated@{f, g}]] // CompilePrint

Is there a better one without listing {f, g}?

The mechanism used by "InlineExternalDefinitions" -> True seems opaque, that is I cannot find any information about how it is implemented using Trace. Would it be useful to you simply to programmatically extract f and g from your Compile expression for use in your own method? — Mr.Wizard, Jul 21 '16 at 10:28
@Mr.Wizard Yeah, if I can programmatically list all the "Global" symbols like f inside Compile and the derived symbols like g, the problem is solved. But I don't know how to do that. — vapor, Jul 21 '16 at 11:35
Would you like to have all non System` Symbols or do you prefer to restrict it to Global` Symbols? — Mr.Wizard, Jul 21 '16 at 11:37
@Mr.Wizard I am not sure how to define that, since all temporary variable is unwanted (including param of Compile and local variable in Module inside compile which has Global context) — vapor, Jul 21 '16 at 11:42
Okay. I'll look at this and try to come up with something useful. — Mr.Wizard, Jul 21 '16 at 11:47

score 7 · Accepted Answer · edited Apr 13 '17 at 12:55

This is a proof of concept of the idea of parsing the Compile expression, extracting Symbols, finding dependent definitions, and inserting them back into Compile.

First a HoldFirst variation of heldCases:

makeHeld[(L_ -> R_) | (L_ :> R_)] := L :> HoldComplete[R];
makeHeld[pat_] := x : pat :> HoldComplete[x];

Attributes[heldCases] = {HoldFirst};

heldCases[expr_, rule_, args___] := 
  Join @@ Cases[Unevaluated @ expr, makeHeld @ rule, args]

Then the main definition:

Attributes[inject] = {HoldFirst};

inject[all : Compile[var_, body_, opts___]] :=
  Complement[
    heldCases[body, s_Symbol /; Context[s] =!= "System`", {-1}, Heads -> True], 
    heldCases[var, s_Symbol | {s_Symbol, __} :> s, {2}]
  ] //
    Apply[FullDefinition] //
    InputForm // ToString // ToHeldExpression //
    Cases[(Set | SetDelayed)[s_Symbol, RHS_] :> (HoldPattern[s] :> RHS)] //
    ( Unevaluated[all] //. # & )

Usage :

Compile[{{x, _Real, 1}}, f /@ x, 
  CompilationOptions -> {"InlineExternalDefinitions" -> True}] // inject

Replacing Unevaluated with Hold in the last line of inject shows the intermediate output:

Hold[Compile[{{x, _Real, 1}}, ((#1 - 1 &)[#1]^2 &) /@ x, 
  CompilationOptions -> {"InlineExternalDefinitions" -> True}]]

Notes

This is rather verbose for what is conceptually a fairly simple operation.
I am not happy with the ToString conversion and back. Possibly I should have extracted Symbols only and used OwnValues as you did, but since I resorted to pulling the definitions with FullDefinition using that information directly seemed a natural approach.
The replacements should really only be done on the body rather than the entire Compile, but the latter was expedient for a proof of concept.

Replace nested external definitions in Compile

1 Answers1

Notes