21

In his last comment on this answer, halirutan shows an example in which the third argument of Compile does not have any effect. The example is as follows.

cfu = Compile[{}, Module[{y = {}}, AppendTo[y, 1]], {{y, _Integer, 1}}];
cfu[]

Outputs

{1.}

rather than {1}

Does anybody know an example where using the third optional argument of Compile does have an effect?

Community
  • 1
Jacob Akkerboom
  • 12,215
  • 45
  • 79
  • 6
    Sure, I've used it a number of times, although right now good examples don't come to mind. In your case, there is no effect because y goes out of scope before Compile can use that argument. The way to solve this problem for local variables is to initialize them inside Module declaration to correct types, and then possibly redefine in the body of Module to the values you really need, like e.g. cfu = Compile[{}, Module[{y = Rest[{0}]}, AppendTo[y, 1]]]. – Leonid Shifrin Feb 28 '14 at 13:32
  • 2
    Good question! I consider myself pretty good at using Compile but find the third argument to be a by mysterious. – Mark McClure Feb 28 '14 at 13:54
  • @MarkMcClure same here, I actually forgot the third argument existed altogether and I was embarrassed for a moment :) – Jacob Akkerboom Feb 28 '14 at 13:59
  • 1
    @LeonidShifrin Compile doesn't use ordinary scoping rules (for example, Module, Block, With are all treated exactly the same), so AIUI your explanation isn't correct. I think the real reason is that the third argument only applies to values that are obtained via calls out of the VM--it is a type hint, rather than a declaration. – Oleksandr R. Mar 01 '14 at 16:02
  • 1
    @OleksandrR. I've used the third argument many times, to give hints to Compile where its type inference wasn't enough to guess the right type for an intermediate expression, and those calls were not out of the VM (IIRC, I may be wrong). I just didn't have the time to dig out examples. Re: scoping - I am aware of Compile not using the standard scoping rules, and you may be right, although I seem to remember that there were cases which actually led me to this conclusion. If / when I am able to dig out some examples, I will get back. – Leonid Shifrin Mar 01 '14 at 18:23
  • 2
    @LeonidShifrin I'd certainly like to see those examples. So far I've never encountered any situation where the third argument does anything useful (or anything at all, really) except for external calls. My experience is that if you can get a code to compile completely (perhaps using hacks like with Rest/Most), normally all the types are correctly inferred. If not, and where there is no suitable hack, I've usually found the the situation to be hopeless, sometimes even resulting in a kernel crash where types do not match what was inferred. – Oleksandr R. Mar 01 '14 at 18:57

2 Answers2

14

To my knowledge, one of the situations where the third argument of Compile is required is when your compiled code needs to make a MainEvaluate call. Often, the situation is the other way around: you have code and you try to compile it down completely, so that no MainEvaluate call is necessary. Honestly, most of the questions about Compile here deal to some extend with this issue.

What is, when it is out of question that some part of your code has to be run by the kernel and cannot be compiled under no circumstances? Then the task is to build this into your compiled code, so that it works and the compiler can determine the type of your external call. Let's assume a simple function which collects some prime numbers:

primes[n_] := Last[Last[Reap[Table[If[PrimeQ[i], Sow[i]], {i, n}]]]]

and now you want to have a compiled function which multiplies such a list with an integer giving you an integer list back. First, you can play around with this snipped

Compile[{{n, _Integer}}, Module[{a = 2},
  a*Last[Last[Reap[Table[If[PrimeQ[i], Sow[i]], {i, n}]]]]
  ]
 ]

but this is not fun and Reap and Sow are bad for compiling anyway. The solution is the third argument to Compile:

fc = Compile[{{n, _Integer}}, Module[{a = 2},
   a*primes[n]
  ], {{primes[_], _Integer, 1}}
]

because you can get a working and readable version of your function. Now assume that you have a compiled function where you need the result of NMinimize or some similar (uncompilable) function. This is your use-case.

As for your initial problem with y and AppendTo: The type of local variables can be coerced in other ways. For instance, if you want to say that y should be an integer list, then initialize it with Most[{1}] which gives you an empty list, but the compiler saw that you want an integer list :-)

cfu = Compile[{}, Module[{y = Most[{1}]}, AppendTo[y, 1]]];
cfu[]
(* {1} *)
halirutan
  • 112,764
  • 7
  • 263
  • 474
  • Ah yes, I see, very nice! I suppose in a simple case we could still provide primes[n] as an argument to the compiled function. But if a lot of calls are made to primes, especially if you don't know how many, it seems this technique can become quite necessary. I was aware of the technique with Most, I much prefer that one, over the one my answer :). – Jacob Akkerboom Feb 28 '14 at 16:27
  • 1
    @JacobAkkerboom Yes, for this simple case you could just provide prime[n], but assume that your compiled function really does something and calculated the parameters (here n) for your external calls. Then this is not possible. The type inference for local variables is one of the black wholes in the documentation and I personally, find it rather obscure that there is no better way to annotate the types. – halirutan Feb 28 '14 at 16:44
  • 1
    As I understand it, the scenario you describe isn't just one of the situations where the third argument of Compile is required but actually the only case where it has any effect at all. That it is impossible to declare the types of local variables with this method is what motivated me to start using these stupid tricks with Most/Rest. I hope the situation will be improved in the future. – Oleksandr R. Mar 01 '14 at 15:36
  • @OleksandrR. It is mostly since the beginning of M.SE that I am very carefully with expressions like the only solution, the only use-case, etc.. It happened too often that I'm surprised by all the ideas here, but I believe too that it is one of the main scenarios. – halirutan Mar 01 '14 at 20:57
4

Leonids comment encouraged me to experiment with unscoped variables. I then the following difference, which is not particularly useful.

cfu2 = Compile[
   {},
   Module[
    {x = y}
    ,
    AppendTo[x, 2]
    ],
   {{y, _Integer, 1}}
   ];

cfu3 = Compile[
   {},
   Module[
    {x = y}
    ,
    AppendTo[x, 2]
    ]
   ];

The functions give different results

y = {}; cfu2[]
y = {}; cfu3[]

{2} 
{2.}

I think the trick mentioned by Leonid in the comments should normally be faster, as it avoids MainEvaluate. We should probably not use this to "declare" variables in Compile. The use case described by halirutan is much more useful.

Jacob Akkerboom
  • 12,215
  • 45
  • 79