Replacement inside held expression

Question

I wish to make a replacement inside a held expression:

f[x_Real] := x^2;
Hold[{2., 3.}] /. n_Real :> f[n]

The desired output is Hold[{4., 9.}], but I get Hold[{f[2.], f[3.]}] instead. What is the best way to make such a replacement without evaluation of the held expression?

Answer 1

Generally, you want the Trott-Strzebonski in-place evaluation technique:

f[x_Real]:=x^2;
Hold[{Hold[2.],Hold[3.]}]/.n_Real:>With[{eval = f[n]},eval/;True]
(* Hold[{Hold[4.],Hold[9.]}] *)

It will inject the evaluated r.h.s. into an arbitrarily deep location in the held expression, where the expression was found that matched the rule pattern. This is in contrast with Evaluate, which is only effective on the first level inside Hold (won't work in the example above). Note that you may evaluate some things and not evaluate others:

g[x_] := x^3;
Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = f[n]}, g[eval] /; True]
(* Hold[{Hold[g[4.]], Hold[g[9.]]}] *)

The basic idea is to exploit the semantics of rules with local variables shared between the body of With and the condition, but within the context of local rules. The eval variable will be evaluated first (regardless of whether the condition ends up being True - as in this case, or False - thanks to @luyuwuli for pointing out the problem in the original wording for this part) , inside the declaration part of With, while the code inside the Condition , here the body of With (g[eval]), is treated then as normally the r.h.s. of RuleDelayed is. It is important that With is used, since it can inject into unevaluated expressions. Module and Block also have the shared variable semantics, but wouldn't work here: while their declaration part would evaluate, they would not be able to communicate that result to their body that remains unevaluated (more precisely, only the part of the body that is inside Condition will remain unevaluated - see below). The body of With above was not evaluated either, however With injects the evaluated part ( eval here) into it - this is why the g function above remained unevaluated when the rule applied. This can be further illustrated by the following:

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval=f[n]},
   With[{eval = eval},g[eval]/;True]]
(* Hold[{Hold[g[4.]],Hold[g[9.]]}] *)

Note b.t.w. that only the part of code inside With that is inside Condition is considered a part of the "composite rule" and therefore not evaluated. So,

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval = f[n]},
    With[{eval = eval},Print[eval];g[eval]/;True]]
(* print: 4. )
( print: 9. )
( Hold[{Hold[g[4.]],Hold[g[9.]]}] *)

But

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval = f[n]},
   With[{eval = eval},(Print[eval];g[eval])/;True]]
(* Hold[{Hold[Print[4.];g[4.]],Hold[Print[9.];g[9.]]}] *)

This should further clarify this mechanism.

Answer 2

RuleCondition provides an undocumented, but very convenient, way to make replacements in held expressions. For example, if we want to square the odd integers in a held list:

Hold[{1, 2, 3, 4, 5}] /. n_Integer :> RuleCondition[n^2, OddQ[n]]
(* Hold[{1, 2, 9, 4, 25}] *)

RuleCondition differs from Condition in that the replacement expression is evaluated before it is substituted. The second argument of RuleCondition may be omitted, defaulting to True:

Hold[{2., 3.}] /. n_Real :> RuleCondition[n^2]
(* Hold[{4., 9.}] *)

It is very unfortunate that RuleCondition has remained undocumented for so long, given its extreme usefulness. The Trott-Strzebonski trick discussed in @Leonid's answer is one way to achieve the same result using only documented symbols:

Hold[{2., 3.}] /. n_Real :> With[{eval = n^2}, eval /; True]
(* Hold[{4., 9.}] *)

A slightly less verbose technique uses Block:

Hold[{2., 3.}] /. n_Real :> Block[{}, n^2 /; True]
(* Hold[{4., 9.}] *)

Judicious use of Trace reveals that both of these techniques ultimately resolve to RuleCondition. One must make up one's mind whether it is better to use the undocumented RuleCondition or rely upon implementation artifacts in With and Block. I suspect that the behaviour is unlikely to change in all three cases as so much Mathematica code depends upon the existing behaviour.

Answer 3

Although less magical, it can be done by ReplacePart

expr = Hold[{2, 3, 4, 5}]
pos = Position[expr, _Integer]
newparts = Extract[expr, pos] /. n_Integer :> n^2
ReplacePart[expr, Thread[pos -> newparts]]

Answer 4

Based on solutions proposed by @Leonid and @WReach I have noticed something peculiar with the use of Block and With for the Trott-Strzebonski solution.

Consider the following code:

f[x_] := x^2;
g[x_] := x^3;

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
Print[eval]; eval /; True
]

this will yield:

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

using Block will give the same result as above.

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
Print@eval; eval /; True
]

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

However, things become strange - as @Leonid pointed out - with the use of Composite expression in Condition:

using With prevents the evaluation of Print

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
(Print[eval]; eval) /; True
]

(* Hold[{Hold[Print[64.]; 64.], Hold[Print[729.]; 729.]}] *)

However, with the use of Block the Print statements get evaluated

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
(Print@eval; eval) /; True
] 

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

Now one begins to wonder how RuleCondition - as mentioned by @WReach - will behave with Composite expression

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> RuleCondition[Print[(g@*f)[n]]; (g@*f)[n]]

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

It turns out that RuleCondition behaves in a similar way to Block

So perhaps not a million dollar but a $5 question: Which one to use (Block, RuleCondition or With)?

In my opinion both have their own advantages. For instance, With will allow you to evaluate a part of the expression and at the same time inject unevaluated code in the substitution process.

With will enable code injection

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
(g[eval]; eval) /; True
]
(* Hold[{Hold[g[64.]; 64.], Hold[g[729.]; 729.]}] *)

In the example above (in spirit with what @Leonid mentioned) we substituted the Reals and also injected code that remained unevaluated i.e. g[64] and g[729]

Block and RuleCondition together with Sow and Reap

However, if the goal is to evaluate some code while making substitution 'Block` can be quite effective:

Reap[Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
  (Sow[PrimeQ@eval]; eval) /; True
  ]][[2, 1]]
];
(* {False,False} *)


Reap[
 Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
 (Sow[EvenQ[Round@eval]]; eval) /; True]
]
(* {Hold[{Hold[64.], Hold[729.]}], {{True, False}}} *)   

In the example above we could the result whether eval generates a Prime whenever the substitution is made. Likewise, this can be achieved using RuleCondition.

Answer 5

Trott-Strzebonski is great, but there is a case when some additional trick is required. At least I couldn't figure this out any other way...

Consider this slightly silly example. You have some expression/association/data structure etc. containing some code with side-effects or undesirable evaluation (basically anything that is ValueQ).

code = <|a :> Echo[1], b :> 1 + 2|>

So the only safe way to extract it is to use extra head argument to Lookup with Hold.

Lookup[code, a, None, Hold]
(* Hold[Echo[1]] *)

Or any other head with HoldAll. And there are other functions with the head argument exist for this exact purpose, another example would be Extract and ToExpression.

The goal is to insert values from code into another held expression based on keys without evaluating anything, like here:

expr = Hold[b, a + b]
...
(* Hold[1 + 2, Echo[1] + (1 + 2)]*)

Applying Trott-Strzebonski naively trying Unevaluated doesn't quite get there:

Hold[b, a + b] /. key_ /; KeyExistsQ[code, key] :> 
  With[{y = Unevaluated @@ Lookup[code, key, None, Hold]}, y /; True]
(* Hold[Unevaluated[1 + 2], Unevaluated[Echo[1]] + Unevaluated[1 + 2]] *)

One can use Position + Extract + ReplacePart here of course with some convoluted expression handling to make delayed rules for ReplacePart:

pos = Position[expr, key_ /; KeyExistsQ[code,key]]
With[{newparts = Flatten[Hold @@ (Lookup[code, #, None, Hold] & @@@ Extract[expr, pos, Hold])]},
  ReplacePart[expr, List @@ Thread[Hold @@ pos :> newparts, Hold]]
]

But here is another trick:

Hold[b, a + b] /. key_ /; KeyExistsQ[code, key] :> With[{
    bind = Function[Null, Unevaluated[{y := #}], HoldAll] @@
        Lookup[code, key, None, Hold]
},
    With[bind, y /; True]
]

The idea is to make a delayed unevaluated binding and then use Trott-Strzebonski with it!

I'm quite surprised it works, I figured it out recently myself and it doesn't seem to work with RuleCondition for some reason too.

One more extra note: instead of Function[Null, ..., HoldAll], GeneralUtilities`HoldFunction can be useful.

---

Just realized how silly this particular example is because association already provides you with the rules and one can just do:

expr /. code

But if the code was instead something like Hold[Echo[1], 1 + 2] and a procedure of extracting subcode from it depended on LHS of a rule and Extract[code, ...] in a much more complicated fashion, then this trick would still work.