Why is head evaluation inconsistent in pattern replacement?

Question

I noticed the following unexpected result:

In[90]:= "foo" /. x_String -> Head[x]

Out[90]= Symbol

Why is it that Head[x] doesn't return String? I would assume that since the pattern matched on a String head, the right hand side of the rule should see the same thing. Furthermore, the following expression gives the same result (assuming foo is undefined):

In[103]:= foo /. x_Symbol -> Head[x]

Out[103]= Symbol

How does one distinguish strings from symbols in the replacement part of a rule in this kind of situation?

Answer 1

Short answer

You should use RuleDelayed:

"foo" /. x_String :> Head[x]

Othewise, the r.h.s of your rule is computed before the match, when x is still a symbol, not bound to anything.

Longer answer

Let's enumerate the main steps the main evaluator goes through, when evaluating your original code.

The case of Rule

• The head ReplaceAll is evaluated (to itself, trivially). Evaluation then goes to the elements (arguments of ReplaceAll), left to right.

• The first argument, "foo", is trivially evaluated to itself

• The second argument, Rule[x_String, Head[x]] is evaluated

  • The head Rule is trivially evaluated.

  • Both of its arguments are evaluated, since Rule does not hold arguments

    • x_String is evaluated to itself
    • Head[x] is evaluated to Symbol, since at this time, in your case, x was not assigned any value. If it was, Head[x] would have evaluated to the head of that value.

• Finally, the internal DownValues for ReplaceAll are allowed to fire, "on the way up" in the recursive evaluation procedure. By this time, the rule you want to apply, is

  x_String -> Symbol
  

  And thus the result.

The case of RuleDelayed

• The head ReplaceAll is evaluated (to itself, trivially). Evaluation then goes to the elements (arguments of ReplaceAll), left to right.

• The first argument, "foo", is trivially evaluated to itself

• The second argument, RuleDelayed[x_String, Head[x]] is evaluated

  • The head RuleDelayed is trivially evaluated.

  • Arguments of RuleDelayed start to evaluate from left to right. However, RuleDelayed is HoldRest, and that is important here.

    • x_String is evaluated to itself
    • Head[x] is not evaluated at this time, due to the HoldRest attribute of enclosing RuleDelayed.

• The internal DownValues for ReplaceAll are allowed to fire, "on the way up" in the recursive evaluation procedure. By this time, the rule you want to apply, is

  x_String :> Head[x]
  

  Now, ReplaceAll is in the position to bind the value of x on the r.h.s. with the result found during the pattern-matching of the pattern on the l.h.s. against your expression ("foo" in this case). This results in a new expression Head["foo"]

• Finally, Head["foo"] evaluates to String.

Can we get the right result with Rule?

To answer your specific question in comments, it also should be clear from the above discussion, that generally one should use RuleDelayed to get the right result, instead of Rule.

One thing you can do, if you insist on using Rule, is the following:

"foo" /. Unevaluated[x_String -> Head[x]]

(* String *)

This works in this case, but note that in this approach, x_String does not evaluate either. This is Ok here, but it is important to note the difference.

Summary

There are just a few things one needs to remember about Rule vs RuleDelayed:

• Use Rule, if you either

  • Do want the r.h.s. to evaluate before the pattern matching / replacement happens
  • Don't care, because for example the r.h.s of the rule is a constant expression, not containing any variables

• Use RuleDelayed in the majority of cases, when you have pattern variables on both sides of the rule.

So in general, when your patterns contain variables, it is RuleDelayed you typically want to use, not Rule.