The Flat Attribute, Unevaluated and the Evaluation Process

Question

To make this question easy to follow let's give some symbols a couple attributes.

Attributes[f] = {Flat};
Attributes[s] = {Orderless};

Now consider the following:

f[a, b, g[c, d], f[e, h], j, k] gives

f[a, b, g[c, d], e, h, j, k]

as expected since f is Flat

Now, we wrap Unevaluated around f

f[a, b, g[c, d], Unevaluated[f[e, h]], j, k] 

This gives

f[a, b, g[c, d], Unevaluated[e], Unevaluated[h], j, k]

To understand what happened here, we use Trace

f[a, b, g[c, d], Unevaluated[f[e, h]], j, k] // Trace

 {f[a, b, g[c, d], f[e, h], j, k], f[a, b, g[c, d], e, h, j, k], f[a,
 b, g[c, d], Unevaluated[e], Unevaluated[h], j, k]}

Finally, let's also wrap Unevaluated around s.

f[a, b, g[c, d], Unevaluated[f[e, h]], Unevaluated[s[n, m]], j, k]

f[a, b, g[c, d], Unevaluated[e], Unevaluated[h], 
 Unevaluated[s[n, m]], j, k]

Now, here comes my question/confusion. In the case of f, the Unevaluated head was removed, so the elements of f( that has a Flat attribute ) was now spliced in and the Unevaluated head replaced on the elements of f. According to the standard evaluation process, if any part of an expression has the form Unevaluated[expr], we replace that part with expr and keep a record of the original expression (So we can later replace the Unevaluated head). Now as I've shown, if the expression expr has a head with Flat Attribute, then the expression gets spliced in (i.e. it actually gets evaluated) but as we saw with s that has the Orderless Attribute, the expression remains unevaluated as described by the Evaluation process. So why does the Flat attribute supersede Unevaluated? This shouldn't be the case according to the standard evaluation process. What am I missing?

Answer 1

This is the expected behaviour of Unevaluated, but is not fully covered in the documentation. Unevaluated is a special head that changes the attributes temporarily so that the function f holds its argument. It is supposed to work as if you had given it one of the HoldFirst/HoldAll, etc. attribute, but only for that evaluation. The key point though, is that the function f does not know about Unevaluated. This is covered in David Wagner's book, "Power programming with Mathematica: The Kernel":

The beauty of Unevaluated is that it is invisible to the function being called — it makes the function temporarily behave as though it had one of the Hold- attributes. [...] Unevaluated may seem similar to Hold, but there is an important difference: Hold (or HoldForm) is visible to the function being called. [...]

[...] the special behavior of Unevaluated is accomplished by removing the head Unevaluated from an expression before rules for the surrounding expression are applied and restoring the head to the expression afterward if no applicable rules were found.

So what happens is:

• The head Unevaluated is removed and f given a temporary Hold like attribute
• The existing rules for f are applied. This causes the expression to be flattened out, because of its Flat attribute.
• Unevaluated is restored to the expression, or whatever remains of it now.

This is exactly what you also see in the Trace output.

You can simply change the outermost head to verify that this behaviour is due to Unevaluated being invisible to the outer function f:

g[a, b, g[c, d], Unevaluated@f[e, h], s[n, m], j, k]
(* g[a, b, g[c, d], Unevaluated[f[e, h]], s[m, n], j, k] *)