Why rule does not work if Flat attribute?

Question

The following works:

In[39]:= ClearAttributes[M, Flat];
Replace[M[F1[a]], M[x_F1] | M[x_F2] -> x]
Replace[M[F2[a]], M[x_F1] | M[x_F2] -> x]
Replace[M[F3[a]], M[x_F1] | M[x_F2] -> x]

Out[40]= F1[a]

Out[41]= F2[a]

Out[42]= M[F3[a]]

the following does not

In[34]:= ClearAttributes[M, Flat];
SetAttributes[M, {Flat}];
Replace[M[F1[a]], M[x_F1] | M[x_F2] -> x]
Replace[M[F2[a]], M[x_F1] | M[x_F2] -> x]
Replace[M[F3[a]], M[x_F1] | M[x_F2] -> x]

Out[36]= M[F1[a]]

Out[37]= M[F2[a]]

Out[38]= M[F3[a]]

why?

Answer 1

I researched Flat a little bit about a year ago when I wrote this answer. It isn't as clear as can be. It needs updating, and perhaps I'll do that soon. In any case, taking the relevant paragraph:

When the pattern matcher finds itself comparing arguments of an expression whose head (let's call it head) has attribute Flat, it behaves differently: patterns of length one (_, r_, Conditions, PatternTests) trigger the pattern matcher to automatically wrap a head around the respective arguments of the expression so both the expression and the pattern have the same number of arguments (could also be a single argument, but it never leaves it as is. Don't know the purpose). In cases where more than one option is possible due to having __s as arguments, it just starts trying one way and if it doesn't match, tries the next).

In[47]:= ClearAll[f]; SetAttributes[f, {Flat, HoldAllComplete}];
         Cases[Hold@f[1], Hold@f[i_] :> Hold[i], {0}]
         Cases[Hold@f[1, 2, 3, 4], Hold@f[1, i_, 4] :> Hold[i], {0}]
         Cases[Hold@f[1, 2, 3, 4], Hold@f[i_, __] :> Hold[i], {0}]
         Cases[Hold@f[1, 2, 3, 4], 
            Hold@f[i_, j__ /; Length[{j}] === 2] :> Hold[i], {0}]

Out[48]= {Hold[f[1]]}

Out[49]= {Hold[f[2, 3]]}

Out[50]= {Hold[f[1]]}

Out[51]= {Hold[f[1, 2]]}

This case

See this

MatchQ[M[F1[a]], M[_F1]]
MatchQ[M[F1[a]], M[_M]]

(* False *)

(* True *)

The fact that your blank _F1 only matches with head F1 doesn't change the fact that it's a blank, so it actually tries to match M[M[F1[a]]] to M[_F1] and fails. You can use M[F1[___]] to prevent this (or set OneIdentity if appropriate)

Answer 2

I don't understand why OneIdentity doesn't work for you but this trick appears to work:

SetAttributes[M, Flat];

rule = M[x : Repeated[_F1, {1}]] | M[x : Repeated[_F2, {1}]] :> x;

Replace[M[F1[a]], rule]
Replace[M[F2[a]], rule]
Replace[M[F3[a]], rule]

F1[a]
F2[a]
M[F3[a]]

Please make sure to use RuleDelayed (:>) as I did above when working with named patterns on the right-hand side.