Module - Symbols out of scope of lexical scoping

Question

The Problem

Consider the following code (1):

In[1]:= x = 2;
        Module[{x, y},
         y[x_] = x;
         y[1]
        ]
Out[2]= 2

I expected the answer to be 1 since I assumed that x would be replaced with a local variable. The code returns the expected result if I replace Module with Block. However, I do not see why x would be out of scope of a lexical scoping construct. The effect can be demonstrated in a more drastic way with the following code (2):

In[3]:= x = 2;
        Module[{x, y},
         y[x_] = (x = 1);
        ];
        x
Out[5]= 1

The x inside the Module is not replaced with a local version and the value of the variable on the outside of the module is changed instead.

My own attempt (Can be skipped)

to trace the problem:

In[6]:= x = 2;
        Module[{x, y},
         y[x_] = x;
         y[1]
        ] // Trace
Out[7]= {Module[{x,y},y[x_]=x;y[1]],{y$2016[x_]=x;y$2016[1],{{x,2},y$2016[x_]=2,2},{y$2016[1],2},2},2}

This shows that the only y is replaced with a local version.

Next I had a look at the Code (3) that the Kernel sees:

In[8]:= x = 2;
        Module[{x, y},
         y[x_] = x;
         y[1]
        ] // Hold // FullForm
Out[9]//FullForm= Hold[Module[List[x,y],CompoundExpression[Set[y[Pattern[x,Blank[]]],x],y[1]]]]

and contrasted it with the code (4):

In[10]:= x = 2;
         Module[{x, y},
          y = x;
          y[1]
         ] // Hold // FullForm
Out[11]//FullForm= Hold[Module[List[x,y],CompoundExpression[Set[y,x],y[1]]]]

From a lexical point of view the difference between code (3) and (4) is only y[Pattern[x,Blank[]]] vs y. However in (3) x is not a local variable and in (4) it is.

Since Module has the Attribute HoldAll, I expect it to treat its Body in a purely lexical way before any of the attributes of the expression inside the body or whatnot apply.

I am aware that Set has the attributes HoldFirst and SequenceHold, but I do not see why this would make a difference in this case since Module localizes the variables even for held expressions.

Does the Kernel see x as a Symbol?:

In[12]:= Clear[x];
         Hold[
         Module[{x, y},
          y[x_] = x;
          y[1]
         ]] // Cases[#, _Symbol, Infinity] &
Out[12]= {x,y,x,x}

This suggests to me that the x on the left and right hand side of the Set is a Symbol and that it should be replaced with a localized version.

Resources considered (Can be skipped)

Since the code is so simple I highly expect this question to be either a lack of my understanding or a duplicate. However unfortunately I could neither find the answer

in the documentation of Module
nor in What are the use cases for different scoping constructs?

nor in the various other questions I checked. A local function inside a Module is not uncommon, however, the combination of Set with a pattern on the left as a localized variable inside a Module seems quite specific. I did not encounter it elsewhere.

The Question

Please explain the result of code (1).

Thanks in advance for answers or helpful pointers.

Kuba · Accepted Answer · 2018-10-30T10:12:37.923

ref / Module / Details

Before evaluating expr, Module substitutes new symbols for each of the local variables that appear anywhere in expr except as local variables in scoping constructs

ref / Set / Details

When it appears in an unevaluated symbolic form, Set is treated as a scoping construct so that variables in nested occurrences are renamed if necessary. (* Kuba: or not scoped by outer scoping constructs *)

The common workaround is to use different names, use Block or to trick Module so it will not 'see' Set[_ , _]:

x = 2;
Module[{x, y}, Set @@ {y[x_], x}; y[1]]
  (*before evaluation, Module can only see `Apply[Set, {y[x_], x}]`*)

The issue is very similar to the automatic renaming feature, except it is the other way around :)

Enforcing correct variable bindings and avoiding renamings for conflicting variables in nested scoping constructs

Here is a comparison of those two features:

Module[{x, y}, Hold[y[x_] = x]] (*nested scoping constructs*)

Hold[y$32040[x_] = x]

Module[{x, y, z}, Hold[y[x_] = x + z]] (*automatic renaming*)

Hold[y$32041[x$_] = x$ + z$32041] (*note x$*)

Module - Symbols out of scope of lexical scoping

The Problem

My own attempt (Can be skipped)

Resources considered (Can be skipped)

The Question

1 Answers1