problem with Shorthand for Scope Environments

Question

I try in this in this question to use some shorthands with the library scopes.

I discover some difficulties and I can't explain why I get problems. (I use pgf 2.1 cvs)

The main code comes from the pgfmanual

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{scopes} 
\begin{tikzpicture}
  { [ultra thick]
    { [red]
      \draw (0mm,10mm) -- (10mm,10mm);
      \draw (0mm,8mm)  -- (10mm,8mm);
}
    \draw (0mm,6mm) -- (10mm,6mm);
  }
  { [green]
    \draw (0mm,4mm) -- (10mm,4mm);
    \draw (0mm,2mm) -- (10mm,2mm);
    \draw[blue] (0mm,0mm) -- (10mm,0mm);
  }
\end{tikzpicture}
\end{document} 

This is perfect. Now I want to draw three times these lines

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{scopes}
\begin{document}
\begin{tikzpicture}
 \foreach \i in {0,5,10}{%
  \begin{scope}[xshift=\i cm]
      { [ultra thick]
         { [red]
           \draw (0mm,10mm) -- (10mm,10mm);
           \draw (0mm,8mm)  -- (10mm,8mm);
     }
         \draw (0mm,6mm) -- (10mm,6mm);
       }
       { [green]
         \draw (0mm,4mm) -- (10mm,4mm);
         \draw (0mm,2mm) -- (10mm,2mm);
         \draw[blue] (0mm,0mm) -- (10mm,0mm);
       }     
  \end{scope}}
\end{tikzpicture}
\end{document} 

This is always perfect but now if I replace \begin{scope} ...\end{scope} by {..}, the code compiles but the scopes disappear.

\begin{tikzpicture}
 \foreach \i in {0,5,10}{%
  {[xshift=\i cm]
      { [ultra thick]
         { [red]
           \draw (0mm,10mm) -- (10mm,10mm);
           \draw (0mm,8mm)  -- (10mm,8mm);
     }
         \draw (0mm,6mm) -- (10mm,6mm);
       }
       { [green]
         \draw (0mm,4mm) -- (10mm,4mm);
         \draw (0mm,2mm) -- (10mm,2mm);
         \draw[blue] (0mm,0mm) -- (10mm,0mm);
       }     
  }}
\end{tikzpicture} 

I know that { is the beginning of a TikZ's scope only if [ comes after { otherwise{..}` is a simple TeX's group.

Is-it possible to explain this problem?

Answer 1

To answer this, you need to know when TikZ looks for the {[...] ... } scoping shortcut. The TikZ parser has many different states and it has to be in the right state to recognise certain syntax, otherwise it either passes over it or complains vociferously.

The check for the scoping shortcut is handled by a macro \tikz@lib@scope@check. Without the scopes library, this is empty. With the scopes library then it becomes equivalent to:

• Look at the next (non-space) token on the stream.
  1. Is it \tikz@intersect@finish? If so, do that and check again.
  2. Is it \par? If so, do that and check again.
  3. Is it \bgroup? If so, look for [ and if found, go into a scope.

An important thing to note is that this is a test on the next token, and there is only limited support for delaying the test.

Now let us see when this is invoked:

1. When a tikzpicture starts. So the first thing in a TikZ picture can be {[red] and that will get detected correctly.
2. When a scope starts. This can be an implicit or explicit one (so nesting of {[..] ..} syntax is fine, as is intermingling with ordinary \begin{scope} .. \end{scope} pairs).

3. When a scope ends. Actually, this is less useful than it at first seems. The problem is that an explicit scope ends within a group so the next token is not the next thing that the user thinks it is but probably an \endgroup (or something else buried in the \end{environment} code). To make use of this, the previous scope has to be either an implicit scope or of the form \scope ... \endscope. In the following, the first line is not red. Note that this can have knock-on effects: if the first implicit scope is not recognised and another follows it, that will also not be recognised.

   \begin{scope}
   \end{scope}
   {[red]
   \draw[ultra thick] (0,0) -- (0,1);
   }
   \scope
   \endscope
   {[red]
   \draw[ultra thick] (2,0) -- (2,1);
   }
   {[red]
   \draw[ultra thick] (1,0) -- (1,1);
   }
   

4. After a path. So an implicit scope can follow a path command.

Unfortunately, none of those match your syntax because by the time the \foreach is processed, the check that is made at the start of the picture has been made and failed (since it found \foreach). So to get the initial implicit scope recognised (and, incidentally, the fact that the first doesn't get recognised has the knock-on effect that none of the others do either: the [ultra thick] one doesn't follow the opening of a scope, so then neither does the [red] one, and the [green] one doesn't follow the ending of a scope so it isn't recognised) we need to invoke the recognition code. Two ways to do this are by ensuring that one of the conditions is met. Either an explicit but groupless scope has to start the \foreach (so \scope\endscope) or an empty path (so \path;).

These might be considered a little "hackish". There is an alternative. This is to ensure that the actual check is carried out at the start of the \foreach command. As the pgffor routines are meant to stand to one side of TikZ, this is not - and should not - be explicitly in the pgffor.code.tex file. But what is there are some hooks that can be invoked: \pgffor@beginhook and \pgffor@endhook (and \pgffor@afterhook). Indeed, these are used by TikZ for when \foreach is encountered inside a path. But they aren't used for external encounters. Since the internal one overwrites these hooks, we can just set them for the picture. As we want to make sure that the scope check is the very last thing that is added to these hooks, I've chosen a slightly circumspect way of doing it:

\tikzset{every picture/.append style={
    execute at begin picture={
      \expandafter\def\expandafter\pgffor@beginhook\expandafter{\pgffor@beginhook\tikz@lib@scope@check}
    }
  }
}

This means that when the picture starts, the code for adding the code is added to the code that is executed at the start of the picture. So it will be added after anything that is added either globally or in the options at the start of the picture. I'm sure it could be done better!

With that, then your code works:

\documentclass{article}
%\url{http://tex.stackexchange.com/q/57289/86}
\usepackage{tikz}
%\usepackage{trace-pgfkeys}
\usetikzlibrary{scopes}
\makeatletter
\tikzset{every picture/.append style={
    execute at begin picture={
      \expandafter\def\expandafter\pgffor@beginhook\expandafter{\pgffor@beginhook\tikz@lib@scope@check}
    }
  }
}
\makeatother
\begin{document}
\begin{tikzpicture}
 \foreach \i in {0,5,10}{%
  {[xshift=\i cm]
      { [ultra thick]
         { [red]
           \draw (0mm,10mm) -- (10mm,10mm);
           \draw (0mm,8mm)  -- (10mm,8mm);
     }
         \draw (0mm,6mm) -- (10mm,6mm);
       }
       { [green]
         \draw (0mm,4mm) -- (10mm,4mm);
         \draw (0mm,2mm) -- (10mm,2mm);
         \draw[blue] (0mm,0mm) -- (10mm,0mm);
       }     
  }
  }
\end{tikzpicture}
\end{document} 

Finally, the trace-pgfkeys was very useful here as it showed straight away that the scopes weren't getting seen at all.