pgfextra vs. graph: a paradox!

Question

Consider the following LaTeX manuscript featuring a TikZ picture of two filled circles. The right circle is drawn inside a \pgfextra. The path's fill color is yellow!

\documentclass{standalone}
\usepackage{tikz}
\begin{document}
    \begin{tikzpicture}
        \path[fill=yellow] (-.5,0) circle(1)
            \pgfextra{\fill (.5,0) circle(1);};
    \end{tikzpicture}
\end{document}

It renders as

                                                

If we now replace the \pgfextra with a graph:

\documentclass{standalone}
\usepackage{tikz}
\usetikzlibrary{graphs}
\begin{document}
    \begin{tikzpicture}
        \path[fill=yellow] (-.5,0) circle(1)
            graph{""[circle,at={(.5,0)},minimum size=2cm,fill]};
    \end{tikzpicture}
\end{document}

we obtain the picture

                                                   

Now here's the paradox.

In both examples the path has the same form:

\path[fill=yellow] (-.5,0) circle(1) ...;

One would expect this to be implemented in one of two ways: either the option is executed at the beginning:

\pgfsetfillcolor{yellow}
\pgfpathcircle{\pgfpoint{-.5cm}{0cm}}{1cm}
...
\pgfusepath{fill}

or just before the path is used:

\pgfpathcircle{\pgfpoint{-.5cm}{0cm}}{1cm}
...
\pgfsetfillcolor{yellow}
\pgfusepath{fill}

The first alternative must be rejected, since then the first example would be implemented as follows

\pgfsetfillcolor{yellow}
\pgfpathcircle{\pgfpoint{-.5cm}{0cm}}{1cm}

% \pgfextra{\fill (.5,0) circle(1);}
\pgfpathcircle{\pgfpoint{.5cm}{0cm}}{1cm}
\pgfusepath{fill}

\pgfusepath{fill}

yielding the picture

                                                   

But the second alternative must also be rejected, because then the implementation of the second example would see the graph created before the fill color has been set:

\pgfpathcircle{\pgfpoint{-.5cm}{0cm}}{1cm}

% graph{""[circle,at={(.5,0)},minimum size=2cm,fill]}

\pgfsetfillcolor{yellow}
\pgfusepath{fill}

and since a graph is locked in a TeX box upon creation, and since the visual appearance of a TeX box is finalized upon creation, the graph node would be filled with black, and the picture would look something like this

                                                   

We have arrived at a contradiction!

Answer 1

\pgfextra does not surround it's contents with a pgfinterruptpath environment, so in:

\path[fill=yellow] (-.5,0) circle [radius=1]
  \pgfextra{\fill [red] (.5,0) circle [radius=1];};

the "yellow" path is effectively continued by the path described by the path inside \pgfextra. This path is then used by "red" path and filled red. Note, that adding a path inside \pgfextra with or without the pgfinterruptpath environment is a really bad idea, the behaviour is essentially undefined. The intended use of \pgfextra is mostly for calculations or some simple TeX stuff, not arbitrary TikZ commands. In addition, with respect to \pgfextra the manual cautions (p.162, my emphasis):

...this operation should only be used by real experts and should only be used deep inside clever macros, not on normal paths.

By using

\path[fill=yellow] (-.5,0) circle [radius=1]
  \pgfextra{
    \pgfinterruptpath
      \fill [red] (.5,0) circle [radius=1];
    \endpgfinterruptpath
  };

A yellow and red circle will be drawn.

Answer 2

The TikZ "engine" parses a \path statement from left to write, executing the parts in their textual order. What "executing" means depends on the part, for instance

• nodes are typeset inside a TeX box which is saved to a register. In the beginning of a node's execution all path options except for font, text and behind path are temporarily cleared and restored at the end of the node's execution, after it has been typeset. Thus, a node is typeset based solely on the options explicitly applied to the node (allowing for the exceptions mentioned above).
• Path-construction operations such as circle(1) are converted to \pgf... commands that are appended to an internal list object called the current path (this is a gross over-simplification, but it will do for the present purposes). It is important to know that a \path statement is wrapped in an implicit TeX scope, however the current path list is global, so it transcends scopes.

• \pgfextra{<body>} temporarily returns control to the normal TikZ processor (as opposed to the path parser), passing it <body> as the text to be processed next. The current TeX scope is not exited, and no new scope is opened. The internal state of the engine at the time the \pgfextra was encountered is not altered, so all the internal data structures created so far while parsing the \path statement remain intact.

  When TikZ finishes processing <body>, control returns to the path parser whose cursor is now set just to the right of the \pgfextra{<body>} expression. Again, the transition back to normal path parsing is not accompanied by any changes to the state of the engine and no scope is closed, since none was opened to begin with.

Options are executed by the \tikzset command. What this entails depends on the option. For instance

• The behind path option makes an internal change to the operation of the TikZ engine: it makes the engine typeset the nodes that are flagged with this option inside a different box, called the background box, than the one in which the other nodes are typeset.

• The fill=<color> option appends the command \pgfsetfillcolor{<color>} to a special list, the options list, consisting of similar basic-layer commands. This list has no effect on the operation of the TikZ engine. It is simply accumulated and eventually written to the dvi file as-is (after the \pgf... commands have been substituted by their \special implementation). Unlike the current path, the options list is local to the TeX scope that implicitly surrounds the \path command, and is set to the empty list in the beginning of this scope.

Thus, the execution of a path part usually means "translating" it into a sequence of low-level commands, and saving this sequence either as a new object, or by appending it to a pre-existing list.

The execution of the semicolon at the end of the TikZ path is special. It entails actually writing all the path parts to the dvi file (this is a gross over-simplification, but it will do for our purposes). Since by the time the semicolon is executed, the execution of the various parts has already resulted in their translation to low-level commands, the main job of the semicolon is to determine the order in which these lists will be dumped to the dvi file.

A simplified version of the semi-colon's execution is:

1. Write out nodes (including graphs, which are treated as glorified nodes) marked with behind path.

2. Write out the options list.

3. Write out the current path.

4. Use the path, i.e. draw it, fill it, etc. As a side effect, the current path object (which, recall, is global) is reset.

5. Write out the rest of the nodes (and graphs).

6. Close the TeX scope that implicitly surrounds the path statement, and restore the graphics state that prevails at the top-level of the TikZ environment. In particular, the default fill-color for the TikZ environment is restored at the end of each path, together with many other defaults of the TikZ environment.

---

Let's now apply all this theory to the examples in the original post.

First example

\path[fill=yellow] (-.5,0) circle(1) \pgfextra{\fill (.5,0) circle(1);};

1. The fill=yellow option is executed. This results in the instruction \pgfsetfillcolor{yellow} being appended to the initially empty options list.

2. (-.5,0) circle(1) is executed. This results in the instruction \pgfpathcircle{\pgfpoint{-.5cm}{0pt}}{1cm} being added to the initially empty current path.

3. \pgfextra{\fill (.5,0) circle(1);} is executed. This causes TikZ to temporarily "forget" that it was in the middle of parsing a path, and to start processing the code \fill (.5,0) circle(1); as if it was written at top-level. However the engine's internal state remains intact, so, in particular, the options list and the current path remain intact.

   1. (.5,0) circle(1) is executed. This results in the instruction \pgfpathcircle{\pgfpoint{.5cm}{0pt}}{1cm} being appended to the current path object, which already contains a circle at (-.5,0).

   2. The semicolon at the end of the path is encountered. This brings about the following sequence of actions.

   3. Firstly, the behind path nodes are written to the dvi files. There are none, so this is a no-op.

   4. Secondly, the options list is written to the dvi file. This is not the same list as the one mentioned above, since options lists are local to the implicit scope that surrounds a \path statement, and since they are set to the empty list in the beginning of this scope. Since no option has been specified for the path \fill (.5,0) circle(1);, the options list is empty, and so this step is a no-op.

   5. Thirdly, the path-construction commands for the current path (consisting of two circles) are written to the dvi file.

   6. Fourthly, the path is filled, i.e. the \specials implementing the command \pgfusepath{fill} are written to the dvi file. Since the top-level option fill=yellow has not yet been applied, the default color is used. In the present case, since no default color has been explicitly specified, either for the TikZ picture or for the document, black is the implicit default color (for this is the default color for all PDF documents). At the end of this step, the (global) current path object is reset, i.e. the current path becomes empty.

   7. Fifthly, the rest of the nodes are written to the dvi file. There are none, so this is a no-op.

   8. Sixthly, the TeX scope that implicitly surrounds the \fill ... statement is closed, and the default settings for the TikZ environment are restored.

4. The top-level semicolon is executed. However, since there are no nodes, and since the current path is empty, there's nothing left to do except writing the options list to the dvi file. However this has no overall effect: the options list changes the fill-color to yellow, but this change is then overturned when the default graphics state for the TikZ environment is restored as the last step in the semicolon's execution.

Second example

\path[fill=yellow] (-.5,0) circle(1)
    graph{""[circle,at={(.5,0)},minimum size=2cm,fill]};

1. As in the first example.

2. As in the first example.

3. The graph is executed. A graph's execution is similar to a node's. Therefore, the graph is typeset inside a TeX box that is saved to a register. When the graph's execution begins, the path options (fill=yellow) are temporarily cleared, so the fill-color does not get set inside the box. For this reason, whatever the fill-color will be at the time the box's contents are spilled to the dvi file will be the one used to render the graph.

4. The semicolon is executed:

   1. behind path nodes are written to the dvi file. There are none, so this is a no-op.

   2. The options list, i.e. \pgfsetcolor{yellow}, is written to the dvi file.

   3. The path-construction commands for the current path, consisting of a circle centered at (-.5,0), are written to the dvi file.

   4. The path is used: the command \pgfusepath{fill} is written to the dvi file. When the dvi file will be rendered on the screen, a single yellow circle will appear at this point. At the end of this step, the (global) current path object is reset, i.e. the current path becomes empty.

   5. The rest of the nodes (and graphs) are written to the dvi file. So now the contents of the graph's box get written out to the dvi file. Since the fill color is not set inside the box, the fill color that will be used is the last one that has been set, namely yellow.

   6. The TeX scope that implicitly surrounds the \path ... statement is closed, and the default settings for the TikZ environment are restored. In particular, the fill-color is reset to the default black.

Third example (Mark Wibrow's example)

\path[fill=yellow] (-.5,0) circle [radius=1]
  \pgfextra{
    \pgfinterruptpath
      \fill [red] (.5,0) circle [radius=1];
    \endpgfinterruptpath
  };

1. As in the first example.

2. As in the first example.

3. The \pgfextra block is executed. This causes TikZ to temporarily "forget" that it was in the middle of parsing a path, and to start processing the \pgfextra body as if it was written at top-level. However the engine's internal state remains intact, so, in particular, the options list and the current path remain intact.

   1. \pgfinterrupt starts a new TeX block, and saves the current path (consisting of a circle centered at (-.5,0)) in a local "private variable", and resets the (global) current path to the empty path.

   2. The option red is executed. This effectively adds the instructions \pgfsetfillcolor{red} and \pgfsetstrokecolor{red} to the options list. This is not the same list as the one mentioned above, since options lists are local to the implicit scope that surrounds a \path statement, and since they are set to the empty list in the beginning of this scope. So at the end of this step, the visible options list consists of only two commands, and these commands set the fill- and stroke color to red.

   3. (.5,0) circle [radius=1] is executed. This results in the instruction \pgfpathcircle{\pgfpoint{.5cm}{0pt}}{1cm} being appended to the empty current path object.

   4. The semicolon at the end of the path is encountered. This causes the current path, consisting of a single circle centered at (.5,0) to be written to the dvi file followed by an instruction to fill this circle with red. After this, the current path is set to the empty path. When the dvi file will be rendered on the screen, a single red circle will appear at this point.

   5. The implicit TeX block that surrounds the \fill [red] (.5,0) circle [radius=1]; command is closed. Since options lists are local to the implicit scope that surrounds a \path statement, the options list is replaced by the previous one consisting of the single command to set the fill color to yellow.

   6. \endpgfinterruptpath ends the TeX block opened by \pgfinterruptpath and sets the global current path object back to the one saved by the \pgfinterruptpath command, namely the one consisting of a circle centered at (-.5,0).

4. The top-level semicolon is executed:

   1. behind path nodes are written to the dvi file. There are none, so this is a no-op.

   2. The options list, i.e. \pgfsetcolor{yellow}, is written to the dvi file.

   3. The path-construction commands for the current path, consisting of a circle centered at (-.5,0), are written to the dvi file.

   4. The path is used: the command \pgfusepath{fill} is written to the dvi file. When the dvi file will be rendered on the screen, a yellow circle will appear at this point. It will be drawn on top of and slightly to the left of the red circle that was drawn before. At the end of this step, the (global) current path object is reset, i.e. the current path becomes empty.

   5. The rest of the nodes (and graphs) are written to the dvi file. There are none, so this is a no-op.

   6. The TeX scope that implicitly surrounds the \path ... statement is closed, and the default settings for the TikZ environment are restored. In particular, the fill-color is reset to the default black.

The resulting picture is