The TikZ pipeline: A path's 'draw' option has no effect on a 'behind path' node

Question

Still trying to figure out the TikZ pipeline (i.e. the sequence of low-level actions that a TikZ path is converted to as it is being "executed") by analysing the source code, specifically <tex installation directory>/tex/generic/pgf/frontendlayer/tikz/tikz.code.tex...

The \tikz@handle macro is dubbed (in the comment preceding this macro) the "Central dispatcher for commands". It is the engine driving the "execution" of a TikZ picture. It works by identifying the smallest semantically meaningful chunk at the beginning of the input stream, and then calling the appropriate handler. When the handlers finish processing, they call \tikz@scan@next@command to move the input stream "cursor" past the just-processed chunk, and then call \tikz@handle recursively.

It is important to realize that the input stream is scanned linearly from start to end, always moving the "cursor" forward, and that the next action to take place is the one determined by the chunk at the current "cursor".

\tikz@handle is implemented as a long, but simple switch statement:

\def\tikz@handle{%
  \let\@next=\tikz@expand%
  \ifx\pgf@let@token(%)
    \let\@next=\tikz@movetoabs%
  \else%
    \ifx\pgf@let@token+%
      \let\@next=\tikz@movetorel%
    \else%
      \ifx\pgf@let@token-%
        \let\@next=\tikz@lineto%
      \else%
        \ifx\pgf@let@token.%
          \let\@next=\tikz@dot%
        \else%
          \ifx\pgf@let@token r%
            \let\@next=\tikz@rect%
          \else%
            \ifx\pgf@let@token n%
              \let\@next=\tikz@fig%
            \else%
              \ifx\pgf@let@token[%]
                \let\@next=\tikz@parse@options%
              \else%
              ...
            \fi%
          \fi%
        \fi%
      \fi%
    \fi%
  \fi%
  \@next%
}

As can be seen in the last ifx clause above, when options, identified by an opening square bracket, are encountered, the \tikz@parse@options handler is invoked. This simple macro is defined as follows:

\def\tikz@parse@options#1]{%
  \tikzset{#1}%
  \tikz@scan@next@command%
}

It does nothing but pass the options to \tikzset, which simply processes the options using the \pgfkeys command, documented in detail in Section 82 of the TikZ & PGF manual for version 3.0.1a, with the default path set to /tikz (see p. 128).

All this boils down to the following conclusion. The "execution" of a path of the form

\path[<options>] ...;

starts with the <options>. The <options> are executed before any other element on the path.

Equipped with this understanding, let's consider the following LaTeX manuscript featuring a simple TikZ picture. The picture consists of a path that is comprised of a single, empty node at the origin with a drawn border. The path has a single option: the draw color is set to red.

\documentclass{standalone}
\usepackage{tikz}
\begin{document}
    \tikz \path[draw=red] (0,0) node[draw] {};
\end{document}

Based on the conclusion highlighted above, we expect the draw=red option to be executed before the node is created, and therefore the node border's color should be red. Our prediction is spot on:

Let's now add the option behind path to the node:

\tikz \path[draw=red] (0,0) node[draw,behind path] {};

This makes no difference to our analysis, and we therefore expect the same picture as the one generated before. Alas, our expectations are crushed: the border is rendered black!

Why, oh why? The added option signifies (p. 215 of the manual) that the node should be drawn behind the path. In the present case this is moot, since there is no path (or, more precisely, the path is degenerate, consisting of a single point at the origin), but even if there were a path, it would not matter: the node's border would still come out black, despite having established that the path options (draw=red in this case) are executed first.

---

Addendum

After some digging around the source code and experimenting I've come to realize that the story is stranger still than what I presented above, and that the real question I should ask is not: 'Why does the node's border in the 2nd example black?', but: 'Why does the node's border in the 1st example red?'

For, you see, it turns out that in the beginning stages of processing a node - every node - the options are cleared, including the stroke-color option, if set, and they remain unset all while the node is being rendered and eventually locked in a TeX box. This is true whether or not the behind path option has been specified.

This means that even in the first example, the one seen above to have a red border, when you view the node after it has just finished rendering and has been enclosed in a TeX box, it looks exactly like the 2nd picture above, the one with the black border.

So how does a black-bordered node that has been locked in a TeX box (which is itself nested in another TeX box), transform later into a red-bordered node? Can a Cushite change his skin, or a leopard its spots?

To flesh out the last four paragraphs' claims with references to the source-code:

• The point in the beginning stages of a node's processing where the options are cleared, is in the macro \tikz@normal@fig:

  \let\tikz@options=\pgfutil@empty
  

  You can verify that, just before this line, \tikz@options contains the instruction \pgfsetstrokecolor{red}, by prepending

  \wlog{options: \meaning\tikz@options}%
  

• The rendering and locking-in-a-TeX-box of a node take place inside the \tikz@fig@continue macro:

  \setbox\tikz@whichbox=\hbox{%
    \unhbox\tikz@whichbox%
    \hbox{{%
        \pgfinterruptpath%
          \pgfscope%
            \tikz@options%
            \setbox\tikz@figbox=\box\pgfutil@voidb@x%
            \setbox\tikz@figbox@bg=\box\pgfutil@voidb@x%
            \pgfmultipartnode{\tikz@shape}{\tikz@anchor}{\tikz@fig@name}{%
              \pgfutil@tempdima=\pgflinewidth%
              {\begingroup\tikz@finish}%
              \global\pgflinewidth=\pgfutil@tempdima%
            }%
          \endpgfscope
        \endpgfinterruptpath%
    }}%
  }%
  

  To view the node's appearance just after it has been rendered and locked in a TeX box, add the following line immediately after the above code:

  \shipout\expandafter\copy\tikz@whichbox%
  

  An extra page will be prepended to the final pdf file, showing this box's contents.

  By the way, you should verify that the \tikz@options above are empty, even when the behind path options has not been given!

• The code that renders the node, shown in the previous bullet-point, delegates to the \tikz@finish macro. This macro is called twice: firstly to render the node, and later to render the overall path. If you \shipout the node's box (\tikz@whichbox) just before it is rendered, which, in the 2nd example, happens in the following line:

  % Step 13: Add labels and nodes
  %
  \box\tikz@figbox%
  

  (\tikz@whichbox expands to \tikz@figbox when the behind path option is not specified), you will notice that the node's border is black! (Note that you will get an extra empty page in the beginning of the pdf file, since when the node is being rendered this box is empty; only once the node has been rendered does this box come to be, enclosing the node.)

  You can verify that no options are applied when the node is being rendered, by appending \wlog{here!} to the end of the ifx branch, and \wlog{there!} just after the \else keyword in the following:

  %
  % Step 3: Setup options
  %
  \ifx\tikz@options\pgfutil@empty%
  \else%
    \pgfsys@beginscope%
      \let\pgfscope@stroke@color=\pgf@strokecolor@global%
      \let\pgfscope@fill@color=\pgf@fillcolor@global%
      \begingroup%
        \tikz@options%
  \fi%
  

  The log will contain two entries: 'here!' and 'there!', resulting from the rendering of the node and path, respectively.

Answer 1

As mentioned in the original post, all path options are turned off locally in the beginning stages of a node's processing. This is true for all nodes, even those not tagged with behind path. By 'locally' I mean that after the node has finished processing, once TikZ resumes scanning the rest of the path, the options are restored to their original values. There are a few exceptional options that are not affected by this, and retain their values, e.g. font, text (I think these are the only exceptions, but I'm not sure), but most options, including the stroke- and fill-color will be turned off in the beginning, and, unless explicitly set as node-options, will remain so till TikZ has finished processing the node.

It should be noted that the behind path path-option is also exceptional in the sense that, while it is technically turned off in the beginning of a node's processing, like all the other path-options, its effect is not canceled, because the turning-off of options just means that no \special directive is emitted for them, however behind path is not translated to a \special directive, but affects the operation of the TikZ engine itself.

It should also be noted that the path-options that are turned off in the beginning of a node's processing are only those that have been defined textually before the node. Those path-options that are defined after the node are not turned off because they have not yet been set, since, as mentioned in the original post, the path is scanned and "executed" linearly. This point is moot for most options, because what's the difference between an option that has been turned-off and one that has not yet been set!? However, the 'exceptional' options (font, text) as well as behind path do behave differently depending on their placement on the path relative to the node. For instance \path[text=blue] node{hello}; will color the text blue, whereas \path node{hello} [text=blue]; won't (unless the default font color has been set to blue beforehand).

Only the node-options as well as the exceptional path-options mentioned above (font and text, if set) will be converted to \special directives for rendering the node. If the PDF or PostScript viewer requires further, unspecified information, such as the stroke color (if the stroke color was not given as a node-option), in order to render the node on the page or screen, the settings that prevailed just before the node began rendering will be used. Thus, both explicit and implicit options are used to render a node.

When TikZ finishes scanning the chunk of text describing the node, the node is typeset inside a TeX box. This means that the \specials needed to render the node are generated and "locked" in the TeX box. From now on the box's contents will remain fixed: nothing will be added, removed, or changed. Moreover, the box's contents contain only literals; they are free of variables, registers, macros, and any other TeX elements that are susceptible to change. Finally, the node's box is given a name (or rather, a number; at any rate, it can be referenced later on).

So far, the node has been rendered inside a named TeX box, but not yet on the page. If the page were to be printed now, the node would not appear on it.

At this point TikZ resumes scanning the path. When the path ends (recognized by a semicolon), all of the path's parts, including the nodes, are rendered on the page. The nodes tagged with behind path are the first to render. They are rendered by simply appending their boxes' contents to the output stream. Then the path's options are set, by which I mean that they are converted to \specials and appended to the output stream. Then the path itself is stroked or filled. Finally, the rest of the nodes are rendered, by appending their boxes' contents to the output stream. This is not the full story, but is sufficient for our present purposes.

When a PDF or PostScript viewer prints or displays the TikZ picture, it does so by carrying out the instructions specified by the \specials in the order in which they were written to the dvi file. In particular, the behind path nodes will be rendered before the path options are executed, and therefore any missing information will be inferred from the graphics state that prevailed before. In particular, if the stroke-color was not specified as a node-option, it will be "inherited" from the graphics state. In the 2nd example given in the original post, this color is the document's default color, namely black, but this can be changed, e.g. by setting an option on the entire TikZ picture (e.g. \tikz[color=blue] \path...) or by establishing the document's default color before the TikZ picture (\color{blue} \tikz \path ...).

However, the rest of the nodes, the ones that were not tagged with behind path will be rendered after the path options have been set, and therefore the path options will be used implicitly whenever explicit node-options are not available. This is the case in the first example.