Parsing the Main Branch of Single-Branched SGF Files (with Regex?)

Question

What I'm trying to accomplish is to parse an SGF file (a text file) into a TikZ diagram, something like this (ideally, I would point to the SGF file itself, instead of inlining a string):

\parsesgf{(;GM[1]FF[4]CA[UTF-8]AP[Sabaki:0.52.2]KM[6.5]SZ[19]DT[2024-02-05];B[as];W[bs];B[cs])}

Becoming, under the hood, this (I've added some boilerplate as well just so we have a minimal, complete example):

\documentclass{article}
\usepackage{tikz}
\newlength{\step}
\begin{document}
  \begin{tikzpicture}
    \setlength{\step}{\dimexpr 10cm / 18 \relax}
\draw[step=\step] (0, 0) grid (10, 10);

\draw[draw = white, fill = black, line width = 0.1mm]
  (0 * 10cm / 18, 0 * 10cm / 18)
  circle [radius = 0.2575cm]
  node[color = white] {1};
\draw[draw = black, fill = white, line width = 0.1mm]
  (1 * 10cm / 18, 0 * 10cm / 18)
  circle [radius = 0.2575cm]
  node[color = black] {2};
\draw[draw = white, fill = black, line width = 0.1mm]
  (2 * 10cm / 18, 0 * 10cm / 18)
  circle [radius = 0.2575cm]
  node[color = white] {3};

\end{tikzpicture}
\end{document}

I don't really know the best way to do this. Is there a good package of this kind of thing? I suppose Regexes would do actually, is there a better way? I'm gonna update this question as I try more stuff.

In this answer, somebody helped me with a PCRE-complete way of parsing SGFs (in PHP), but I don't know to what extent that can be done in TeX. At any rate, maybe the key-value part of that recipe is useful:

/(?<key>[A-Z]+)\[(?<value>(?:\\\]|[^\]])*)\]/gy

---

Parsing SGFs is something I would like to include in a package I'm trying to build here. Hopefully, you can find some useful macros about building Go diagrams there.

SGF is the representation of a tree structure in text, but, for simplicity, at first, I think I would like to only parse the main branch, in files with only one branch.

(There are some SGF parsers in JS available out there — including mine and Sabaki's — maybe they're useful references, or maybe they could be invoked by TeX somehow?)

---

In the game of Go, since the board is much bigger than in Chess, we typically don't use coordinates but visual editors (or paper kifus) — two of the best editors are CGoban and Sabaki — in order to save games. In the end, the Smart Game Format (SGF) is the standard (it also supports other games actually), and it ends up saving the file in text format anyways.

Here's an example of an SGF file:

(;GM[1]FF[4]CA[UTF-8]AP[CGoban:3]ST[2]
RU[Japanese]SZ[19]KM[6.50]
DT[2023-12-25]
;B[pd]
;W[dd]
;B[dp]
;W[pp]
;AW[ji]AB[jj]PL[B]
;B[jq]CR[pp]LB[dd:A][jd:C][pd:B]TR[jj]SQ[ji]MA[dp])

• () denote branches, and data is within []
• The label before [] denotes the type of the data
• ; are node delimiters
• B and W are Black and White moves, respectively
• AB and AW are added or edited stones
• LB is a label on top of a coordinate, :A is the A label
• CR is a circle label
• TR is an triangle label
• SQ is a square label
• MA is a cross (X) label

The whole grammar can be defined as:

Collection     = { GameTree }
GameTree       = "(" RootNode NodeSequence { Tail } ")"
Tail           = "(" NodeSequence { Tail } ")"
NodeSequence   = { Node }
RootNode       = Node
Node           = ";" { Property }
Property       = PropIdent PropValue { PropValue }
PropIdent      = UcLetter { UcLetter }
PropValue      = "[" Value "]"
UcLetter       = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" |
                 "J" | "K" | "L" | "M" | "N" | "O" | "P" | "Q" | "R" |
                 "S" | "T" | "U" | "V" | "W" | "X" | "Y" | "Z"

---

Edit after @StevenB.Segletes' Answers

Following @StevenB.Segletes' listofitems package, and very helpful answers, especially this one, I've been able to create the following MWE:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{shapes.geometric}
\usepackage{listofitems}
%-----------------------------------------------------------
% Drawing Stones
% From this answer by @DavidCarlisle.
\newcommand\notwhite{black}
\newcommand\notblack{white}
% From this answer by @DavidCarlisle.
\ExplSyntaxOn
  \cs_generate_variant:Nn \int_from_alph:n {e}
\NewExpandableDocumentCommand{\stringToCoordX}{ m }{
    \int_from_alph:e { \use_i:nn #1 }
  }
  \NewExpandableDocumentCommand{\stringToCoordY}{ m }{
    \boardSize + 1 - ~\int_from_alph:e { \use_ii:nn #1 }
  }
\ExplSyntaxOff
\newcommand{\setCoords}[1]{
  \pgfmathsetmacro{\x}{\stringToCoordX{#1} - 1}
  \pgfmathsetmacro{\y}{\stringToCoordY{#1} - 1}
}
\newcommand{\drawStoneFromSgfCoords}[2]{%
  \setCoords{#2}
\draw[draw = \UseName{not#1}, fill = #1, line width = 0.1mm]
    (\x * \step, \y * \step)
    circle [radius = 0.2575cm];
}
\newcommand{\drawMoveFromSgfCoords}[2]{
  \drawStoneFromSgfCoords{#1}{#2}
  \textLabel{#1}{#2}{\themoveCounter}
\stepMoveCounter
}
\newcounter{moveCounter}
\setcounter{moveCounter}{1}
\newcommand{\stepMoveCounter}{
  \stepcounter{moveCounter}
}
%-----------------------------------------------------------
% Labels
\newcommand{\textLabel}[3]{
  \setCoords{#2}
\draw (\x * \step, \y * \step) 
    node[color = \UseName{not#1}] {#3};
}
\newcommand{\crossLabel}[2]{
  \setCoords{#2}
\draw (\x * \step, \y * \step) 
    node[color = \UseName{not#1}] {X};
}
\newcommand{\triangleLabel}[2]{
  \setCoords{#2}
\draw (\x * \step, \y * \step) 
    node[
      isosceles triangle,
      draw                          = #1,
      line width                    = 0.5mm,
      fill                          = \UseName{not#1},
      minimum height                = \step * 10,
      minimum width                 = \step * 10,
      rotate                        = 90,
      isosceles triangle apex angle = 60,
      inner sep                     = 0pt,
    ] {};
}
\newcommand{\squareLabel}[2]{
  \setCoords{#2}
\draw (\x * \step, \y * \step) 
    node[
      draw         = #1,
      line width   = 0.5mm,
      fill         = \UseName{not#1},
      minimum size = \step * 10,
      inner sep    = 0pt,
    ] {};
}
\newcommand{\circleLabel}[2]{
  \setCoords{#2}
\draw[
    draw         = #1,
    line width   = 0.5mm,
    fill         = \UseName{not#1},
    inner sep    = 0pt,
  ] (\x * \step, \y * \step) 
    circle[radius = \step / 4];
}
%-----------------------------------------------------------
% SGF Parser
% From this answer by @StevenB.Segletes.
\long\def\Firstof#1#2\endFirstof{#1}
\newcommand\thecolorofB{black}
\newcommand\thecolorofAB{black}
\newcommand\thecolorofW{white}
\newcommand\thecolorofAW{white}
\newcommand\thecolorofMA{white}
\newcommand\thecolorofCR{white}
\newcommand\thecolorofTR{white}
\newcommand\thecolorofSQ{white}
\newcommand\thecolorofLB{white}
\long\def\Keytypeof#1{\csname thekeytypeof#1\endcsname}
\newcommand\thekeytypeofB{M}  % black move
\newcommand\thekeytypeofAB{A} % added (edited) black stone
\newcommand\thekeytypeofW{M}  % white move
\newcommand\thekeytypeofAW{A} % added (edited) white stone
\newcommand\thekeytypeofMA{K} % cross (mark) label
\newcommand\thekeytypeofCR{C} % circle label
\newcommand\thekeytypeofTR{T} % triangle label
\newcommand\thekeytypeofSQ{S} % square label
\newcommand\thekeytypeofLB{L} % text label
\ignoreemptyitems
\newcommand{\parseSgf}[1]{%
  \setsepchar{;||(||)/]/[}%
  \readlist*\Z{#1}%
\foreachitem \i \in \Z[]{%
    \foreachitem \z \in \Z[\icnt]{%
      \itemtomacro\Z[\icnt, \zcnt, 1]\KeyName
      \itemtomacro\Z[\icnt, \zcnt, 2]\KeyValue
  \edef\tmp{{\csname thecolorof\KeyName\endcsname}{\KeyValue}}

  \if\Keytypeof\KeyName M
    \expandafter\drawMoveFromSgfCoords\tmp
  \fi
  \if\Keytypeof\KeyName A
    \expandafter\drawStoneFromSgfCoords\tmp
  \fi
  \if\Keytypeof\KeyName K
    \expandafter\crossLabel\tmp
  \fi
  \if\Keytypeof\KeyName C
    \expandafter\circleLabel\tmp
  \fi
  \if\Keytypeof\KeyName T
    \expandafter\triangleLabel\tmp
  \fi
  \if\Keytypeof\KeyName S
    \expandafter\squareLabel\tmp
  \fi
  \if\Keytypeof\KeyName L
    \expandafter\textLabel\tmp % this macro has 1 extra argument compared to the other ones (the text itself)
  \fi
}

}%
}
%-----------------------------------------------------------
% SGFs
\def\sgfA{;B[ab];W[cd]}
\def\sgfB{(
  ;GM[1]FF[4]CA[UTF-8]AP[Sabaki:0.52.2]KM[6.5]SZ[19]DT[2024-02-05]
  ;B[as]
  ;W[bs]
  ;B[cs]
)}
\def\sgfC{(
  ;GM[1]FF[4]CA[UTF-8]AP[Sabaki:0.52.2]KM[6.5]SZ[19]DT[2024-02-05]
  ;B[as]
  ;W[bs]
  ;B[cs]
  ;PL[W]AB[dq]AW[eq]
)}
\def\sgfD{( % Basically sgfE with no labels (LB)
  ;GM[1]FF[4]CA[UTF-8]AP[CGoban:3]ST[2]RU[Japanese]SZ[19]KM[6.50]DT[2023-12-25]
  ;B[pd]
  ;W[dd]
  ;B[dp]
  ;W[pp]
  ;AW[ji]AB[jj]PL[B]
  ;B[jq]CR[pp]TR[jj]SQ[ji]MA[dp]
)}
\def\sgfE{(
  ;GM[1]FF[4]CA[UTF-8]AP[CGoban:3]ST[2]RU[Japanese]SZ[19]KM[6.50]DT[2023-12-25]
  ;B[pd]
  ;W[dd]
  ;B[dp]
  ;W[pp]
  ;AW[ji]AB[jj]PL[B]
  ;B[jq]CR[pp]LB[dd:A][jd:C][pd:B]TR[jj]SQ[ji]MA[dp]
  % The LB data is apparently the only weird one.
  % Besides having the LB[<coords>:<label>] format,
  % If many of them are on the same node, there will only be one LB for all of them.
)}
%-----------------------------------------------------------
% Setup
\pgfmathsetmacro{\boardDimension}{10}
\pgfmathsetmacro{\boardSize}{19}
\pgfmathsetmacro{\step}{\boardDimension / (\boardSize - 1)}
%-----------------------------------------------------------
\begin{document}
  \begin{tikzpicture}
    \draw[step=\step] (0, 0) grid (10, 10);
\parseSgf{\sgfD}

\textLabel{white}{ab}{A}
\triangleLabel{black}{ac}
\squareLabel{black}{ad}
\circleLabel{black}{ae}
\crossLabel{white}{af}

\end{tikzpicture}
\end{document}

Essentially here's what's missing:

• The label key (LB) is the only one with a weird format, out of the essential keys. In the previous answer, @StevenB.Segletes did manage to cover the format LB[<coords>:<label>], however, when multiple labels are on the same node, it gets even weirder, because they get grouped together, without a key, e.g. LB[dd:A][jd:C][pd:B] (this actually happens to every label key apparently, see the example below).
• Since the labels are usually marked after a stone has been placed, I don't know how I would be able to find the appropriate color for them when parsing. When parsing, since the parser doesn't know if there's a black or a white stone underneath the label, it won't know which color to paint with. Maybe there should be a list of coords vs colors, and then the parser should check before painting the label, I don't know.

Actually, the grouping shown by LB happens to every label and edited or added stones:

(
  ;GM[1]FF[4]CA[UTF-8]AP[CGoban:3]ST[2]RU[Japanese]SZ[19]KM[6.50]DT[2023-12-25]
  ;B[pd]
  ;W[dd]
  ;B[dp]
  ;W[pp]
  ;AW[ji][jn][kn][ln]AB[jj][jm][km]TR[jp][kp][lp]PL[B]
  ;B[jq]CR[pp]LB[dd:A][jd:C][pd:B]TR[jj]SQ[ji]MA[dp]
)

Answer 1

I think this takes care of all of the issues required by the OP.

The challenge with this answer, compared with the OP's prior questions, is that a new valid syntax was introduced in the OP's description of the SGF files. Previously, it was implied that keys always took the form of <Keyname>[<Value>] or <Keyname>[<Value>:<Label>]. Newly introduced syntax, in fact, allows for <Keyname>[<Value1>][<Value2>][<Value3>]..., which had the effect of invalidating my prior parsing logic. This allowed syntax variation may appear for any valid key name.

The solution was to no longer ignore empty items in the list parsing, but to allow them, so that <Key1>[<Value1>][<Value2>] could effectively be parsed and understood as the equivalent of <Key1>[<Value1>]<implied-Key1>[<Value2>]. Of course, allowing for blank list items messed up other facets of the parsing that had to be resolved.

In the end, I required 5-level parsing:

1. Anything delimited by ( or ) I call a Branch

2. Within each branch, anything delimited by ; I call a Group

3. Within each group, anything delimited by ] I call a Key (though it will include the key name and its associated value)

4. Within each key, the [ delimiter separates the key-name from the key-value

5. Within each key-value, a : can be used to delimit the actual value from an associated label.

With this approach, I could effectively parse the OP's test SGF:

(
  ;GM[1]FF[4]CA[UTF-8]AP[CGoban:3]ST[2]RU[Japanese]SZ[19]KM[6.50]DT[2023-12-25]
  ;B[pd]
  ;W[dd]
  ;B[dp]
  ;W[pp]
  ;AW[ji][jn][kn][ln]AB[jj][jm][km]TR[jp][kp][lp]PL[B]
  ;B[jq]CR[pp]LB[dd:A][jd:C][pd:B]TR[jj]SQ[ji]MA[dp]
)

Here is the MWE:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{shapes.geometric}
\usepackage{listofitems}
%-----------------------------------------------------------
% Drawing Stones
% From this answer by @DavidCarlisle.
\newcommand\notwhite{black}
\newcommand\notblack{white}
% From this answer by @DavidCarlisle.
\ExplSyntaxOn
  \cs_generate_variant:Nn \int_from_alph:n {e}
\NewExpandableDocumentCommand{\stringToCoordX}{ m }{
    \int_from_alph:e { \use_i:nn #1 }
  }
  \NewExpandableDocumentCommand{\stringToCoordY}{ m }{
    \boardSize + 1 - ~\int_from_alph:e { \use_ii:nn #1 }
  }
\ExplSyntaxOff
\newcommand{\setCoords}[1]{
  \pgfmathsetmacro{\x}{\stringToCoordX{#1} - 1}
  \pgfmathsetmacro{\y}{\stringToCoordY{#1} - 1}
}
\newcommand{\drawStoneFromSgfCoords}[2]{%
  \setCoords{#2}
\draw[draw = \UseName{not#1}, fill = #1, line width = 0.1mm]
    (\x * \step, \y * \step)
    circle [radius = 0.2575cm];
}
\newcommand{\drawMoveFromSgfCoords}[2]{
  \expandafter\xdef\csname thecolorat#2\endcsname{#1}% SAVE COLOR AT #2
  \drawStoneFromSgfCoords{#1}{#2}
  \textLabel{#1}{#2}{\themoveCounter}
\stepMoveCounter
}
\newcounter{moveCounter}
\setcounter{moveCounter}{1}
\newcommand{\stepMoveCounter}{
  \stepcounter{moveCounter}
}
%-----------------------------------------------------------
% Labels
\newcommand\StoneColor[1]{% COMPLEMENTARY COLOR OF REFERENCED STONE
  \ifcsname thecolorat#1\endcsname
    \csname thecolorat#1\endcsname
  \else
    white% COMPLEMENT OF DEFAULT LABEL COLOR WHEN NOT ATOP EXISTING STONE
  \fi
}
\newcommand{\textLabel}[3]{%
  \setCoords{#2}%
  \draw (\x * \step, \y * \step) 
  node[
    color = -\StoneColor{#2},
    fill = \StoneColor{#2},
    inner sep=1pt
  ] {#3};
}
\newcommand{\crossLabel}[2]{
  \setCoords{#2}
  \draw (\x * \step, \y * \step) 
  node[
    color = -\StoneColor{#2},
    fill = \StoneColor{#2},
    inner sep=1pt
  ] {$\times$};
}
\newcommand{\triangleLabel}[2]{
  \setCoords{#2}
  \draw (\x * \step, \y * \step) 
    node[
      isosceles triangle,
      draw                          = #1,
      line width                    = 0.5mm,
      color                         = -\StoneColor{#2},
      fill                          = -\StoneColor{#2},
      minimum height                = \step * 10,
      minimum width                 = \step * 10,
      rotate                        = 90,
      isosceles triangle apex angle = 60,
      inner sep                     = 0pt,
    ] {};
}
\newcommand{\squareLabel}[2]{
  \setCoords{#2}
\draw (\x * \step, \y * \step) 
    node[
      draw         = #1,
      line width   = 0.5mm,
      color        = -\StoneColor{#2},
      fill         = -\StoneColor{#2},
      minimum size = \step * 10,
      inner sep    = 0pt,
    ] {};
}
\newcommand{\circleLabel}[2]{
  \setCoords{#2}
\draw[
    draw         = #1,
    line width   = 0.5mm,
    color        = -\StoneColor{#2},
    fill         = -\StoneColor{#2},
    inner sep    = 0pt,
  ] (\x * \step, \y * \step) 
    circle[radius = \step / 4];
}
%-----------------------------------------------------------
% SGF Parser
% From this answer by @StevenB.Segletes.
\long\def\Firstof#1#2\endFirstof{#1}
\newcommand\thecolorofB{black}
\newcommand\thecolorofAB{black}
\newcommand\thecolorofW{white}
\newcommand\thecolorofAW{white}
\newcommand\thecolorofMA{white}
\newcommand\thecolorofCR{white}
\newcommand\thecolorofTR{white}
\newcommand\thecolorofSQ{white}
\newcommand\thecolorofLB{white}
\long\def\Keytypeof#1{\csname thekeytypeof#1\endcsname}
\newcommand\thekeytypeofB{M}  % black move
\newcommand\thekeytypeofAB{A} % added (edited) black stone
\newcommand\thekeytypeofW{M}  % white move
\newcommand\thekeytypeofAW{A} % added (edited) white stone
\newcommand\thekeytypeofMA{K} % cross (mark) label
\newcommand\thekeytypeofCR{C} % circle label
\newcommand\thekeytypeofTR{T} % triangle label
\newcommand\thekeytypeofSQ{S} % square label
\newcommand\thekeytypeofLB{L} % text label
\newcommand{\parseSgf}[1]{%
  \setsepchar{(||)/;/]/[/:}%
  \readlist*\Z{#1}%
\foreachitem \Branch \in \Z[]{%
  \foreachitem \Group \in \Z[\Branchcnt]{%
    \foreachitem \Key \in \Z[\Branchcnt, \Groupcnt]{%
     \if\relax\Key\relax% IF BLANK KEY & VALUE, SKIP
     \else
      \itemtomacro\Z[\Branchcnt, \Groupcnt, \Keycnt, 1]\KeyName
      \if\relax\KeyName\relax% IF VALUE, BUT NO KEYNAME, USE RECENT KEYNAME
        \let\KeyName\MostRecentKeyname
      \else
        \xdef\MostRecentKeyname{\KeyName}%
      \fi
      \itemtomacro\Z[\Branchcnt, \Groupcnt, \Keycnt, 2, 1]\KeyValue
  \edef\tmp{{\csname thecolorof\KeyName\endcsname}{\KeyValue}}%

  \if\Keytypeof\KeyName M
    \expandafter\drawMoveFromSgfCoords\tmp
  \fi
  \if\Keytypeof\KeyName A
    \expandafter\drawStoneFromSgfCoords\tmp
  \fi
  \if\Keytypeof\KeyName K
    \expandafter\crossLabel\tmp
  \fi
  \if\Keytypeof\KeyName C
    \expandafter\circleLabel\tmp
  \fi
  \if\Keytypeof\KeyName T
    \expandafter\triangleLabel\tmp
  \fi
  \if\Keytypeof\KeyName S
    \expandafter\squareLabel\tmp
  \fi
  \if\Keytypeof\KeyName L
    \expandafter\textLabel\tmp{\Z[\Branchcnt, \Groupcnt, \Keycnt, 2, 2]} 
  \fi
 \fi
}

}%
  }%
}
%-----------------------------------------------------------
% SGFs
\def\sgfE{(
  ;GM[1]FF[4]CA[UTF-8]AP[CGoban:3]ST[2]RU[Japanese]SZ[19]KM[6.50]DT[2023-12-25]
  ;B[pd]
  ;W[dd]
  ;B[dp]
  ;W[pp]
  ;AW[ji][jn][kn][ln]AB[jj][jm][km]TR[jp][kp][lp]PL[B]
  ;B[jq]CR[pp]LB[dd:A][jd:C][pd:B]TR[jj]SQ[ji]MA[dp]
)}
%-----------------------------------------------------------
% Setup
\pgfmathsetmacro{\boardDimension}{10}
\pgfmathsetmacro{\boardSize}{19}
\pgfmathsetmacro{\step}{\boardDimension / (\boardSize - 1)}
%-----------------------------------------------------------
\begin{document}
  \begin{tikzpicture}
    \draw[step=\step] (0, 0) grid (10, 10);
\parseSgf{\sgfE}

\textLabel{white}{ab}{A}
\triangleLabel{black}{ac}
\squareLabel{black}{ad}
\circleLabel{black}{ae}
\crossLabel{white}{af}

\end{tikzpicture}
\end{document}

Several issues required surmounting:

1. To place a \textLabel over an existing stone, I had to remember the color of the underlying stone, so that I could have the label in the complementary color. I saved the stone's color (at creation time) in \thecolorat<coordinate>. Then, when a text label was called for, I used \StoneColor{<coordinate>} to calculate the complementary color...if no stone had been created at that location previously, I use a default color by assuming the underlying color as white(so that the complementary label is black).

2. When overwriting an existing (default) label, I had to add fill of the stone color, with an inner sep, to obliterate the prior label.

3. With geometric labels (square, circle, triangle), both color and fill needed to be specified in the complementary color, in order to achieve uniformity across the board.

4. As mentioned earlier, 5-level parsing was required and obtained via \setsepchar{(||)/;/]/[/:}.

5. If an item representing a "key" was totally blank (no keyname or value), such as the case when (;... is parsed, the parsing will skip to the next item in the parsed "key" list.

6. While iterating through all the key actions, if the keyname is present, a global copy of it is saved via \xdef\MostRecentKeyname{\KeyName}.

7. If a keyname is blank but an associated key-value exists, the most recent non-blank keyname is used, via \let\KeyName\MostRecentKeyname. This covers cases with implied keynames, such as AB[jj][jm][km], in which a black stone is added at each of the three specified coordinates.

8. For keys of the text-label type, such as LB[dd:A], the 5th level of parsing was needed to separate the coordinate dd from the label A.

9. For the cross label, I used $\times$ instead of X.