Purpose of \expandafter in a particular macro

Question

Consider the following set of definitions...

\documentclass[12pt]{article}

\def\foreach{%
  \begingroup
  \catcode`\^^M=12 \xforeach}

{\catcode`\^^M=12
  \gdef\xforeach #1^^M{%
    \first #1,,\endgroup}}

\def\first #1,{%
  \if,#1,
  \else Do something with `#1'.\par\expandafter\first
  \fi}

\begin{document}
\foreach a,cde
\end{document}

I would be happy if you could explain to me the meaning (or function) of \expandafter in the macro \first defined above.

My guess is that the \expandafter closes the \if-test and, consequently, \first is inserted. I wouldn't be surprised if I'm wrong.

I will appreciate a detailed description of experienced TeX users.

Answer 1

Your guess is correct. \expandafter always skips over one token (in this case, \first) and expands the next one (in this case, \fi). The expansion of \fi (or \else, which are somewhat similar) ends the current conditional and removes the \fi token.

Suppose the \expandafter wasn't there: the first iteration of \first (which could use a better name, in my opinion) would be:

\if,a,
\else Do something with `a'.\par\first
\fi cde,,\endgroup

The \if test would be false, so The \else branch would be taken. Do something with `a'.\par would be typeset and then \first would expand once more. This time, \first would grab (everything to the next ,) \fi cde as argument, and the next iteration would be:

%   V----V frozen \relax
\if,\relax\fi cde,
\else Do something with `\fi cde'.\par\first
\fi cde,,\endgroup

Now the \if test would see the \fi token before the conditional was complete, so TeX would insert a frozen \relax and the test \if,\relax would yield false, so TeX would skip to the next \fi, which is the one right after the \relax. Now cde, would be (wrongly) typeset and then TeX, still in the \else branch of the first iteration of \first, would see another \else and would complain:

! Extra \else.
\first #1,->\if ,#1, \else
                           Do something with `#1'.\par \first \fi
l.19 \foreach a,cde

---

That said, I'd change your code a bit:

1. I'd issue the \endgroup before the loop starts, so that you wouldn't need to worry, for example, with the common issue of definitions inside a PGF \foreach;
2. I'd move all the Do something with `#1' outside the conditional, so that possible conditional tokens (\if, \else, \fi, etc.) in the argument won't interfere with the conditional of the loop; and
3. I'd use a safer emptiness test (see here for some examples); \if,#1, is dangerous in case of conditional tokens in the argument (as you saw above) and in case the item you're looping contains a comma as in, for example \foreach a,{,b},c. Better yet, I'd use a unique token to test the end of the loop, so that empty items are allowed (they are not in your current code).

That said, here's the changed code:

\documentclass{article}
\makeatletter
\def\foreach{%
  \begingroup
  \catcode`\^^M=12 \xforeach}
{\catcode`\^^M=12
  \gdef\xforeach #1^^M{%
  \endgroup%
    \first #1,,}}
\def\first #1,{%
  \if\relax\detokenize{#1}\relax
    \expandafter\@firstoftwo
  \else
    \expandafter\@secondoftwo
  \fi
    {\@gobble}%
    {Do something with `#1'.\par}%
  \first}
\makeatother
\begin{document}
\foreach a,cde
\end{document}

You could also extend it to put the do something code inline:

\documentclass{article}
\makeatletter
\def\foreach{%
  \begingroup
  \catcode`\^^M=12 \xforeach}
{\catcode`\^^M=12
  \long\gdef\xforeach #1^^M#2{%
  \endgroup%
  \def\marian@temp##1{#2}%
    \first #1,,}}
\def\first #1,{%
  \if\relax\detokenize{#1}\relax
    \expandafter\@firstoftwo
  \else
    \expandafter\@secondoftwo
  \fi
    {\@gobble}%
    {\marian@temp{#1}}%
  \first}
\makeatother
\begin{document}
\foreach a,cde
  {Do something with `#1'.\par}
\end{document}

Or with expl3's \clist_map_inline:nn:

\documentclass{article}
\usepackage{expl3}
\ExplSyntaxOn
\cs_new_protected:Npn \foreach
  {
    \group_begin:
      \char_set_catcode_other:N \^^M
      \__marian_foreach:wn
  }
\group_begin:
\char_set_catcode_other:N \^^M
\cs_new_protected:Npn \__marian_foreach:wn #1 ^^M #2
  {
    \group_end:
    \clist_map_inline:nn {#1} {#2}
  }
\group_end:
\ExplSyntaxOff
\begin{document}
\foreach a,cde
  {Do something with `#1'.\par}
\end{document}