I'm confused about how expansion is happening here

Question

I was really pleased that I could come up with a means of building the contents of a tabular environment using \foreach \x in {...} {...} structure.

But I was curious about what exactly it would take to break my code. Here's my preamble, which works for generally non-complex content (whatever that's supposed to mean):

\documentclass{article}
\usepackage[margin=0.5in]{geometry}
\usepackage{amsmath,amssymb,array}
\usepackage{booktabs}
\usepackage{etoolbox}
\usepackage{pgfkeys,pgffor}

\pgfkeys{/ae/agenda/.cd,
  items for the agenda/.initial=,
  title/.initial=,
} 
\def\aeset#1{\pgfkeys{/ae/agenda/.cd,#1}}
\def\aeget#1{\pgfkeysvalueof{/ae/agenda/#1}}

\makeatletter

\newcounter{agendaitemcounter}
\def\agenda@item@number{%%
  \stepcounter{agendaitemcounter}%%
  \arabic{agendaitemcounter}.)}

\def\full@agenda{}
\long\def\build@agenda{%%
  \foreach \x in \list@of@agenda@items {
    \xdef\full@agenda{%%
      \expandonce\full@agenda
      \expandonce\agenda@item@number
      \expandonce\aeamp
      \expandonce\x
      \noexpand\\
      \noexpand\midrule}
  }}

\def\setagenda#1{%%
  \bgroup
  \aeset{#1}%%
  \edef\list@of@agenda@items{\aeget{items for the agenda}}%%
  \build@agenda
  \begin{minipage}[t]{4in}\centering
    {\large \aeget{title}}%%
    \par\vspace{1ex}
    \begin{tabular}{cp{3in}}\toprule
      \full@agenda          \bottomrule
    \end{tabular}%%
  \end{minipage}%%
  \setcounter{agendaitemcounter}{0}%%
  \egroup
  \def\full@agenda{}%%
  }

\makeatother

\renewcommand{\arraystretch}{1.5}

\def\aeamp{&}
\def\aepar{\par}
\def\dollars#1{\$#1}%$
\def\mypoly{\[x^2+1\]}

\pagestyle{empty}

So, I started playing with things to see where the macros in the preamble would break: it wasn't too hard to find something.

The example code below compiles fine as written with the above preamble (except for the one commented-out line that contains \noexpand\par). However, if I start removing \noexpand commands from these working lines, errors start popping up.

What I can't quite understand is why certain structures don't break (e.g. \frac{...}{...}, x^2, and \infty) while other---superficially similar or simpler?---structures (e.g. \cos or \$) do break my code.

\begin{document}

\setagenda{
  title={To-do list},
  items for the agenda={%%
    {Line of random content, but nothing too fancy.},
    {Not too involved math: $x\cdot y$ },
    {A bit more involved, but it works! $\displaystyle{}\frac{1}{2d}$ },
    { Random collection of math symbols: $\inf \infty x^2+x$ },
    %% I need to use `\noexpand` to avoid getting an error.
    {This line works!\noexpand\aepar Hello world }
    %%{This line fails!\noexpand\par Hello world }
    { what follows fails without \noexpand\texttt{noexpand}},
    { Trying something which covers multiple lines 
      \belowdisplayskip=0pt%%
      \belowdisplayshortskip=0pt%%
      \noexpand\mypoly
    },
    {Transfer \noexpand\dollars{3000} for trip to Venice},
    { $x\noexpand\cos(x)$ How many suitcases do I need?},
    { What's this going to result in? 
      \belowdisplayskip=0pt%%
      \belowdisplayshortskip=0pt%%
      \noexpand\[ x^2+1\noexpand\]
    }
  }%%
}

\vspace{1cm}

\setagenda{
  title={something else},
  items for the agenda={%%
    {first item},
    {second item}}}

\end{document}

Why is it that some math commands work and others don't. I thought that, because I'm issuing \expandonce in the macro used to build the contents of the tabular environment, these errors shouldn't arise.

To labor the point a bit more, when I try to look at this in miniature, without all the noise of the tabular environment, I get even more confused.

The following MWE compiles fine:

\documentclass{article}
\usepackage{etoolbox}
\usepackage{pgffor}
\begin{document}
\let\mytrial\relax
\foreach \x in {$\cos$,
                $x^2+x$}
  {\xdef\mytrial{\x}}
\mytrial
\end{document}

So, there's nothing inherently evil about \cos and \xdef being used together. But when I try to make this look a bit more like the code above in \setagenda, things go awry in ways I can't quite fathom.

\documentclass{article}
\usepackage{etoolbox}
\usepackage{pgffor}
\begin{document}
\let\mytrial\relax
\foreach \x in {$\cos$,
                $x^2+x$}
  {\xdef\mytrial{\mytrial\x}}
\mytrial
\end{document}

I don't even get an error. TeX just enters an infinite loop (it's as though there's a LaTeX3'ish quark in there or something).

Well, I have \let something to be \relax in there. So, I tried

\documentclass{article}
\usepackage{etoolbox}
\usepackage{pgffor}
\begin{document}
\def\mytrial{\relax}
%%\def\mytrial{}%%<--- this line works too!
\foreach \x in {$\cos$,
                $x^2+x$}
  {\xdef\mytrial{\mytrial\x}}
\mytrial
\end{document}

which does work, which totally flabbergasts me! How is this last example any different from what I'm doing inside \setagenda? There aren't even any \expandonce commands in there to save things for later.

My understanding of \noexpand\x is that

\xdef\mycmd{\noexpand\somecmd}

results in

\mycmd=\somecmd

My understanding of \expandonce\x where \x=\noexpand\cos is that

\xdef\mycmd{\expandonce\x}

results in

\mycmd=\noexpand\cos

So if I've got something like

\mycmd=\noexpand\cos
\x=\noexpand\tan

and then try

\xdef\mycmd{\expandonce\mycmd \expandonce\x}

I expect the result to be

\mycmd=\noexpand\cos \noexpand\tan

What's happening? What am I not understanding here? I guess I should conclude that there's something going on inside the tabular environment that I haven't tapped yet with my examples. But I'm not really sure what that would be.

Answer 1

There are so many things in this question that it's difficult to analyze each one.

Let's tackle the \cos problem. Without amsmath, its definition is

*\show\cos
\cos:
macro:->\mathop {\operator@font cos}\nolimits

(here and in what follows I report the output an interactive session; * is the prompt in interactive sessions). The expansion of \operator@font is \mathgroup\symoperators which are two unexpandable tokens. If we try, we get

*\xdef\x{\cos}\show\x
> \x=macro:
->\mathop {\mathgroup \symoperators cos}\nolimits .

However, when amsmath is loaded, the situation is different:

*\RequirePackage{amsmath}
*\show\cos
\cos=macro:
->\qopname \relax o{cos}.

and \qopname is defined via \DeclareRobustCommand so it most probably won't survive \edef (or \xdef which is simply \global\edef). Indeed we find

% amsopn.sty, line 49:
\DeclareRobustCommand{\qopname}[3]{%
  \mathop{#1\kern\z@\operator@font#3}%
  \csname n#2limits@\endcsname}

and

% amsopn.sty, line 38:
\def\nolimits@{\@ifnextchar\limits{\nolimits\@gobble}{\nolimits}}

which finally is the culprit: it contains \@ifnextchar. When doing expanded definitions one should use \protected@xdef to overcome the issue. For instance, we'd get

*\protected@xdef\x{\cos}\show\x
> \x=macro:
->\protect \qopname  \relax o{cos}.

that doesn't show the problem.

What about the \relax question? It's easily explained. When a control sequence is unexpandable, \edef doesn't do anything with it. And \relax is unexpandable. Your code

\let\mytrial\relax
\foreach \x in {a,b}
  {\xdef\mytrial{\mytrial\x}}

(I use simply a and b as the tokens you use just make the thing more complicated to follow, but the added complication is irrelevant) is equivalent to

\let\mytrial\relax
\xdef\mytrial{\mytrial a}
\xdef\mytrial{\mytrial b}

Note that when processing \edef\cs{<balanced text>}, first TeX computes the replacement text by doing expansions until unexpandable tokens remain; let's call the result <balanced text expanded>; only after doing this, TeX performs the equivalent of \def\cs{<balanced text expanded>}. In doing this last step, TeX doesn't interpret the tokens in the replacement text in any way.

After this first \xdef, the expansion of \mytrial is \mytrial a, and now \mytrial is not unexpandable any more. So the second \xdef will try to expand \mytrial; guess what happens? Here it is

\mytrial b -> \mytrial ab -> \mytrial aab -> ...

because \edef and \xdef go “all the way”, as already said: they do expansion until only unexpandable tokens remain. Of course, when you have \def\mytrial{}, in the first step the expansion will be simply a and no infinite loop occurs.