15

As far as I have understand the tokenization in LaTeX, x_\text y will be split up into

  1. x
  2. _
  3. \text
  4. y

Now the _ will consume/bind to the next token. However, \text takes an argument as well. If _ consumes \text, the latter does not have an argument.

When I do something like \frac 1 \sqrt 2, it does not work in contrast to the first one, and this is the behavior that I would except.

Why does the first example work at all?

egreg
  • 1,121,712
Martin Ueding
  • 2,873
  • 4
  • 23
  • 38
  • @egreg Yes, you're right of course! I deleted my comment. It is expanded before it is sent to "TeX's stomach" according to TeXbook. Can you shed some light on this? – Henri Menke May 04 '14 at 19:42
  • @HenriMenke As you see from my answer, it's a bit more complex. ;-) – egreg May 04 '14 at 19:47

2 Answers2

20

You have to know that _ (in math mode, of course, and with normal setting of category codes) will expand the next token in order to see if a brace (explicit or implicit) follows. This is common for primitives that want a <general text> as their argument (so also \uppercase, for instance).

Thus _ will try expanding the following token, because it's not { or \bgroup. The first level expansion of \text is

\protect\text•

(where • denotes a space in the name of the control sequence). Since we are in a normal situation (typesetting), \protect is equivalent to \relax. Here another feature comes in: when a <general text> is being looked for, spaces and \relax tokens are ignored (they're called a <filler> in the TeXbook), so this \protect disappears. Now \text• is not a brace, so TeX continues expanding it; the first level expansion is

\ifmmode\expandafter\text@\else\expandafter\mbox\fi

OK, we're in math mode, so we're left with

\expandafter\text@\else\expandafter\mbox\fi

Hmm, \expandafter is not a brace, expand it! This causes the expansion of \else, therefore just \text@ remains. Here we are, almost! The definition of \text@ in amstext.sty is

\def\text@#1{{\mathchoice
  {\textdef@\displaystyle\f@size{#1}}%
  {\textdef@\textstyle\f@size{\firstchoice@false #1}}%
  {\textdef@\textstyle\sf@size{\firstchoice@false #1}}%
  {\textdef@\textstyle \ssf@size{\firstchoice@false #1}}%
  \check@mathfonts
  }%
}

so, since TeX wants to expand it, the argument is looked for; y is a token which is not a brace that delimits an argument, so it is the argument.

The expansion is then

{\mathchoice...}

with y in place of #1. Hurray! The open brace has been found! Go on and typeset the subscript.

Final word: always use braces around the subscript and arguments; your input will be clearer. So

$x_{\text{y}}$

and nothing else.

egreg
  • 1,121,712
  • 1
    Assuming of course that _ is not an active token actually absorbing an argument! – Joseph Wright May 04 '14 at 19:47
  • @JosephWright I've added that I assume _ has the usual category code. – egreg May 04 '14 at 19:48
  • 1
    Thanks for blowing my mind. I only had a hunch that TeX would have a “Alice in Wonderland” experience in store for me so far … – Martin Ueding May 04 '14 at 19:51
  • 1
    @queueoverflow Did you like the <filler> part? – egreg May 04 '14 at 19:52
  • So far, everything that has an @ lets me think about it being internal and that I do not need to think about it, if I just want to use it. Kind of like assembly language. I just had no idea that there are so many layers of expansion to it. – Martin Ueding May 04 '14 at 19:59
  • @queueoverflow take a small document and add \tracingall and see the expansions logged in the log file. (don't use a beamer document unless you have a big disk:-) – David Carlisle May 04 '14 at 20:57
5

\text is defined in amstext.sty by means of \text@ where

\text@ #1->{\mathchoice{...}\check@mathfonts}

which adds braces around its expansion text. Thus the braces from the expansion of \text@ will enclose all stuff in the index.

Compare for instance:

\def\bad#1{\rm #1}
\def\good#1{{\rm #1}}
% $x_\bad y$% <- Missing { inserted
$x_\good y$
\bye
Henri Menke
  • 109,596