15

I'm still trying to understand how exactly tex parses documents. This time I'm trying to understand all the parameter-tokens-handling in \newcommand and friends, which looks like black magic to me - the crux here seems to happen in \@yargdef and \@yargd@f, which can be found in latex.ltx lines 868 and 877 respectively. The aim is obviously to create the parameter string that will ultimately be following a \def\foo, but I don't understand how this works.

To explain where I think my problem is, I'll have to explain my thinking a little, unfortunately. So: If I do \newcommand\foo[2]{\bar}, then as far as I can tell, this ultimately (after various checks for whether \foo is already defined and checking for optional arguments etc.pp.) expands to \@argdef\foo[2]{\bar}.

I'll go step by step how I read the definitions:

  • \@argdef is defined via: 
    \long\def\@argdef#1[#2]#3{%
\@ifdefinable #1{\@yargdef#1\@ne{#2}{#3}}}

    So we end up with @yargdef\foo@ne{2} (leaving {\bar}).

  • \@yargdefis defined thus: 
    \long \def \@yargdef #1#2#3{%
\ifx#2\tw@
\def\reserved@b##11{[####1]}%
\else
\let\reserved@b\@gobble
\fi
\expandafter
\@yargd@f \expandafter{\number #3}#1%
}

    which defines \reserved@b as @gobble and ultimately expands to @yargd@f{2}\foo (leaving {\bar}).

  • Now \@yargd@f is the weird part. It is defined via: 
    \long \def \@yargd@f#1#2{%
\def \reserved@a ##1#1##2##{%
\expandafter\def\expandafter#2\reserved@b ##1#1%
}%
\l@ngrel@x \reserved@a 0##1##2##3##4##5##6##7##8##9###1%
}

    So the expansion in my case should be (if I'm not mistaken?):

    \def\reserved@a #12#2#{%
\expandafter\def\expandafter\foo\reserved@b #12%
}%
\l@ngrel@x \reserved@a 0#1#2#3#4#5#6#7#8#9#2

    So in the application of \reserved@a, the first argument should be 0#1# and the second argument should be... empty, I guess, since the immediate following token is already #...? Leaving 3#4#5#6#7#8#9#2{\bar} as the next tokens?

This must be where my thinking goes wrong, because if I pretend that string before the {\bar} isn't there, the rest makes sense: \reserved@a then expands to \expandafter\def\expandafter\foo\reserved@b 0#1#2 - \reserved@b is defined as \@gobble which just eats the first token it finds, so the expansion of that should be \def\foo#1#2 - which would be followed by the {\bar}.

So apparently I'm interpreting the argument-string of \reserved@a wrong? What exactly is the second argument of \reserved@a in its application, if not empty? The only possible other explanation I can think of is that it eats to the next #-token, but even then, it would only eat the #3 part and leave a whole string of parameter tokens...?

am I expanding 0##1##2##3##4##5##6##7##8##9###1 or the ##1#1##2## in the definition of \reserved@a wrong?

Dennis Müller
  • 399

2 Answers2

12

Your reasoning is correct except for what the #{ parameter does.

When TeX sees a # in the <parameter text> of a macro (as in \def\macro<parameter text>{<replacement text>}), the following token can be either a digit in the range 1–9, or a {. When the parameter is "the dreaded weird #{ parameter", then TeX behaves as if the delimiter of the previous argument was a { (since you can't use a { in the <parameter text>).

In this definition (remember the {\bar} is still in the input stream):

\def\reserved@a #12#2#{%
\expandafter\def\expandafter\foo\reserved@b #12%
}%
\l@ngrel@x \reserved@a 0#1#2#3#4#5#6#7#8#9#2{\bar }

when \reserved@a expands, #1 will be, as you said, 0#1#, and #2 will be everything up to the next {, and not empty, so #3#4#5#6#7#8#9#2, delimited by the { in {\bar }. This definition of \reserved@a effectively throws away the remaining parameter tokens. So your "pretend[ing] that string before the {\bar} isn't there" was actually correct. The rest of the code executes as you assumed.

A quote from The TeXbook, about the #{ parameter:

A special extension is allowed to these rules: If the very last character of the <parameter text> is #, so that this # is immediately followed by {, TeX will behave as if the { had been inserted at the right end of both the parameter text and the replacement text. For example, if you say '\def\a#1#{\hbox to #1}', the subsequent text '\a3pt{x}' will expand to '\hbox to 3pt{x}', because the argument of \a is delimited by a left brace.

Phelype Oleinik
  • 70,814
9

I've already dealt with the problem in my book on LaTeX programming. My example is slightly different, but it's easier to copy. ;-) Suppose we have

\newcommand{\xyz}[2]{ab#1cd#2ef}

This becomes

\@star@or@long\new@command[2]{ab#1cd#2ef}

As there is no * after \new@command, this does \let\l@ngrel@x=\long and \@star@or@long disappears. Now \new@command is expanded, yielding

\@testopt{\@newcommand\xyz}0[2]{ab#1cd#2ef}

This becomes

\kernel@ifnextchar[{\@newcommand\xyz}{\@newcommand\xyz[{0}]}[2]{ab#1cd#2ef}

The purpose is to add [0] if the number of arguments is not given. Since there is a [, we get

\@newcommand\xyz[2]{ab#1cd#2ef}

Now comes the lookup for a further optional argument, which I skip, and we obtain

\@argdef\xyz[2]{ab#1cd#2ef}

The token \xyz is checked for definability; if it is not definable we get the error message, otherwise

\@yargdef\xyz\@ne{2}{ab#1cd#2ef}

and this is where the real fun starts:

\ifx\@ne\tw@
  \def\reserved@b#11{[##1]}
\else
  \let\reserved@b\@gobble
\fi
\expandafter\@yargd@f\expandafter{\number2}\xyz{ab#1cd#2ef}

This defines \reserved@b to be \@gobble and leads to

\@yargd@f{2}\xyz{ab#1cd#2ef}

Recall the definition of \@yarg@def:

\long\def\@yargd@f#1#2{%
  \def\reserved@a##1#1##2##{%
    \expandafter\def\expandafter#2\reserved@b##1#1}%
  \l@ngrel@x\reserved@a0##1##2##3##4##5##6##7##8##9###1%
}

The arguments are #1=2 and #2=\xyz, so we get (double ## are here reduced to a single #)

\def\reserved@a#12#2#{\expandafter\def\expandafter\xyz\reserved@b#12}
\l@ngrel@x\reserved@a0#1#2#3#4#5#6#7#8#9##1{ab#1cd#2ef}

The first argument to \reserved@a is delimited by 2, the second argument by {. This is a particular feature of TeX: the final argument can be delimited by the opening brace of the replacement text.

Thus the first argument is, in our case, 0#1#. Since \l@ngrel@x is \long it triggers the expansion of \reserved@a with the stated arguments:

\long\expandafter\def\expandafter\xyz\reserved@b0#1#2{ab#1cd#2ef}

Since \reserved@b is \@gobble, we end up with

\long\def\xyz{ab#1cd#2ef}
egreg
  • 1,121,712