The laws of expansion (\chardef\foo=\foo)

Question

Fact The file-name of the default font for a pdfTeX-compiled LaTeX document on my system is cmr10. (How do I know? \fontname\font. See here.)
Fact The font defined in the file cmr10 has a glyph of the Greek capital letter lambda at code point 3, and a glyph of the mathematical symbol for the empty set at code point 31. (How do I know? \fonttable{cmr10}. See here.)

Consider the following (La)TeX manuscript, inspired by this answer.

\documentclass{article}
\begin{document}
\def\foo{1}
\chardef\foo=3\foo
\end{document}

Compiling the manuscript with pdfTeX Version 3.14159265-2.6-1.40.18 (TeX Live 2017) results in a pdf file, whose body consists of a single letter: a Greek capital letter lambda.

This goes contrary to my expectations that the body of the pdf would consist of the mathematical symbol for the empty set.

What did I base my expectation on? On Donald Knuth's The TeXbook, 20th revised printing (Addison-Wesley 1991), where, on p. 215, he lists all the cases when expandable tokens are not expanded.

If you peruse the list you will notice that in the case of \chardef, macro expansion is inhibited only in the stage of reading the control sequence to be defined. (Compare this to, say, \def, where expansion is additionally inhibited while absorbing the parameter text and the replacement text.)

Returning to the manuscript I started with, it can be deduced from Knuth's list of non-expandables that the second occurrence of control sequence \foo in the line

\chardef\foo=3\foo

is expanded. Note that at the time this expansion takes place the \chardef definition is still processing, so the control sequence \foo must still be defined as a macro with the replacement text 1. Hence this line redefines \foo to be \char31.

Interestingly, the following, similar manuscript produces the expected result (i.e. the empty-set symbol).

\documentclass{article}
\begin{document}
\def\foo{1}
\chardef\bar=3\foo\bar
\end{document}

Joseph Wright · Accepted Answer · 2017-06-23T08:16:31.123

The key is that \chardef\<name> turns \<name> temporarily into \relax. We can see this with \tracingall:

\def\foo{1}
\tracingall
\chardef\bar=3\foo\bar

yields

{\chardef}
{changing \bar=macro:->\mathaccent "7016\relax }
{into \bar=\relax}

\foo ->1
{changing \bar=\relax}
{into \bar=\char"1F}
{\char"1F}

whereas with

\def\foo{1}
\tracingall
\chardef\foo=3\foo

we get

{\chardef}
{changing \foo=macro:->1}
{into \foo=\relax}
{changing \foo=\relax}
{into \foo=\char"3}
{\char"3}

So we get the outcomes you observe, not because of \chardef failing to expand material but because of the nature of the material at the point it is examined.

Knuth says:

\TeX\ takes precautions so that constructions like `|\chardef\cs=10\cs|' and
`|\font\cs=name\cs|' won't expand the second |\cs| until
the assignments are done.

(Chapter 24, 'Summary of Vertical Mode, page 278).

I'd strongly recommend terminating the scanner here with an explicit space or \relax, not relying on what can be tricky-to-follow (though documented) behaviour of other tokens. — Joseph Wright, Jun 23 '17 at 08:35

score 6 · Answer 2 · answered Jun 23 '17 at 08:33

Quoting the TeXbook (p. 278)

TeX takes precautions so that constructions like ‘\chardef\cs=10\cs|’ and ‘\font\cs=name\cs’ won’t expand the second \cs until the assignments are done.

Those precautions are covered in module 1224 of tex.web:

A ‘shorthand def’ is one of the definition-like primitives described in the module.

Knuth warns that the book doesn't always tell the complete truth.

The laws of expansion (\chardef\foo=\foo)

2 Answers2

Linked