3

Knuth's tex book introduced \if<token1><token2>. I wrote the following to test it

\def\tOne{abc}
\def\tTwo{abc}

\ifx\tOne\tTwo
True
\else
False
\fi


\if\tOne{abc}
True
\else
False
\fi


\if{abc}\tOne
True
\else
False
\fi

The result is

True

False

False

Why did the last two cases evaluate to False?

\tOne \tTwo are two tokens. {abc} is a grouping, right? Isn't it a token?

How does TeX evaluate \if\tOne{abc}?

Gqqnbig
  • 564
  • 1
    You say: "{abc} is a grouping, right? Isn't it a token?" Yes, it isn't a token, it is five tokens: {, a, b, c, }. TeX does not supports a primitive to comparing two strings or token sequences. Token sequences can be compared by \ifx when macros with them are previously defined and used \ifx followed by names of these macros. pdfTeX supports \pdfstrcmp{first}{second} which expands to -1, 0 1 and can be used in the context \ifnum\pdfstrcmp{ab}{cd}>0. LuaTeX does not supports this pimitive because it can be done by Lua. – wipet Apr 28 '22 at 05:34
  • Isn't it token? No, it isn't. See https://tex.stackexchange.com/q/104023/4427 – egreg Apr 28 '22 at 06:55

1 Answers1

6

\if expands the following tokens until it finds two non-expandable tokens.

  • \if\tOne{abc} becomes \if ab... and a and b are not equal. The remaining tokens c{abc} are skipped.

    Compare with:

    \def\tX{aax}
\if\tX true\else false\fi

    The second line expands in the next step to \if aaxtrue\else\false\fi and a and a are equal, thus xtrue is the result.

  • \if{abc}\tOne becomes \if{a... and { and a are not equal. The remaining tokens bc}\tOne are skipped.

\ifx does not expand, it just compares the meaning of the next two tokens. \tOne and \tTwo are macros with the same definition, therefore \ifx\tOne\tTwo evaluates to true.

Tokens are the result of the lexical analysis. For example, \if \tOne {abc} is a string with 15 characters. After tokenization there are 7 tokens: <\if>, <\tOne>, <{>, <a>, <b>, <c>, <}>. The space characters vanish in this process. When TeX reads the letters for the command sequence (e.g. '\', 't', 'O', 'n', 'e') the next non-letter marks the end of the command sequence. If the non-letter is a space token, it will be ignored.

The tokenization is controlled by the category codes for the characters. The same string \if \tOne {abc} is tokenized into 15 tokens <\>, <i>, <f>, < >, <{>, <a>, <b>, <c>, <}> in verbatim mode, where \\ , the space and the braces have the category code "other" (as digits or punctuation characters).

BTW, {abc} has a different meaning in the context of macro arguments. Here, the curly braces (more exactly, characters with category code 1 and 2) can be used to group tokens to an argument group:

\def\x#1{[#1]}
\x abc % -> [a]bc
\x{abc}def % -> [abc]def
\x{a{b}c}def % -> [a{b}c]def

The outer curly braces are removed, when read as argument as shown in the third and fourth line. Also, the curly braces must be properly matched.

Heiko Oberdiek
  • 271,626