28

In The LaTeX3 Interfaces we find:

\token_if_macro:NTF <token> {<true code>} {<false code>}

Tests if the <token> is a TeX macro.

\token_if_cs:NTF <token> {<true code>} {<false code>}

Tests if the <token> is a control sequence.

So I asked myself: what is the difference?

cgnieder
  • 66,645
  • 1
    Actually – I've done my homework :) – I believe I found the answer myself: a control sequence is every token starting with the escapechar, i.e., every token with category code 16. A macro is every control sequence (or active character with a definition) defined with \def and friends, i.e., every token where \meaning says “macro:”. If that is correct I'm happy to self-answer my question. – cgnieder Feb 14 '17 at 20:32
  • 2
    correct about macros, but there are no tokens with category code 16 (not really) basically every token that isn't a non-active character token is a control sequence token. – David Carlisle Feb 14 '17 at 20:34
  • @DavidCarlisle Yes, I know. I was referring to this quote from TeX by Topic: “TeX expands whatever is after \ifcat until two unexpandable tokens are found; these are then compared with respect to their category codes. Control sequence tokens are considered to have category code 16 […]” – cgnieder Feb 14 '17 at 20:37
  • BTW, is there a list of all the catcodes somewhere? I can't find one in Knuth. – John Kormylo Feb 14 '17 at 20:40
  • 1
    @JohnKormylo for example in section 2.3 of TeX by Topic or here: http://tex.stackexchange.com/questions/16410/what-are-category-codes – cgnieder Feb 14 '17 at 20:43
  • The \meaning of macros starts with one of the following phrases: macro:-> or \long macro:-> or \outer macro:-> or \long\outer macro:-> . A macro can be either a control sequence token (i.e. a control word token or a control symbol token) or an active character token. But I doubt that there is a test based on processing macro-arguments which is suitable for \outer macros as well. – Ulrich Diez Feb 15 '17 at 08:23
  • @JohnKormylo Have a look at the table in Chapter 7: How TeX reads what you type of The TeXbook by Donald E. Knuth. Also have a look at exercise 7.3 contained in that chapter. – Ulrich Diez Feb 15 '17 at 08:30
  • 1
    About the remark "control sequence is every token starting with the escapechar, i.e., every token with category code 16" : The category-code-16-thingie with \ifcat-comparison with control sequence tokens only applies to such control sequence tokens that are not \letequal to a non-active character token. E.g., with \let\bgroup={ the category code of the control sequence token \bgroup will usually be 1 as the catcode of {usually is 1 also. Preceding \noexpand doesn't do anything about that as it also wouldn't do anything about that when preceding the {-character-token. – Ulrich Diez Feb 15 '17 at 08:40
  • @UlrichDiez interesting. This means that in the sense of those tests \bgroup is neither a macro nor a control sequence… – cgnieder Feb 15 '17 at 08:57
  • Yes. Nonetheless \bgroup definitely is a control word token while control word tokens in turn are control sequence tokens. Besides this I am not quite sure whether \tokencheck is the correct name for a control sequence token that actually does not necessarily work on single tokens but actually does work on macro arguments while macro arguments might consist of several tokens or be empty as well... – Ulrich Diez Feb 15 '17 at 09:13
  • @UlrichDiez I know it isn't defined to actually only grap tokens. It was and is nothing more than a superficial test for me while I was trying to understand \token_if_macro:N(TF) and \token_if_cs:N(TF). Maybe I should delete it from the answer… – cgnieder Feb 15 '17 at 09:22
  • 1
    A non expandable test for control sequence tokens could be applying \string to the same token several times with the integer parameter \escapechar having different values. If the results of "stringifying" differ, then the token in question is a control sequence token. Otherwise it is not. – Ulrich Diez Feb 15 '17 at 09:30
  • Preliminarily to macro- and control-sequence checks a test might be needed for finding out whether the argument consists of exactly one token. (Hint: If the argument is not empty while not starting with a catcode 1 character token while in case of the argument not starting with a space gobbling one undelimited argument from the argument yields emptiness and in case of the argument starting with a space token gobbling that space token yields emptiness, then it consists of a single token.) – Ulrich Diez Feb 15 '17 at 09:31
  • No, don't delete it from the answer. Just point out that the requirement of the argument consisting of exactly one token must be fulfilled so people intending to "borrow" your code can avoid unpredictable behaviour with "edge cases" like the argument being empty or consisting of several tokens. – Ulrich Diez Feb 15 '17 at 14:32

1 Answers1

23

In short:

Control sequences are all tokens starting with the escapechar (usually \). There is one exception: Control sequence tokens which have been let to a non-active character token (like \bgroup) are not control sequences in the sense of \token_if_cs:N(TF).

Macros are all control sequences or active characters defined with \def and friends.

Examples (in the sense of the conditionals of the question):

  • \newcommand or \documentclass are both control sequences and macros,
  • \def, \relax, or \dimen@ are control sequences but not macros, and
  • ~ is a macro but not a control sequence
  • \bgroup is neither a macro nor a control sequence.

(assuming standard definitions). Undefined control sequences like \@undefined are no macros and active characters with no definition are neither macros nor control sequences.

A look at the definitions:

If we look at the definition of both conditionals we find:

  1. \token_if_cs:N(TF):

    \prg_new_conditional:Npnn \token_if_cs:N #1 { p , T ,  F , TF }
      {
        \if_catcode:w \exp_not:N #1 \scan_stop:
          \prg_return_true: \else: \prg_return_false: \fi:
      }
    

    This means every token which has the same category code as \scan_stop: (\relax) as \if_catcode:w (\ifcat) sees it. Citing TeX by Topic that is the “special category code 16”.

    TeX expands whatever is after \ifcat until two unexpandable tokens are found; these are then compared with respect to their category codes. Control sequence tokens are considered to have category code 16, which makes them all equal to each other, and unequal to all character tokens.

  2. \token_if_macro:N(TF):

    \use:x
      {
        \prg_new_conditional:Npnn \exp_not:N \token_if_macro:N ##1
          { p , T ,  F , TF }
          {
            \exp_not:N \exp_after:wN \exp_not:N \__token_if_macro_p:w
            \exp_not:N \token_to_meaning:N ##1 \tl_to_str:n { ma : }
              \exp_not:N \q_stop
          }
        \cs_new:Npn \exp_not:N  \__token_if_macro_p:w
          ##1 \tl_to_str:n { ma } ##2 \c_colon_str ##3 \exp_not:N \q_stop
      }
          {
            \if_int_compare:w \__str_if_eq_x:nn { #2 } { cro } = \c_zero
                \prg_return_true:
            \else:
                \prg_return_false:
            \fi:
          }
    

    This checks if \meaning<token> contains the string macro: (all category code 12 characters) which is true if the token has been defined with \def and friends.


A small test file:

Remark: the test macro \tokentest below not actually grabs the first token but is defined with a standard non-delimited argument.

\documentclass[twocolumn]{article}
\usepackage[T1]{fontenc}
\usepackage{xparse}

\newcommand*\cs[1]{\expandafter\texttt\expandafter{\string#1}}

\ExplSyntaxOn \cs_new_protected:Npn \token_check:N #1 { \noindent \cs #1 ~ is~ \token_if_macro:NF #1 { \emph {not} ~ } a~ macro \ \cs #1 ~ is~ \token_if_cs:NF #1 { \emph {not} ~ } a~ control~ sequence \par meaning: ~ \expandafter \texttt \expandafter { \meaning #1 } \par \medskip }

\NewDocumentCommand \tokencheck {m} { \token_check:N #1 } \ExplSyntaxOff

\def\a{} \def\b#1{} \long\def\c{} \protected\def\d{} \def\1{}

\begin{document}

\makeatletter

\subsection*{Defined with \cs\def} \tokencheck\a \tokencheck\b \tokencheck\c \tokencheck\d \tokencheck\1 \tokencheck~

\subsection*{Undefined} \tokencheck@undefined

\newpage

\subsection*{Primitives} \tokencheck\def \tokencheck\relax \tokencheck\fi \tokencheck\ \relax

\subsection*{Defined with \cs\newbox, \cs\chardef, \dots} \tokencheck&amp; \tokencheck\strutbox \tokencheck\dimen@ \tokencheck\skip@

\makeatother

\end{document}

enter image description here

cgnieder
  • 66,645
  • Great answer! How can I tell in plain TeX whether a certain control sequence is a macro? – Evan Aad Jul 16 '17 at 07:40
  • @Evan Aad: \show\mycs will write info about the control sequence \mycs to the output stream. \meaning\mycs will write info about the control sequence \mycs into the typeset document. – Evan Aad Jul 16 '17 at 08:21