4

This is my first question here on TeXSE, and my second question on the StackExchange community. I am reading Notes on Programming in TeX by Dr. Christian Feuersänger as part of a general introduction to TeX; I've been using LaTeX for about a year, and I want to learn in general terms how the TeX system works, i.e. macros, counters, groups, etc. Indeed, after this document I plan to read TeX for the Impatient for a more thorough introduction to the system.

Subsections 2.1 through 2.4 from the Notes briefly discuss macros, counters and tokens, but I don't quite understand what is the difference between a counter and a token. Counters can store an integer value, and can be incremented/decremented by x with the syntax \advance\count0 by x, where \count0 is a previously defined counter. What does each do? How does TeX understand each one? Which purposes do both serve?

Thanks

Edit:

Thanks for the answers. Relating to the one from Phelype Oleinik, sorry, what I meant with tokens is instead registers. Feuersänger says the following regarding registers:

There are also 255 token registers which can be thought of as special string variables. Of course, every macro assignment \def\macro{content} is also some kind of string variable, but token registers are special: their contents won't be expanded when used with \the\toks<number>. This can be used for fine grained expansion control, see Section 2.3 below (p. 3).

Further, he gives the example

\toks0={abc}%
\toks1={DEF}%
The value is now \the\toks0 \the\toks1.

A counter cannot clearly store the literals abc, but what is then the difference between registers and macros, when one uses a macro to store some control sequence, i. e. just a plain word? Both expand to their content, right?

George1917
  • 41
  • 6
  • 7
    Tokens and counters are completely different things. A counter is like an integer variable in other programming languages, as you already said. A token is any single "thing" seen by TeX (roughly speaking). For example, the line \advance\count0 by x contains the 8 tokens: \advance, \count, 0, <space>, b, y, <space>, x. Other than that, I don't think I understand your question. – Phelype Oleinik Dec 15 '21 at 01:49
  • 2
    The concept of token relates to how information is digested by TeX. In the simplest conceptualization, digestion of the input stream of a document occurs one byte at a time, except for control sequences, which are sequences of bytes that begin with an escape character byte of category code 0 (the \ byte is the default escape character). This is why multi-byte entities such as \advance and \count constitute single tokens. Reference: https://en.wikibooks.org/wiki/TeX/catcode – Steven B. Segletes Dec 15 '21 at 03:40
  • From the outside: A concept of tokens is needed because the input stream is a mixture of instructions and material to be typeset: counter tokens store integers; a box can store formatted material; a token stream t,e,x,t will typeset text in the current font, location, colour; a macro is a token that expands to its definition (itself a token stream, and which may have more macros, and/or primitives, and/or text) and may or may not take parameters. A group, {...}, is local: what happens in the group stays in the group, e.g. font change (unless global actions are occurring). – Cicada Dec 15 '21 at 06:16
  • Plus, a macro can (re)define itself, which comes in very handy. – Cicada Dec 15 '21 at 06:19
  • With the update, looks like https://tex.stackexchange.com/questions/107102/using-toks-registers to me (the registers you are interested in are normally referred to as 'toks') – Joseph Wright Dec 16 '21 at 06:31
  • Also see for example https://tex.stackexchange.com/questions/39747/is-toks-necessary-to-define-gaddtomacro-can-token-registers-be-avoided-in-g – Joseph Wright Dec 16 '21 at 06:32
  • 2
    If you are really looking for a "thorough introduction to the system", you should imo just read the TeXbook. – schtandard Dec 16 '21 at 06:37
  • Thanks Joseph Wright for both suggestions, I will definitively give it a read. And yes, probably I should read the TeXBook, but I found TeX by Topic by Victor Eijkhout which appears to be what I need. – George1917 Dec 20 '21 at 19:36

0 Answers0