Modify catcode in the preamble or in the \AtBeginDocument?

Question

I want to modify the catcode of comma, with the following code

\documentclass{article}
\AtBeginDocument{%
  \catcode`\,=\active
  \protected\def,{...}%
}
\begin{document}
a,b,c $a,b,c$.
\end{document}

I get the following error message:

! Missing control sequence inserted.
<inserted text> 
                \inaccessible 
l.6 \begin{document}

But with the following code, I get correct result:

\documentclass{article}
\begingroup
  \catcode`\,=\active
  \protected\gdef,{...}%
\endgroup
\AtBeginDocument{%
  \catcode`\,=\active
}
\begin{document}
a,b,c $a,b,c$.
\end{document}

I'm wondering what are the differences between these two ways. Why the first way causes problem?

Edit: Althougn David Carlisle give another related question, but I think this question maybe not the same as that question, since both examples contains

\AtBeginDocument{%
  \catcode`\,=\active
}

I could not understand why only one example causes problem.

As explained in the linked answer this is the standard issue that catcode changes (like \verb to not apply to tokens, so do not work in the argument of another command) — David Carlisle, May 22 '15 at 09:34
@DavidCarlisle But I put \catcode,=\activeinside\AtBeginDocument` in both of the examples. I think it is not the same as your linked answer. — Z.H., May 22 '15 at 10:04
@Z.H. You need \catcode`\,=active outside of \AtBeginDocument as well: the tokenization needs to be correct. — Joseph Wright, May 22 '15 at 10:11
@Z.H. it is the same (and independent of anything specific about \AtBeginDocument I'm not sure what difference you see? — David Carlisle, May 22 '15 at 10:11
@JosephWright I am still confused, you means the catcode change doesn't reset after the group? — Z.H., May 22 '15 at 10:23
@DavidCarlisle The linked question doesn't explain how to solve the issue: either this question should be reopened with an answer that does, or the linked one needs editing. — Joseph Wright, May 22 '15 at 10:25
@JosephWright hmm perhaps this is a better dup http://tex.stackexchange.com/questions/12638/how-to-change-catcode-in-a-macro?rq=1 (or we could put an answer here, but it's such a FAQ it must be a dup of something:-) — David Carlisle, May 22 '15 at 10:36
@DavidCarlisle Oh I understand it now. In fact, the statement you could not change catcodes inside a command in the linked question is wrong, The correct statement is you could not change catcodes and redefine the active character inside the same command. — Z.H., May 22 '15 at 11:03
@Z.H. it isn't the definition that you can not do it is that the catcode change does not affect the tokens in the argument \mbox{\catcode\,=13 zzzz , zzzz}it is a normal, non-active,between thezzz` whatever that code is doing, definition or use. — David Carlisle, May 22 '15 at 11:05
@DavidCarlisle Is there any way to make TeX do it anyway? Or would it involve a \special and a "fake" version of TeX with different handling of \def? — 1010011010, May 22 '15 at 12:25
@1010011010 why? why not just use the standard tex syntax \catcode\,=13\mbox{ zzzz , zzzz}` obviously if you use some unspecified non-tex program it could do whatever you specify, but it would have to be very different from TeX so the question of making some Tex syntax work on that new system may not make sense at all. — David Carlisle, May 22 '15 at 12:28

Heiko Oberdiek · Accepted Answer · 2015-05-22T18:18:34.533

In the argument of \AtBeginDocument, the comma is not active. The comma becomes active, when the argument with the category code setting is executed later in \begin{document}. Also keep in mind, that the category codes gets assigned during the tokenization. When an argument is read, the bytes of the input are tokenized with the current category code regime. When the argument is executed, the category change of the comma will affect later tokenizations, but not the tokens of the remainder of the argument.

Lowercase tilde trick

The typical pattern is to make use of the tilde character, because it is active in LaTeX by default. Then \lowercase is used to convert the tilde to the comma. The category code does not change in this process and the result is an active comma. A group prevents a permanent setting of the lowercase setting for the tilde.

\documentclass{article}
\AtBeginDocument{%
  \catcode`\,=\active
  \begingroup
    \lccode`\~=`\, %
  \lowercase{\endgroup
    \protected\def~%
  }{...}%
}
\begin{document}
a,b,c $a,b,c$.
\end{document}

Changing the category code for the whole statement

Another pattern is to change the category code for the whole statement including the argument. First the original category code of the comma is saved, e.g. via a resetting macro (\ResetCommaCatcode). Then the category code of the comma is changed. Now \AtBeginDocument is called, the argument now contains an active comma token (in \protected\def,). Afterwards the category code of the comma is reset to the previous value.

\documentclass{article}

\edef\ResetCommaCatcode{%
  \catcode`\noexpand\,=\the\catcode`\,\relax
}
\catcode`\,=\active
\AtBeginDocument{%
  \catcode`\,=\active
  \protected\def,{...}%
}
\ResetCommaCatcode

\begin{document}
a,b,c $a,b,c$.
\end{document}

Modify catcode in the preamble or in the \AtBeginDocument?

1 Answers1

Lowercase tilde trick

Changing the category code for the whole statement

Linked