How could one change the catcodes inside the definition of an environment?

Question

I tried to find a clever and robust way to use non-letter symbols like %, _, ], [ etc.inside the definition of an environment. I tried for example to change [ and ] to letters for using them locally. The following code worked well

\begingroup
\catcode`\[=11
\catcode`\]=11
\def\[a{some unicode character}
\def\]m{something else}

\[a\]m
\endgroup

but when I tried to roll up the above code into an special enviroment like

\newenvironment{newaccents}{\begingroup%
\catcode`\[=11%
\catcode`\]=11%
\def\[a{some unicode character}
\def\]m{something else}
}{%
\catcode`\[=12%
\catcode`\]=12%
 \endgroup}

the TeX comlains with

Undefined control sequence.
 \begin{newaccents}\[a
                           \end{newaccents}

So my questions is: How one could change in a local special way the codes of non-letters characters so that they can to be used inside the definition of local-defined commands?

Answer 1

Assuming you really want to do it, you need a two-step approach: first define the commands you want under the right category code setup

\begingroup
\catcode`[=11 \catcode`]=11
\gdef\[a{foo}
\gdef\]m{baz}
\endgroup

Then you can define the environment so that it changes the category codes:

\newenvironment{newaccents}
  {\catcode`[=11 \catcode`]=11 }
  {}

The reason why your attempt fails is that when you do

\newenvironment{newaccents}{\begingroup%
\catcode`\[=11%
\catcode`\]=11%
\def\[a{some unicode character}
\def\]m{something else}
}{%
\catcode`\[=12%
\catcode`\]=12%
 \endgroup}

TeX has already absorbed the tokens in the three arguments to \newenvironment under a regime where [ and ] have category code 12, so they cannot be changed any more.

The global definitions I give in the first code block have no influence on usage outside the newaccents environment, where the category code of [ and ] is 12. You can still use \[ and \] with their original meaning within the newaccents environment, provided you ensure they're followed by a space.

In any case, you should recall that an environment where category code changes are performed cannot be used in the argument to a command.

---

There is a different strategy for obtaining a similar result without category code changes.

\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}

\DeclareTextCommand{\NAlbrack}{T1}{?} % default is not really defined
\DeclareTextCompositeCommand{\NAlbrack}{T1}{a}{foo}
% other composites of the form \[x

\DeclareTextCommand{\NArbrack}{T1}{?} % default is not really defined
\DeclareTextCompositeCommand{\NArbrack}{T1}{m}{baz}
% other composites of the form \]x

\newenvironment{newaccents}
  {\let\[\NAlbrack \let\]\NArbrack}
  {}

\begin{document}

\[a=b\]

\begin{newaccents}
Does \[a print `foo'?

Does \]m print `baz'?
\end{newaccents}

\end{document}

This of course means \[ and \] cannot be used for displays, but I don't think it's a problem for your application.

I used T1 as the font encoding, but you can use \encodingdefault, provided the code is after loading the package for font encodings, be it fontenc or fontspec.

The trick is to define a command \NAlbrack that can look forward for a declared combination, in this case an a. If it finds it, it prints whatever you defined the combination to be, otherwise it prints a ? (just an arbitrary choice, you can define it to do something else in case of no predefined combinations).

In the body of newaccents, I simply change \[ to be the same as \NAlbrack.

Answer 2

Change the catcodes temporarily, for the duration of your definition; then they go back to default, until your environment re-activates them the way you do now.

\catcode`[=11
\catcode`]=11
\newenvironment{newaccents}{
      \catcode`[=11
      \catcode`]=11
\def\[a{(Accent)}
}{}
\catcode`[=12
\catcode`]=12

Note that \begin will begin a group for you; and since catcodes are local, there's no need to set them back by hand.

Edit: Of course \global\newenvironment doesn't work. Fixed now, sorry for the distraction.

Answer 3

I don't see the point in having cat code changes here: \[a is a 'valid' command name, if really needed.

Since all is defined in an environment, \begingroup and \endgroup aren't needed here at all.

\documentclass{article}

\usepackage[utf8]{inputenc}


\newenvironment{newaccents}{%\begingroup%
%\catcode`\[=11%
%\catcode`\]=11%
\def\[a{some unicode character}%
\def\]m{something else}%
}{%
%\catcode`\[=12%
%\catcode`\]=12%
% \endgroup
}

\begin{document}
\begin{newaccents}
\[a äöü \]m
\end{newaccents}
\end{document}

Answer 4

The following answer doesn't add much to @egreg's answer, but shows an alternative, slightly more compact route to the same result which might possibly be illuminating.

As noted, the crucial thing to know here is that TeX assigns a (catcode, charactercode) pair to every character once, when it is first read. That means that your \def\[a has to happen in a context where [ will be read with catcode 11.

The second thing to know is that an environment \begin{foo} ... \end{foo} expands into (essentially) \begingroup\foo ... \endfoo\endgroup (plus some bookkeeping); that is, you can (mostly) define an environment {foo} simply by defining the pair of macros \foo and \endfoo.

Thus:

\begingroup
\catcode`\[=11 % treat [ as a letter within this group
\gdef\newaccents{% Now define the {newaccents} environment
  \def\[a{foo}% a macro with a two-letter name
  \catcode`\[=11 % make [ a letter within the body of this environment
}
\global\let\endnewaccents=\relax
\endgroup

Notice that \def\[a happened within the group, so it's interpreted as two letters. The \gdef means that the definition of this and of \foo ‘escape’ the group.

After that...

\begin{document}
Here is text.
\[
E=mc^2.
\]

\begin{newaccents}
  Here is some \[a text, with displayed maths:
  \[ F=ma. \]
\end{newaccents}

\end{document}

Doing it this way means that you don't have to wish that \global\newenvironment worked. Also, a difference between this solution and @egreg's (too slight to be much of an advantage) is that \[a isn't defined at all, even inaccessibly, outside the {newaccent} environment.