Using a character as a macro within another macro in (La)TeX

Question

Goal: What I would like to do is to allow _ and ^ to be used to generate sub and superscripts in text mode only within a specific macro. I know that I can make _ and ^ active throughout the document, but that messes with other packages which assume that they will not be.

I've dug around a bit, and haven't found an example that implements something close to what I want.

I've been basing my code off of examples like https://texfaq.org/FAQ-actinarg This works for creating a command in which the argument is typeset with _ and ^ being active:

\documentclass{article}
\begin{document}
\begingroup
\catcode`_=\active%
\gdef_#1{\ensuremath{{}\sb{#1}}}%
\catcode`^=\active%
\gdef^#1{\ensuremath{{}\sp{#1}}}%
\endgroup
\def\subsup{%
  \begingroup
    \catcode_=\active \catcode^=\active
    \Xsubsup
}
\def\Xsubsup#1{%
    #1%
  \endgroup
}
\subsup{foo_x bar^y baz}
% foo_a bar^b baz % this will cause an error because _ is not active.

Now, if I add the function that I want to embed this in:

\DeclareRobustCommand*\newMacro[2][\null]{
  % other processing and formatting not directly relevant to this example
  \subsup{#2}%
}
\newMacro{foo_x bar^y baz}
\end{document}

I get math mode errors:

ERROR: Missing $ inserted.
--- TeX said ---
<inserted text> 
                $
l.28 \newMacro{foo_x bar^y baz}

So I tried to embed the subsup macros into newMacro (I've repeated some of the code from above for ease of copying and pasting.):

\documentclass{article}
\begin{document}
\begingroup
\catcode`_=\active%
\gdef_#1{\ensuremath{{}\sb{#1}}}%
\catcode`^=\active%
\gdef^#1{\ensuremath{{}\sp{#1}}}%
\endgroup
\DeclareRobustCommand*\newMacro[2][\null]{
\def\subsup{%
  \begingroup
    \catcode_=\active \catcode^=\active
    \Xsubsup
}
\def\Xsubsup#1{%
    #1%
  \endgroup
}
  % other processing and formatting not directly relevant to this example
  \subsup{#2}%
}
\subsup{foo_x bar^y baz} %this is undefined, and so fails because newMacro hasn't been called yet.
\newMacro{foo_x bar^y baz}
\subsup{foo_x bar^y baz} % this works because it comes after newMacro has been called.
\end{document}

This still doesn't work, but now I get the error:

ERROR: Use of \Xsubsup doesn't match its definition.
--- TeX said ---
\newMacro  ...\Xsubsup #1{#1\endgroup } \subsup {
                                                  #2}
l.28 \newMacro{foo_x bar^y baz}

Why do the two subsup functions work only when not embedded in another macro? Why do I get errors that Xsubsup doesn't match its definition only when it is embedded within another macro? Is there anyway to get around this?

Answer 1

Short answer is that it is not possible.

This is exactly why you can not use \verb in the argument of another macro. catcode assignments only affect the conversion of unparsed characters in the input file into character tokens. Once you pass token lists around as #1 etc you are passing lists of tokens and each token has a catcode already so the catcode assignments are not looked at.

so when you have

\subsup{#2}%

It is passed as token list that has already been constructed so although the catcodes of _ are changed it doesn't matter tex is not going to see an unparsed _ character only a _ token that already has its normal catcode.

However if you are prepared to be very modern and use an e-Tex extension from the 1990's then you can re-tokenize a tokenlist using the \scantokens primitive. \scantokens essentially writes its arguments out to an internal file stream and then reads it back as if from \input using the catcodes in effect so changing

\def\Xsubsup#1{%
    #1%
  \endgroup

to

\def\Xsubsup##1{%
   \scantokens{##1}%
  \endgroup
}

Fixes things, The meet its definition error is because you had use #1 rather than ##1 in the inner definition.

Answer 2

As an alternative to David's answer, you could try having \newMacro take fewer arguments and let the final call to \subsup absorb what would be the second argument. Here's my suggested code:

\documentclass{article}
\begin{document}

\makeatletter
\DeclareRobustCommand\newMacro[1][\null]{
  \let\subsup\@subsup
  % Other stuff
  \subsup
}

\def\@subsup#{
  \bgroup
  \catcode`^=\active
  \catcode`_=\active
  \let\next=
}
\makeatother

\begingroup
\catcode`^=\active
\catcode`_=\active
\gdef^#1{\ensuremath{{}\sp{#1}}}
\gdef_#1{\ensuremath{{}\sb{#1}}}
\endgroup

\newMacro{ab_c}

\end{document}

I like the #{ trick in delimited macros. It enforces the use of braces in the argument to be passed to \subsup and, simultaneously, uses the braces to form a group that protects the catcode change.

This of course only works if the last thing that \newMacro does is call \subsup. However, even if it wants to do other stuff, in the manner of

\newcommand\newMacro[1]{
  % stuff
  \subsup{#1}
  % more stuff
}

you can simply break it into several macros, in much the same way that your original definition of \subsup was broken, with the help of the somewhat-obscure \aftergroup directive:

% #1 is a token to put after \subsup
\def\subsup#1#{
  \bgroup
  % catcode changes
  \aftergroup#1
  \let\next=
}
\def\newMacro{
  % stuff
  \subsup{\XnewMacro}
}
\def\XnewMacro{
  % more stuff
}

And in this way you can completely skip over the "argument" in which the catcode changes are to occur.