How to prepend and append tokens to a macro definition in the current scope only?

Question

I am facing the problem of both prepending and appending some tokens to the definition of a macro that does not take arguments.

I found out that in the LaTeX2e kernel there is the macro \g@addto@macro for appending the tokens #2 to the definition of the control sequence #1:

> \g@addto@macro=\long macro:
#1#2->\begingroup \toks@ \expandafter {#1#2}\xdef #1{\the \toks@ }\endgroup .
l.2 \show\g@addto@macro

Seems the token register \toks@ is used in order to avoid the need of doubling hashes.
Seems all assignments take place inside a local scope introduced by \begingroup..\endgroup in order to avoid the changes of \toks@ to be permanent.
Seems \xdef is used for having the redefinition outlast the closure of the local scope.

This way the redefinition is global.

Is there a way of appending and prepending tokens to the definition of a macro that does not take arguments whereby the redefinition of the macro in question is not global but restricted to the current scope and whereby also scratch registers do not get changed permanently within the current scope?

You can use \preto and \appto from etoolbox that obey scoping rules. — egreg, Dec 05 '16 at 16:46
BTW, this question feels like it might be [tag:tex-core]: are you looking for a LaTeX-only answer or a generic one (or a plain one, perhaps)? — Joseph Wright, Dec 05 '16 at 17:42

score 9 · Answer 1 · edited Apr 13 '17 at 12:34

The LaTeX kernel version uses only TeX90 primitives and as observed in the question it's very tricky to avoid issues with # tokens and leave all scratch variables unchanged in that scenario. (I was going to say that a global assignment was the only safe way in classical TeX, but I note Ulrich Diez has found a way!)

With e-TeX, we can use the expandable primitive \unexpanded, which acts as an anonymous toks and dosn't therefore need the grouping. This is used by the LaTeX etoolbox package, but the concepts are easy enough in primitives:

\protected\long\def\appto#1#2{%
  \edef#1{%
    \unexpanded\expandafter{#1#2}%
  }%
}
\protected\long\def\preto#1#2{%
  \edef#1{%
    \unexpanded{#2}%
    \unexpanded\expandafter{#1}%
  }%
}

(The etoolbox versions cover the case where #1 is undefined, though whether this is required or not depends on the use case and the semantics you feel apply!)

Ulrich Diez · Answer 2 · 2016-12-06T00:23:30.910

[Due to objections of Bruno Le Floch I saw the need for a major edit of my answer:]

You can use two scratch token registers. One for \edefing and another one for resetting both itself and the one that was used for \edefing:

\documentclass{article}

\makeatletter
%
% \Addtohook{<control sequence>}{<tokens to prepend>}{<tokens to append>}
% =======================================================================
%
% <control sequence> is a control sequence that does not take any arguments.
% \Addtohook _within_the_current_scope_ prepends to the definition of that
% control sequence the <tokens to prepend> and appends the definition of
% that control sequence the <tokens to append>.
\newcommand\Addtohook[3]{%
  \@temptokena\expandafter{%
              \expandafter\@temptokena
              \expandafter{%
              \the\expandafter\@temptokena
                  \expandafter}%
                  \expandafter\toks@
                  \expandafter{%
                  \the\toks@}%
  }%
  \toks@{#2}%
  \toks@\expandafter{\the\expandafter\toks@#1#3}%
  \edef#1{\the\toks@}%
  \the\@temptokena
}%

%\makeatother

\newcommand\myhook{\def\Middlepiece##1{Argument Middlepiece:##1}}%

\begin{document}

\ttfamily
\@temptokena{tEST tEST}%
\toks@{Test Test}%
\string\the\string\@temptokena=\the\@temptokena

\string\the\string\toks@=\the\toks@

\string\myhook=\meaning\myhook

\begingroup
\Addtohook{\myhook}%
          {\def\Frontpiece#1{Argument Frontpiece:#1}}%
          {\def\Tailpiece#1{Argument Tailpiece:#1}}%

\string\myhook=\meaning\myhook

\string\the\string\@temptokena=\the\@temptokena

\string\the\string\toks@=\the\toks@
\endgroup

\string\myhook=\meaning\myhook

\end{document}

My favorite way (proposed by Bruno Le Floch) is using \edef and only one scratch token register.

\documentclass{article}

\makeatletter
%
% Paraphernalia:
% ==============
\newcommand\UD@exchange[2]{#2#1}%
%
% \Addtohook{<control sequence>}{<tokens to prepend>}{<tokens to append>}
% =======================================================================
%
% <control sequence> is a control sequence that does not take any arguments.
% \Addtohook _within_the_current_scope_ prepends to the definition of that
% control sequence the <tokens to prepend> and appends the definition of
% that control sequence the <tokens to append>.
\newcommand\Addtohook[3]{%
  \expandafter\UD@exchange
  \expandafter{%
  \expandafter\toks@
  \expandafter{%
  \the\toks@}}{%
  \toks@\expandafter\expandafter
        \expandafter{%
        \expandafter\UD@exchange
        \expandafter{#1#3}{#2}}\edef#1{\the\toks@}}%
}%

%\makeatother

\newcommand\myhook{\def\Middlepiece##1{Argument Middlepiece:##1}}%

\begin{document}

\ttfamily
\@temptokena{tEST tEST}%
\toks@{Test Test}%
\string\the\string\@temptokena=\the\@temptokena

\string\the\string\toks@=\the\toks@

\string\myhook=\meaning\myhook

\begingroup
\Addtohook{\myhook}%
          {\def\Frontpiece#1{Argument Frontpiece:#1}}%
          {\def\Tailpiece#1{Argument Tailpiece:#1}}%

\string\myhook=\meaning\myhook

\string\the\string\@temptokena=\the\@temptokena

\string\the\string\toks@=\the\toks@
\endgroup

\string\myhook=\meaning\myhook

\end{document}

The methods shown so far don't preserve the \long-status of macros defined in terms of \long.

This can be implemented by checking whether the \meaning contains the phrase \long in the right place:

\documentclass{article}

\makeatletter
%
% Paraphernalia:
% ==============
\newcommand\UD@exchange[2]{#2#1}%
\newcommand\UD@firstoftwo[2]{#1}%
\newcommand\UD@secondoftwo[2]{#2}%
%
%------------------------------------------------------------------------------
% Check whether argument is empty:
%..............................................................................
% \UD@CheckWhetherNull{<Argument which is to be checked>}%
%                     {<Tokens to be delivered in case that argument which is
%                       to be checked is empty>}%
%                     {<Tokens to be delivered in case that argument which is
%                       to be checked is not empty>}%
%
% The gist of this macro comes from Robert R. Schneck's \ifempty-macro:
% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>
%
% (\romannumeral expansion was introduced by me in order to overcome the
%  concerns and worries about improperly balanced \if..\else..\fi constructs.)
\newcommand\UD@CheckWhetherNull[1]{%
  \romannumeral0\expandafter\UD@secondoftwo\string{\expandafter
  \UD@secondoftwo\expandafter{\expandafter{\string#1}\expandafter
  \UD@secondoftwo\string}\expandafter\UD@firstoftwo\expandafter{\expandafter
  \UD@secondoftwo\string}\expandafter\expandafter\UD@firstoftwo{ }{}%
  \UD@secondoftwo}{\expandafter\expandafter\UD@firstoftwo{ }{}\UD@firstoftwo}%
}%
%-----------------------------------------------------------------------------
% Check whether argument's first token is a catcode-1-character:
%.............................................................................
% \UD@CheckWhetherBrace{<Argument which is to be checked>}%
%                      {<Tokens to be delivered in case that argument which is
%                        to be checked has leading catcode-1-token>}%
%                      {<Tokens to be delivered in case that argument which is
%                        to be checked has no leading catcode-1-token>}%
\newcommand\UD@CheckWhetherBrace[1]{%
  \romannumeral0\expandafter\UD@secondoftwo\expandafter{\expandafter{%
  \string#1.}\expandafter\UD@firstoftwo\expandafter{\expandafter
  \UD@secondoftwo\string}\expandafter\expandafter\UD@firstoftwo{ }{}%
  \UD@firstoftwo}{\expandafter\expandafter\UD@firstoftwo{ }{}\UD@secondoftwo}%
}%
%-----------------------------------------------------------------------------
% Check whether argument has a first token whose meaning starts with the
% phrase \long macro:-> :
%.............................................................................
% \UD@CheckWhetherLeadingLong{<argument which is to be checked>}%
%                            {<Tokens to be delivered in case <argument which
%                              is to be checked> has 1st token whose meaning
%                              has leading phrase \long macro:-> >}%
%                            {<Tokens to be delivered in case <argument which
%                              is to be checked> has no 1st token whose
%                              meaning has leading phrase \long macro:-> >}%
\begingroup
\def\UDtempa#1{%
  \endgroup
  \newcommand\UD@CheckWhetherLeadingLong[1]{%
    \romannumeral0\UD@CheckWhetherBrace{##1}%
    {\expandafter\expandafter\UD@firstoftwo{ }{}\UD@secondoftwo}%
    {\expandafter\expandafter\expandafter\UD@secondoftwo
     \expandafter\string\expandafter{%
     \expandafter\UD@CheckWhetherLeadingLongB\meaning##1.#1}{}}%
  }%
  \newcommand\UD@CheckWhetherLeadingLongB{}%
  \long\def\UD@CheckWhetherLeadingLongB##1#1{%
    \UD@CheckWhetherNull{##1}%
    {\UD@exchange{\UD@firstoftwo}}{\UD@exchange{\UD@secondoftwo}}%
    {\UD@exchange{ }{\expandafter\expandafter\expandafter\expandafter
     \expandafter\expandafter\expandafter}\expandafter\expandafter
     \expandafter}\expandafter\UD@secondoftwo\expandafter{\string}%
  }%
}%
\begingroup
\edef\UDtempa{%
  \string\long\UD@firstoftwo{ }{}%
  \string m\string a\string c\string r\string o\string:\string-\string>%
}%
\expandafter\endgroup\expandafter\UDtempa\expandafter{\UDtempa}%
%-----------------------------------------------------------------------------
% Within the current scope prepend and append some tokens to the definition
% of a control sequence that does not take arguments:
%.............................................................................
% \Addtohook{<control sequence>}{<tokens to prepend>}{<tokens to append>}
\newcommand\Addtohook[3]{%
    \expandafter\UD@exchange\expandafter{%
      \expandafter\toks@\expandafter{\the\toks@}%
    }{%
      \toks@\expandafter\expandafter\expandafter{%
      \expandafter\UD@exchange\expandafter{#1#3}{#2}}%
      \UD@CheckWhetherLeadingLong{#1}{\long}{}\edef#1{\the\toks@}%
    }%
}%

%\makeatother

\newcommand\myhook{\def\Middlepiece##1{Argument Middlepiece:##1}}%

\newcommand*\myhookB{\def\Middlepiece##1{Argument Middlepiece:##1}}%

\begin{document}

\ttfamily
\@temptokena{tEST tEST}%
\toks@{Test Test}%

Appending to \string\long-macro:

\string\the\string\@temptokena=\the\@temptokena

\string\the\string\toks@=\the\toks@

\string\myhook=\meaning\myhook

\begingroup
\Addtohook{\myhook}%
          {\def\Frontpiece#1{Argument Frontpiece:#1}}%
          {\def\Tailpiece#1{Argument Tailpiece:#1}}%

\string\myhook=\meaning\myhook

\string\the\string\@temptokena=\the\@temptokena

\string\the\string\toks@=\the\toks@
\endgroup

\string\myhook=\meaning\myhook

\hrulefill

Appending to non-\string\long-macro:

\string\the\string\@temptokena=\the\@temptokena

\string\the\string\toks@=\the\toks@

\string\myhookB=\meaning\myhookB

\begingroup
\Addtohook{\myhookB}%
          {\def\Frontpiece#1{Argument Frontpiece:#1}}%
          {\def\Tailpiece#1{Argument Tailpiece:#1}}%

\string\myhookB=\meaning\myhookB

\string\the\string\@temptokena=\the\@temptokena

\string\the\string\toks@=\the\toks@
\endgroup

\string\myhookB=\meaning\myhookB

\end{document}

Question: Is there a method for appending to the definition of a macro that does not take arguments in case in the meantime some tokens of the definition of that macro have been redefined in terms of \outer? (I did not find an answer by now.)

I.e.

\newcommand\myhook{\def\Middlepiece##1{Argument Middlepiece:##1}}%
\outer\def\Middlepiece{This is outer now!}%
...
\Addtohook{\myhook}%
          {\def\Frontpiece#1{Argument Frontpiece:#1}}%
          {\def\Tailpiece#1{Argument Tailpiece:#1}}%

Did you overlook the simpler \long\gdef\swap#1#2{#2#1} \long\gdef\Addtohook#1#2#3{\expandafter\swap\expandafter{\expandafter\toks@\expandafter{\the\toks@}}{\expandafter\expandafter\expandafter\toks@ \expandafter\expandafter\expandafter{\expandafter\swap\expandafter{#1}{#2}#3}\edef#1{\the\toks@}}} or did I miss some advantage of your method? — Bruno Le Floch, Dec 05 '16 at 22:28
By the way, I wrote some time back the package multiexpand which looks like it does some of what your expandtimes code does. — Bruno Le Floch, Dec 05 '16 at 22:30
I believe that the only way to do the last task you mention (adding to a hook containing outer macros) is only possible if one is ready to decompose the contents of the hook token by token. Or use \suppressoutererror in LuaTeX. — Bruno Le Floch, Dec 05 '16 at 22:33
@BrunoLeFloch I don't know what I was thinking when wrapping the thing into an unnecessary \toks@{..}..\the\toks@. — Ulrich Diez, Dec 06 '16 at 00:00

score 4 · Answer 3 · answered Dec 05 '16 at 22:43

For adding tokens to a parameterless macro you can use, for simple cases, \preto and \appto from the etoolbox package, which obey the standard scoping rules.

\documentclass{article}
\usepackage{etoolbox}

\begin{document}

\newcommand{\foo}{FOO}

\foo

\begingroup\preto\foo{AAA}\foo\endgroup

\foo

\end{document}

will print

FOO
AAAFOO
FOO

If you have something more complicated, for instance prepending or appending code with #, such as a \def command, you can use regexpatch:

\documentclass{article}

\usepackage{regexpatch}

\begin{document}

\newcommand{\foo}[1]{FOO#1FOO}

\texttt{\meaning\foo}\par
\foo{x}

\bigskip

\begingroup
\xpretocmd\foo{\def\AAA##1{AAA##1AAA}}{}{}
\texttt{\meaning\foo}\par
\foo{x} \AAA{y}
\endgroup

\bigskip

\texttt{\meaning\foo}\par
\foo{x}

\end{document}

Note the natural syntax: you want double # characters, so you type ##1.

There is also \xapptocmd for appending.

score 3 · Answer 4 · answered Dec 05 '16 at 17:32

Use the xpatch package. It offers commands to prepend and append stuff to macro bodies, as well as to replace tokens. By putting the replacement into a group, it stays local.

The following document is typeset by the code below it.

\documentclass{article}
\usepackage{xpatch}
\begin{document}

\section{Some section to be referenced}\label{sec}

This is a normal reference to section~\ref{sec}.
\bgroup
\xpretocmd\ref{***}{}{}%
\xapptocmd\ref{+++}{}{}%
This is a strange reference to section~\ref{sec}.
\egroup
This is a normal reference to section~\ref{sec}.

\end{document}

How to prepend and append tokens to a macro definition in the current scope only?

4 Answers4

Linked