Functionality of `\unrestored@protected@xdef`?

Question

     So, I was reading texdoc tamethebeast (Taming the BeaST: The B to X of BibTeX) when I came across a macro I'd never seen before in its discussion of the implementation of \thebibliography; this was the macro \@mkboth. This answer to the question 'What do \@mkboth and \markboth do?' explains what that does, but I then came across another macro inside that command's definition that I've also never seen before; that macro was \unrestored@protected@xdef. Neither the LaTeX2e unofficial reference manual (August 2017) nor this web listing documenting TeX Primitive Control Sequences contains any mention of its function. That leads me to wonder what it does, so can anyone answer that and/or point me toward a reference that does explain how it works?

Answer 1

The command is about defining macros with expansion. The TeX primitive \edef defines a macro like \def, but it expands the definition text. For example:

\def\Who{World}
\edef\Hello{Hello \Who}

The macro \Who is expanded and the definition text of \Hello is Hello World.

\xdef is just the global variant of \edef: \xdef = \global\edef.

In many circumstances, the hard expansion by \edef or \xdef will fail, if fragile macros are expanded. For example, the following macro is not robust:

\def\FragileMacro{%
  \def\MyMacro{...}%
  ...
}

If \FragileMacro is used inside \edef or \xdef, then \def is not an expandable token and is just kept without further processing, i.e. it does not define anything. Then \MyMacro is tried to be expanded. If it is undefined, then an error message because of an undefined command is thrown. Or, it expands to some non-active characters. Then, a later execution of \FragileMacro will fail, because \def cannot define non-active characters.

The LaTeX way is protection:

• \protect before a fragile macro prevents its expansion.
• Macros can be defined as robust command via \DeclareRobustCommand. It smuggles \protect before an internal macro that contains the possibly fragile command text.

There are several problematic contexts, e.g. displaying something in the .log file/console, writing to a file, or in this case expansion by \edef or \xdef. The correct definition of \protect depends on these contexts. In the case of \edef or \xdef it is \@unexpandable@protect:

\def\@unexpandable@protect{\noexpand\protect\noexpand}

Then

\let\protect\@unexpandable@protect
\edef\FooBar{\protect\FragileCommand}

becomes after the first expansion of \protect to

\edef\FooBar{\noexpand\protect\noexpand\FragileCommand}

\noexpand prevents the expansion of the next token, and the definition is equivalent to

\def\FooBar{\protect\FragileCommand}

\protect is retained and \FragileCommand is prevented from expansion.

The internal LaTeX macros \@protected@edef and \@protected@xdef are the LaTeX counterparts for plain TeX's \edef and \xdef:

\def\protected@edef{%
   \let\@@protect\protect
   \let\protect\@unexpandable@protect
   \afterassignment\restore@protect
   \edef
}
\def\protected@xdef{%
   \let\@@protect\protect
   \let\protect\@unexpandable@protect
   \afterassignment\restore@protect
   \xdef
}

First \@@protect gets the current meaning of \protect. Then \protect is redefined according to the context (expanded definition). \protect gets the meaning of \@unexpandable@protect. The redefinition of \protect is reverted by \afterassignment. It calls \restore@protect right after the next assignment that is \edef or \xdef. \restore@protect restores the definition of \protect with the previously defined \@@protect:

\def\restore@protect{\let\protect\@@protect}

The version \unrestored@protected@xdef is an optimized version of \@protected@edef that still redefines \protect, but leaves out the restoration of \protect:

\def\unrestored@protected@xdef{%
   \let\protect\@unexpandable@protect
   \xdef
}

The macro definition is smaller and faster, it avoids a macro expansion and two assignments. But, it can be only used in special circumstances, typically at the end of a group:

\begingroup
  ...
  \@unrestored@protected@xdef\FooBar{\protect\FragileMacro}%
\endgroup

At the end of the group, any local changes (here, the redefinition of \protect) is reverted anyway. The explicit restoration of \protect is not needed.

Typical use cases in latex.ltx:

\def\@xfootnote[#1]{%
   \begingroup
     \csname c@\@mpfn\endcsname #1\relax
     \unrestored@protected@xdef\@thefnmark{\thempfn}%
   \endgroup
   \@footnotemark\@footnotetext
}
\def\@xfootnotemark[#1]{%
   \begingroup
      \c@footnote #1\relax
      \unrestored@protected@xdef\@thefnmark{\thefootnote}%
   \endgroup
   \@footnotemark
}
\def\@xfootnotenext[#1]{%
  \begingroup
     \csname c@\@mpfn\endcsname #1\relax
     \unrestored@protected@xdef\@thefnmark{\thempfn}%
  \endgroup
  \@footnotetext
}

The group can also be seen in \markboth:

\def\markboth#1#2{%
  \begingroup
    \let\label\relax \let\index\relax \let\glossary\relax
    \unrestored@protected@xdef\@themark {{#1}{#2}}%
    \@temptokena \expandafter{\@themark}%
    \mark{\the\@temptokena}%
  \endgroup
  \if@nobreak\ifvmode\nobreak\fi\fi
}

The group seems to be missing in \@markright:

\def\@markright#1#2#3{\@temptokena {#1}%
  \unrestored@protected@xdef\@themark{{\the\@temptokena}{#3}}%
}

But \@markright itself is called inside a group:

\def\markright#1{%
  \begingroup
    \let\label\relax \let\index\relax \let\glossary\relax
    \expandafter\@markright\@themark {#1}%
    \@temptokena \expandafter{\@themark}%
    \mark{\the\@temptokena}%
  \endgroup
  \if@nobreak\ifvmode\nobreak\fi\fi
}

After the definition of \@themark by \@markright, the new contents for the marks register needs to be added to \mark. However, \mark again would expand the argument (\@themark is already expanded by \@unrestored@protected@xdef). Therefore, a scratch token register \@temptokena is used as trick. \the before a token register expands the token register only once, the contents are not expanded further.