commands with optional arguments in \write

Question

To extract text from huge LaTeX-files in a safe and controllable manner, I am having LaTeX itself write to a textfile. To retain and adapt some formatting informations and structural elements I have given the file a new header where I redefine the commands for this special purpose. This works fine until I need commands with optional arguments.

Using newfile for convenience:

\documentclass{minimal}

\usepackage{newfile}
\newoutputstream{out}
\openoutputfile{out.text}{out}


\newcommand{\standardexample}[3]{#1:#2(#3)}
\newcommand{\optionalexample}[3][default]{#1:#2(#3)}

\begin{document}

text \standardexample{1}{2}{3} and \optionalexample[not default]{2}{3} or \optionalexample{2}{3}

\addtostream{out}{
text \standardexample{1}{2}{3} and \optionalexample[not default]{2}{3} or \optionalexample{2}{3}
}

\closeoutputstream{out}

\end{document}

I get

text 1:2(3) and not default:2(3) or default:2(3)

in the pdf as expected, but

text 1:2(3) and \optionalexample[not default]{2}{3} or \optionalexample{2}{3}

in the text-file with no errors in the log.

Testing without newfile, e.g.

\documentclass{minimal}

\newcommand{\standardexample}[3]{#1:#2(#3)}
\newcommand{\optionalexample}[3][default]{#1:#2(#3)}


\newwrite\tempfile

\begin{document}
\immediate\openout\tempfile=\jobname.tmp
\immediate\write\tempfile{text \standardexample{1}{2}{3} and \optionalexample[not default]{2}{3} or \optionalexample{2}{3}}
\immediate\closeout\tempfile


text \standardexample{1}{2}{3} and \optionalexample[not default]{2}{3} or \optionalexample{2}{3}


\end{document}

the typesetting is broken off by a couple of error messages (some of which are even included in the tmp-file), but I can't really make sense of them.

Both an expanation and a workaround would be great.

Answer 1

Commands defined with an optional argument are automatically robustified, but this means they survive \protected@write; they definitely don't survive \write exactly because of the protection mechanism.

It mostly depends on what you want to write to the auxiliary file; if you need the expansion of those commands you're out of luck, because they must perform assignments in order to produce their replacement text.

With \protected@write, an input such as

\protected@write\tempfile{}{\optionalexample[not default]{2}{3} or \optionalexample{2}{3}}

would write

text \optionalexample [not default]{2}{3} or \optionalexample {2}{3}

You can find an \immediate version in Writing \\ to a File

If you instead need the full expansion of the commands, you can use xparse:

\usepackage{xparse}
\DeclareExpandableDocumentCommand{\optionalexample}{O{default}mm}{#1:#2(#3)}

and this will work in \immediate\write.

---

You can find some discussion about a macro with optional argument defined with \newcommand at Are commands defined by \newcommand[.][.]{.} robust?

There you can see that your \optionalexample macro becomes, when TeX expands it

\@protected@testopt \optionalexample \\optionalexample {default}

Now the problem starts: when you do \immediate\write, the value of \protect is \@typeset@protect (which in turn means \relax, but it's irrelevant. Here's the definition of \@protected@testopt:

% latex.ltx, line 619:
\def\@protected@testopt#1{%%
  \ifx\protect\@typeset@protect
    \expandafter\@testopt
  \else
    \@x@protect#1%
  \fi}

In this case the conditional returns true, so TeX is presented with

\@testopt \optionalexample \\optionalexample {default}

What does \@testopt do? Here it is:

% latex.ltx, line 617:
\long\def\@testopt#1#2{%
  \kernel@ifnextchar[{#1}{#1[{#2}]}}

and this is the culprit! During a \write only macro expansion is performed, not assignments, so \kernel@ifnextchar cannot work, because it does \futurelet in order to scan the following token.