Expanding control sequences and writing the raw result to a file

Question

I'm attempting to automatically generate a BiBTeX file from within LaTeX. I'd like to expand all control sequences, but disable any active character expansion in the output. The idea is that the following pseudo-code:

\documentclass{article}
\usepackage[utf8]{inputenc}
\begin{document}
  \def\macro{ěščř}
  \output{\jobname.output}{\macro žýáíé}
\end{document}

should generate a file containing ěščřžýáíé in UTF-8.

Plain TeX

I first arrived at this Plain TeX solution:

\documentclass{article}
\usepackage[utf8]{inputenc}
\begin{document}
  \def\macro{ěščř}
  \immediate\newwrite\fd
  \immediate\openout\fd=\jobname.output
  \immediate\write\fd{\macro žýáíé}
  \immediate\closeout\fd
\end{document}

This generated a file containing

\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {e\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 e\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {s\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 s\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {c\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 c\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {r\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 r\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {z\global \mathchardef \accent@spacefactor \spacefactor }\accent 20 z\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {y\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 y\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {a\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 a\egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {\OT1\i \global \mathchardef \accent@spacefactor \spacefactor }\accent 19 \OT1\i \egroup \spacefactor \accent@spacefactor \unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {e\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 e\egroup \spacefactor \accent@spacefactor

Whoa. Inserting \input{\jobname.output} typesets the expected output, so this is clearly an expanded form of the ěščřžýáíé string, but not what I wanted.

The newfile LaTeX package

The \addtostream command

The next thing I tried was the \addtostream command provided by the newfile LaTeX package.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{newfile}
\begin{document}
  \newoutputstream{out}
  \openoutputfile{\jobname.output}{out}
  \def\macro{ěščř}
  \addtostream{out}{\macro žýáíé}
  \closeoutputstream{out}
\end{document}

This generated a file containing

\IeC {\v e}\IeC {\v s}\IeC {\v c}\IeC {\v r}\IeC {\v z}\IeC {\'y}\IeC {\'a}\IeC {\'\i }\IeC {\'e}

Again, not what I wanted.

The writeverbatim environment

Next I tried the writeverbatim environment provided by the package.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{newfile}
\begin{document}
  \newoutputstream{out}
  \openoutputfile{\jobname.output}{out}
  \begin{writeverbatim}{out}ěščřžýáíé\end{writeverbatim}
  \closeoutputstream{out}
\end{document}

This generated a file containing ěščřžýáíé in UTF-8, just as I wanted, but I need control sequence expansion. Any ideas?

Answer 1

You can disable interpretation of UTF-8 characters using the same mechanism that's used for interpreting them, that is redefine \UTF@two@octets and similar macros to produce the string representation of the following characters.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}

\makeatletter
\newcommand{\disable@UTF}{%
  \def\UTFviii@two@octets##1##2{\string##1\string##2}%
  \def\UTFviii@three@octets##1##2##3{\string##1\string##2\string##3}%
  \def\UTFviii@four@octets##1##2##3##4{\string##1\string##2\string##3\string##4}%
}
\newwrite\witiko@out
\immediate\openout\witiko@out=\jobname.dat
\newcommand{\witikowrite}[1]{%
  \protected@write\witiko@out{\disable@UTF}{#1}%
}
\makeatother

\begin{document}

Something for activating \texttt{\string\write}

\witikowrite{\emph{ěščřžýáíé}}

\end{document}

This will write in the .dat file

\emph  {ěščřžýáíé}

Depending on what you need, you might want to add other “neutralizing” assignments in the second argument to \protected@write.

Answer 2

If I use csplain or pdfcsplain (i.e. plain TeX with a little modification) with the source (in UTF8):

\newwrite\fout
\immediate\openout\fout=test.txt
\immediate\write\fout{ěščřžýáíé}
\def\macro{žřů}
\immediate\write\fout{\macro}
\bye

then I get the result in the file test.txt which reads in UTF8:

ěščřžýáíé
žřů

Just simple. The effect is here due to csplain is the format which activates encTeX extension (an extension for UTF8 encoding for pdfTeX).

Answer 3

Have a look at the following (taken from one of my blog posts), I use it to be able to store also \\ in external files to generate address files. From what I can see it does work for your characters as well.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{xpatch}

\makeatletter
% get a copy of `\protected@write
\let\protected@iwrite\protected@write
% patch the copy to add \immediate
\xpatchcmd{\protected@iwrite}{\write}{\immediate\write}{}{}
\makeatother


\newwrite\tempfile
\newcommand{\Anschrift}{ěščř}
\immediate\openout\tempfile=Anschrift.txt
\makeatletter
\protected@iwrite\tempfile{\let\\\relax}{\Anschrift žýáíé}
\immediate\closeout\tempfile
\makeatother

\begin{document}

\input{Anschrift.txt}

\end{document}