3

I am trying to output some information to an auxiliary file using \write and among other things I have authors names that often include accented letters. If I use the string explicitly, the accented letters are expanded into TeX commands, but I can solve this by using \unexpanded. However, if the string is contained in a variable, and I use \unexpanded, I get the name of the variable instead of its content. The command \unexpanded works fine again if the variable is a command parameter, for example #1.

I don't have an understanding of expansion that is clear enough to sort this out, so any help is appreciated. All I want to do is to write the content of a variable holding a string with accented letters to the text file.

Here is a minimal example:

\documentclass{article}
\usepackage[utf8]{inputenc}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}
\immediate\write\file{Test 1}                 % works fine
\immediate\write\file{José}                   % expands into TeX code
\immediate\write\file{\unexpanded{José}}      % works fine
\def\mystring{José}
\immediate\write\file{\mystring}              % expands into TeX code
\immediate\write\file{\unexpanded{\mystring}} % outputs \mystring literally
\newcommand{\writeaccented}[1]{
  \immediate\write\file{\unexpanded{#1}}
}
\writeaccented{José}                          % works fine
\end{document}

EDIT: I received two answers to use either \expandafter\unexpanded\expandafter{\mystring} or \detokenize\expandafter{\mystring}. These work fine on the minimal example above, but not on the real example, making me think that the expansion in my case is happening earlier. Here is the new minimal example that is closer to my real case:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{xstring}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}
\def\listofauthors{José Saramago, Luigi Pirandello and Władysław Reymont}
\makeatletter
\StrSubstitute[0]{\listofauthors}{, }{,}[\@cpauthors]
\StrSubstitute[0]{\@cpauthors}{ and }{,}[\@cpauthors]
\@for \@cpauthor:=\@cpauthors\do{%
  \immediate\write\file{<author>\expandafter\unexpanded\expandafter{\@cpauthor}</author>}
}
\makeatother
\end{document}

EDIT 2: adding \usepackage[T1]{fontenc} prevents the expansion into TeX commands, but outputs accented letters in ISO-8859 encoding instead of unicode. I tried both with and without the proposed solutions.

EDIT 3: the solution that worked for me is in https://tex.stackexchange.com/a/390119/36823. Thank you!

Giampiero Salvi
  • 31

3 Answers3

4

You can use \detokenize:

\documentclass{article}
\usepackage[utf8]{inputenc}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}

\def\mystring{José}

\immediate\write\file{\detokenize\expandafter{\mystring}} % outputs \mystring literally

\end{document}

The contents of the _extra file is

José

The hex dump:

0000000 4a 6f 73 c3 a9 0a                              
0000006

The UTF-8 representation of é is indeed C3 A9.

A different approach for the more complicated situation:

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{xparse}

\ExplSyntaxOn
\iow_new:N \g_giampiero_output_stream
\tl_new:N \l__giampiero_write_temp_tl


\AtBeginDocument
 {
   \iow_open:Nn \g_giampiero_output_stream { \c_sys_jobname_str _extra.txt }
 }
\AtEndDocument
 {
  \iow_close:N \g_giampiero_output_stream
 }

\NewDocumentCommand{\writeauthors}{sm}
 {% *-version for doing a variable
  \IfBooleanTF{#1}
   {
    \giampiero_writeauthors:V #2
   }
   {
    \giampiero_writeauthors:n { #2 }
   }
 }

\cs_new_protected:Nn \giampiero_writeauthors:n
 {
  \tl_set:Nn \l__giampiero_write_temp_tl { #1 }
  \tl_replace_all:Nnn \l__giampiero_write_temp_tl { ~and~ } { , }
  \seq_set_split:NnV \l__giampiero_write_temp_seq { , } \l__giampiero_write_temp_tl
  \seq_map_inline:Nn \l__giampiero_write_temp_seq
   {
    \giampiero_write_generic:nn { author } { ##1 }
   }
 }
% This can be used for other similar write operations
\cs_new_protected:Nn \giampiero_write_generic:nn
 {
  \iow_now:Nn \g_giampiero_output_stream { <#1>#2</#1> }
 }

\cs_generate_variant:Nn \seq_set_split:Nnn {NnV}
\cs_generate_variant:Nn \giampiero_writeauthors:n { V }
\ExplSyntaxOff

\newcommand\listofauthors{José Saramago, Luigi Pirandello and Władysław Reymont}

\begin{document}

\writeauthors{José Saramago, Luigi Pirandello and Władysław Reymont}

\writeauthors*{\listofauthors}

\end{document}

Here is the contents of the generated file:

<author>José Saramago</author>
<author>Luigi Pirandello</author>
<author>Władysław Reymont</author>
<author>José Saramago</author>
<author>Luigi Pirandello</author>
<author>Władysław Reymont</author>
egreg
  • 1,121,712
3

Expansion is a weird beast in TeX ;-)

\immediate\write\file{\expandafter\unexpanded\expandafter{\mystring}} %

will expand the content of \mystring first before \unexpanded can 'protect' it again. 

\documentclass{article}
\usepackage[utf8]{inputenc}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}
\immediate\write\file{Test 1}                 % works fine
\immediate\write\file{José}                   % expands into TeX code
\immediate\write\file{\unexpanded{José}}      % works fine
\def\mystring{José}
\immediate\write\file{\mystring}              % expands into TeX code
\immediate\write\file{\expandafter\unexpanded\expandafter{\mystring}} % outputs \mystring literally
\newcommand{\writeaccented}[1]{
  \immediate\write\file{\unexpanded{#1}}
}
\writeaccented{José}                          % works fine
\end{document}

Output: (%%%% annotation by me)

Test 1
Jos\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {e\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 e\egroup \spacefactor \accent@spacefactor 
José
Jos\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {e\global \mathchardef \accent@spacefactor \spacefactor }\accent 19 e\egroup \spacefactor \accent@spacefactor 
José%%%%% This one is from \expandafter\unexpanded\expandafter{\mystring}
José
  • \expandafter\unexpanded\expandafter can be shortened to \unexpanded\expandafter (i.e. the first one is not needed). Same with \detokenize. –  Sep 06 '17 at 17:22
2

Answer to example with xstring

Package xstring knows several expansion modes, see section "3.1 Expansion of arguments" of the documentation.

In this case, \listofauthors should contain the plain author names as given in the example. Then, mode \expandarg just expands the first token, macro \listofauthors, but without further expansion.

The example limits the effect of \expandarg by a group. It uses \detokenize and removes a unnecessary \expandafter.

\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{xstring}

\newwrite\file
\immediate\openout\file=\jobname_extra.txt
\AtEndDocument{\closeout\file}

\begin{document}
\def\listofauthors{José Saramago, Luigi Pirandello and Władysław Reymont}
\makeatletter
\begingroup
  \expandarg
  \StrSubstitute[0]{\listofauthors}{, }{,}[\@cpauthors]
  \StrSubstitute[0]{\@cpauthors}{ and }{,}[\@cpauthors]
  \@for \@cpauthor:=\@cpauthors\do{%
    \immediate\write\file{<author>\detokenize\expandafter{\@cpauthor}</author>}
  }
\endgroup
\makeatother
\end{document}

File \jobname_extra.txt:

<author>José Saramago</author>
<author>Luigi Pirandello</author>
<author>Władysław Reymont</author>
Heiko Oberdiek
  • 271,626