newwrite out of a table with umlauts

Question

I have a follow-up question on https://tex.stackexchange.com/a/598981/46023

Everything works in the MWE there, but when used in my original example it no longer works (note: originaltable.csv is a greatly shortened version of the original, what comparisons, calculations, etc. are made of).

The umlaut 'ä' in the word 'Molybdän', as written in originaltable.csv, is the problem.

\begin{filecontents}[overwrite]{originaltable.csv}
Z; Symbol; Name
2; He; Helium
41; Mo; Molybdän
\end{filecontents}
\documentclass{article}
\usepackage{pgfplotstable}
\pgfplotsset{compat=newest}
\pgfplotstableread[col sep=semicolon, header=true]{originaltable.csv} {\psetable}
% Methode % https://tex.stackexchange.com/a/598981/46023
\makeatletter
\long\def\protected@iwrite#1#2#3{\begingroup%
#2%
\let\protect@unexpandable@protect%
\edef\reserved@a{\immediate\write#1{#3}}%
\reserved@a%
\endgroup%
\if@nobreak\ifvmode\nobreak\fi\fi}
\newcommand{\mywrite}[2]{\protected@iwrite#1{}{#2}}%
\makeatother
\begin{document}
\newwrite\out
\immediate\openout\out=out.csv
\foreach[count=\n from 0] \row in {0,1}{%%
% In:
\pgfplotstablegetelem{\row}{Z}\of{\psetable}
\xdef\Z{\pgfplotsretval}%
\pgfplotstablegetelem{\row}{Symbol}\of{\psetable}%
\xdef\Symbol{\pgfplotsretval}%
\pgfplotstablegetelem{\row}{Name}\of{\psetable}%
\xdef\Name{\pgfplotsretval}%
% Out:
\mywrite\out{\Z; \Symbol; \Name}% does not work :(
}%%
\immediate\closeout\out
%\input{out.csv}
% Target in: 
\pgfplotstableread[col sep=semicolon, header=false]{out.csv}{\mytable}
\pgfplotstabletypeset[string type]{\mytable}
\end{document}

you have used \xdef (again not a latex command so already broken all accent commands before you get to the write. — David Carlisle, May 29 '21 at 06:59
I think newrite in your title is meant to be newwrite. (I can't submit an edit for just one character.) — LSpice, May 29 '21 at 14:39
I don't konow what's the point with def, xdef, let etc.; but I foccused it here: https://tex.stackexchange.com/questions/599122/read-out-and-edit-data-from-a-table-with-def-edef-let-etc-by-using-a-pgf-l — cis, May 29 '21 at 17:21

score 6 · Answer 1 · answered May 29 '21 at 05:30

The problem is how pdfLaTeX deals with ä (and other accented characters). The fast and simple solution: Use XeLaTeX or LuaLaTeX.

Where you're running into trouble is when you do \xdef on \pgfplotsretval. You bypass any expansion protection that might have been available. The simplest workaround for this is to not use \xdef but instead to use \let. I talked a bit about \let vs defining a macro at this answer https://tex.stackexchange.com/a/598902/202780 yesterday. This is one of those cases where \let is exactly the right tool for the job.

If you had wanted to do \edef, the options provided by LaTeX require going outside the realm of user-level commands with either the private command, \protected@edef or the expl3 equivalent, which escapes me at the moment and I can't find in my cursory look through interface3.pdf but I'm sure David Carlisle will jump in and supply it even though in this case \let is the best solution.

There is no expl3 variant of protected@edef, the whole idea of protection and the various meaning of \protect is a LaTeX concept and don't belong into the expl3 programming layer. — Ulrike Fischer, May 29 '21 at 06:56
@UlrikeFischer That would be why I couldn't find it. I do have a vague notion of David pointing me at something else beyond the \protected@*def commands but I couldn't find his comment nor what he talked about (I also checked xparse in case that was what I should have been looking for). ‍♂️ — Don Hosek, May 29 '21 at 16:19

score 2 · Answer 2 · answered May 29 '21 at 17:06

There is no reason for using \xdef (which would break accented characters).

Since the write operation happens inside the loop cycle and it's \immediate, there's no need for global definitions either. You just need to save \pgfplotsretval under a different (local) name.

\begin{filecontents}[overwrite]{\jobname-original.csv}
Z; Symbol; Name
2; He; Helium
41; Mo; Molybdän
\end{filecontents}
\documentclass{article}
\usepackage{pgfplotstable}
\pgfplotsset{compat=newest}
\pgfplotstableread[col sep=semicolon, header=true]{\jobname-original.csv} {\psetable}
% Methode % https://tex.stackexchange.com/a/598981/46023
\makeatletter
\providecommand\protected@iwrite[3]{%
  \begingroup
  #2%
  \let\protect@unexpandable@protect
  \edef\reserved@a{\immediate\write#1{#3}}%
  \reserved@a
  \endgroup
  \if@nobreak\ifvmode\nobreak\fi\fi
}
\newcommand{\mywrite}[2]{\protected@iwrite#1{}{#2}}%
\makeatother
\begin{document}
\newwrite\out
\immediate\openout\out=\jobname.csv
\foreach[count=\n from 0] \row in {0,1}{%%
  % In:
  \pgfplotstablegetelem{\row}{Z}\of{\psetable}%
  \let\Z\pgfplotsretval
  \pgfplotstablegetelem{\row}{Symbol}\of{\psetable}%
  \let\Symbol\pgfplotsretval
  \pgfplotstablegetelem{\row}{Name}\of{\psetable}%
  \let\Name\pgfplotsretval
  % Out:
  \mywrite\out{\Z; \Symbol; \Name}% works!
}
\immediate\closeout\out
%\input{out.csv}
% Target in: 
\pgfplotstableread[col sep=semicolon, header=false]{\jobname.csv}{\mytable}
\pgfplotstabletypeset[string type]{\mytable}
\end{document}

(File names changed to avoid clobbering my files.)

The contents of the written file is

2; He; Helium
41; Mo; Molybdän

as expected.

An expl3 version.

\begin{filecontents}[overwrite]{\jobname-original.csv}
Z; Symbol; Name
2; He; Helium
41; Mo; Molybdän
\end{filecontents}
\documentclass{article}
\usepackage{pgfplotstable}
\pgfplotsset{compat=newest}
\pgfplotstableread[col sep=semicolon, header=true]{\jobname-original.csv} {\psetable}
\ExplSyntaxOn
\NewDocumentCommand{\startwritingfile}{mm}
 {% #1 = output stream, #2 = file name
  \iow_new:N #1
  \iow_open:Nn #1 { #2 }
 }
\NewDocumentCommand{\finishwritingfile}{m}
 {% #1 = output stream
  \iow_close:N #1
 }
\NewDocumentCommand{\mywrite}{mm}
 {% #1 = output stream, #2 = tokens to write
  \iow_now:Nx #1 { \text_expand:n { #2 } }
 }
\ExplSyntaxOff
\begin{document}
\startwritingfile{\out}{\jobname.csv}
\foreach[count=\n from 0] \row in {0,1}{%%
  % In:
  \pgfplotstablegetelem{\row}{Z}\of{\psetable}%
  \let\Z\pgfplotsretval
  \pgfplotstablegetelem{\row}{Symbol}\of{\psetable}%
  \let\Symbol\pgfplotsretval
  \pgfplotstablegetelem{\row}{Name}\of{\psetable}%
  \let\Name\pgfplotsretval
  % Out:
  \mywrite\out{\Z; \Symbol; \Name}% works!
}
\finishwritingfile{\out}
%\input{out.csv}
% Target in: 
\pgfplotstableread[col sep=semicolon, header=false]{\jobname.csv}{\mytable}
\pgfplotstabletypeset[string type]{\mytable}
\end{document}

score 0 · Answer 3 · answered May 29 '21 at 17:29

As pointed out in some comments: LuaLaTeX let it work without any special method.

% arara: lualatex
\begin{filecontents}[overwrite]{originaltable.csv}
Z; Symbol; Name
2; He; Helium
41; Mo; Molybdän
\end{filecontents}
\documentclass{article}
\usepackage{pgfplotstable}
\pgfplotsset{compat=newest}
\pgfplotstableread[col sep=semicolon, header=true]{originaltable.csv} {\psetable}
\begin{document}
\newwrite\out
\immediate\openout\out=out.csv
\foreach[count=\n from 0] \row in {0,1}{%%
% In:
\pgfplotstablegetelem{\row}{Z}\of{\psetable}
\xdef\Z{\pgfplotsretval}%
\pgfplotstablegetelem{\row}{Symbol}\of{\psetable}%
\xdef\Symbol{\pgfplotsretval}%
\pgfplotstablegetelem{\row}{Name}\of{\psetable}%
\xdef\Name{\pgfplotsretval}%
% Out:
\immediate\write\out{\Z; \Symbol; \Name}% does not work :(
}%%
\immediate\closeout\out
%\input{out.csv}
% Target out: 
\pgfplotstableread[col sep=semicolon, header=false]{out.csv}{\mytable}
\pgfplotstabletypeset[string type]{\mytable}
\end{document}

newwrite out of a table with umlauts

3 Answers3

Linked