allow comments in file read by readarray

Question

I'm using readarray to read files containing key=value pairs, as shown here. I would like to also allow comments in the file read, is it possible to make that work (without resorting to lualatex, I'd like to use plain pdftex)?

Code (with example file contents):

\begin{filecontents*}[overwrite]{setup.txt}
% comment
distance = 60 % test
% another comment
gridRows = 29
markerSide = 4
\end{filecontents*}
\usepackage{readarray}
\readarraysepchar{=}
\readdef{setup.txt}{\fileData}
\readarray\fileData\setupInfo[-,2]
% turn each key into its own command
\newcounter{keyCount}
\setcounter{keyCount}{0}%
\whiledo{\value{keyCount} < \setupInfoROWS}{%
    \stepcounter{keyCount}%
    \typeout{\arabic{keyCount}: \setupInfo[\arabic{keyCount},1] -> \setupInfo[\arabic{keyCount},2]}
    \expandafter\xdef\csname \setupInfo[\arabic{keyCount},1]\endcsname{%
        \setupInfo[\arabic{keyCount},2]}%
}

After discussion with Steven below, i find that comments are already possible if they are on their own line (except for the first line in the file) Or if they are after a key-value pair. Removing the comment on the first line, the above code outputs:

1: distance -> 60
2: gridRows -> 29
3: markerSide -> 4

Can you please make a full compilable example? – egreg Sep 15 '21 at 08:39 — egreg, Sep 15 '21 at 08:39

Steven B. Segletes · Accepted Answer · 2021-09-15T10:20:29.787

The readarray package does honor comments already.

The issue here is not that a comment exists, but that the first record of the file is totally blank (by way of a comment). The package uses the first record to analyze how many fields are found per record (which is useful information for digesting a future array). This is where it gets hung up. If you eliminate the comment on the first record of the file, things work as expected.

However, I should probably have the package anticipate such a case. So I will give you two options: honor blank records, but don't get hung up if initial record is blank, OR ignore blank records.

I am editing my initial answer to provide both those options in one MWE. Here, I introduce a new \ifignoreblankreadarrayrecords, which can be made true or false. The MWE shows both options being used. The redefined \@readdef macro uses comments to highlight where changes were made to the original macro definition.

%dbA
\begin{filecontents*}[overwrite]{databaseA.csv}
% comment
distance = 60 % in cm
markerSide = 4 % in deg
\end{filecontents*}
\documentclass{article}
\usepackage{readarray}[2021-08-08]
\makeatletter
\newif\ifignoreblankreadarrayrecords
\def@readdef#1#2#3{%
  \clear@array{#3}%
  \edef\former@recordcount{\csname #3CELLS\endcsname}%
  \def\first@row{T}%
  \def\first@plane{T}%
  \catcode\endlinechar=\readarrayendlinechar\relax %
  \def#2{}%
  \setcounter{@record}{0}%
  \openin\rdar@file=#1%
  \ifignoreblankreadarrayrecords\def\rdar@iftest{\rdar@fileline\empty}\else
  \def\rdar@iftest{01}\fi
  \loop\unless\ifeof\rdar@file%
    \read\rdar@file to\rdar@fileline % Reads file line into \rdar@fileline%
    \expandafter\ifx\rdar@iftest\else% PERFORM \ifignoreblankreadarrayrecords TEST
      \addtocounter{@record}{1}%
      \expandafter\g@addto@macro\expandafter#2\expandafter{\rdar@fileline}%
      \ifx\rdar@fileline\empty\else\expandafter\g@addto@macro%
        \expandafter#2\expandafter{\read@array@sepchar}%\fi% <---DON'T \fi HERE
        \if T\first@row\read@array{#2}\setcounter{@col}{\numexpr(\Arg@listlen-1)}%
          \edef\ncols{\arabic{@col}}\def\first@row{F}\setcounter{@row}{1}%
        \else%
          \if T\first@plane%
            \ifx\rdar@fileline\empty
              \edef\nrows{\arabic{@row}}\def\first@plane{F}%
            \else
              \addtocounter{@row}{1}%
            \fi
          \fi%
        \fi%
      \fi% <---EXTEND \fi TO HERE TO NOT ANALYZE INITIAL BLANK LINE
      \def\record@name{\csname #3[\the@record]\endcsname}%
      \expandafter\expandafter\expandafter\expandafter\expandafter\expandafter%
        \expandafter\def\expandafter\record@name\expandafter{\rdar@fileline}%
    \fi% FINISH \ifignoreblankreadarrayrecords TEST
  \repeat%
  \edef\nrecords{\arabic{@record}}%
  \expandafter\edef\csname #3PLANES\endcsname{0}%
  \expandafter\edef\csname #3ROWS\endcsname{\nrecords}%
  \expandafter\edef\csname #3COLS\endcsname{0}%
  \expandafter\edef\csname #3CELLS\endcsname{\nrecords}%
  \closein\rdar@file%
  \catcode\endlinechar=5 %
  \define@rootmacro{#3}%
}
\makeatother
\renewcommand\typesetrowsepchar{\}
\renewcommand\typesetcolsepchar{&}
\begin{document}
\readarraysepchar{=}
Ignore blank records: TRUE
\ignoreblankreadarrayrecordstrue
\readdef{databaseA.csv}\dbA
1st array record is ``\ArrayRecord[1]''
2nd array record is ``\ArrayRecord[2]''
\readarray\dbA\arrayA[-,\ncols]
\begin{tabular}{cc}
\hline
\typesetarray\arrayA\
\hline
\end{tabular}
\bigskip
Ignore blank records: FALSE
\ignoreblankreadarrayrecordsfalse
\readdef{databaseA.csv}\dbA
2nd array record is ``\ArrayRecord[2]''
4th array record is ``\ArrayRecord[4]''
\readarray\dbA\arrayA[-,\ncols]
\begin{tabular}{cc}
\hline
\typesetarray\arrayA\
\hline
\end{tabular}
\end{document}

Super! If i understand things correctly, whether ignore blank records is true or false does not matter to me since i cast the data into a 2D array: \readarray\fileData\setupInfo[-,3]. That is, with your patched version, simply calling \readdef{config.txt}{\fileData} and then \readarray\fileData\setupInfo[-,3] would now work if config.txt contains a comment on the first line? — Diederick C. Niehorster, Sep 15 '21 at 12:47
@DiederickC.Niehorster Correct. The "ignore blank records" option only affects record number access to the data. The result of the \readdef should not be affected, other than not choking on a comment that occurs in the first record. — Steven B. Segletes, Sep 15 '21 at 12:59
@DiederickC.Niehorster One point, however...you speak of three columns, but I see only two fields of data. Maybe I misunderstood what you intended for comments...as in LaTeX source code, material after a catcode-14 % token is discarded and will not appear in the \readdef. Is that what you wanted? — Steven B. Segletes, Sep 15 '21 at 13:06
I have provided a more complete code example now. I noted the \readarraysepchar{=} in your code, which means i only need two columns, not three (in the original code i got from the linked question, the = was its own column. Its just the comment on the first line now that makes trouble. Will you release an update to the package? Is there a way to request a specific version of your package (I note the date in \usepackage{readarray}[2021-08-08], but am not quite sure how to use that, where to find the date?)? — Diederick C. Niehorster, Sep 15 '21 at 17:31
oh no, that doesn't work. Now the spaces around the equals sign are not stripped. That means the code creates \ distance as a command (apparently possible?). Is it possible to strip those during the read-in? or should i do so myself? — Diederick C. Niehorster, Sep 15 '21 at 17:34
The latest version of the package is available at https://ctan.org/pkg/readarray. If your version is older than [2021-08-08], a warning will appear in the log file. In all cases, your log file will tell you what version you are using. I do intend to make this (or something close to it) a part of the readarray update. I am, at the moment, in the midst of retirement preparations, and so am a bit busy for a few more weeks. — Steven B. Segletes, Sep 15 '21 at 17:34
@DiederickC.Niehorster You can strip leading/trailing spaces with \readarray* (star version). The manual is also at ctan.org — Steven B. Segletes, Sep 15 '21 at 17:35
Hehe, i read it once, should have consulted it again. works like a charm. Thanks and good luck with the retirement preparations! — Diederick C. Niehorster, Sep 15 '21 at 17:46
@DiederickC.Niehorster You are welcome. Perhaps consider saying thanks with an upvote and/or an accepted answer (the uparrow and/or check mark to the left of my answer). — Steven B. Segletes, Sep 15 '21 at 17:50

allow comments in file read by readarray

1 Answers1