3

I am trying to plot a very large .csv file with multiple columns using pgfplots. I want to use the first column for all my plots. My usual routine is to do something like:

\addplot [solid] table [x=A,y=B]{mydata.cvs};
\addplot [solid] table [x=A,y=C]{mydata.cvs};
\addplot [solid] table [x=A,y=nth]{mydata.cvs};

This work fines in combination with LuaLatex (dynamic memory) and it does not take that much time to generate the plot. However, I decided to improve my coding (think that it could also increase the compiling speed) by just reading the table once at the beginning using:

\pgfplotstableread[col sep=semicolon]{mydata.csv}\mytableone

and then:

\addplot [solid] table [x=A,y=B]{\mytableone};
\addplot [solid] table [x=A,y=C]{\mytableone};
\addplot [solid] table [x=A,y=nth]{\mytableone};

After doing this, the process take a LOT more time and sometimes it does not even compile and crash. Can someone explain me why there is a huge difference in compiling speed and what could be causing the crash?

Sorry but I don't have a good example and cannot share my data (anyway is very big).

Thanks =)

P.D. I also want to select a range in the column, for that I use this solution select rows table pgfplots. Not sure if this can cause problem too.

Community
  • 1
Gabriel
  • 209
  • So far no answer between the difference between the two approaches. I decided to keep the first one, to read the table in each \addplot command. I used the \newcommand to store the table path for all the \addplot, e.g. \newcommand{\tableA}{../data/P1/stresses.csv}. – Gabriel Mar 31 '16 at 13:38

1 Answers1

1

Your observation is correct: re-reading the same file multiple times scales considerably better than reusing a "file handle".

The reason for this unexpected behavior is related to how programming in TeX works (or, in this case, does not work): there simply is no (efficient) array data type. The only way to implement array data types is to use the 'normal', global macro space, and that results (resulted) in much more memory consumption and reached the strict limits of TeX easily.

Lua would have efficient array data types, but the pgfplots core for tables has no dedicated Lua representation yet.

Some more details for the difference between re-reading files and reading it once are documented in the manual for key "read completely" (certainly not the best spot).

Henri Menke
  • 109,596
Christian Feuersänger
  • 55,840